Multi-sensor sensing data processing method and system for multi-sensory immersive experience cabin

Abstract

This application discloses a multi-sensor sensing data processing method and system for a multi-sensor immersive experience cabin, relating to the field of multi-sensor immersive experience cabin technology. The multi-sensor sensing data processing method for the multi-sensor immersive experience cabin includes: collecting operational data from various sensors and special effects devices within the multi-sensor immersive experience cabin and performing structured processing; optimizing deployment schemes and collaborative communication protocols to construct a multi-source sensing and special effects collaborative network; collecting multimodal raw data at a dynamic acquisition frequency and performing time-series calibration and dynamic error correction to generate fused data; processing the fused data into a standardized training sample set and inputting it into a feature learning model to establish a feature-level mapping relationship between user interaction actions and special effects device responses; and linking interactive responses with safety monitoring. This application, through precise collaboration between multiple sensors and special effects devices, effectively improves the coherence and accuracy of the multi-sensor immersive experience while ensuring safe equipment operation.

Description

Technical Field

[0001] This application relates to the field of multi-sensory immersive experience cabin technology, and in particular to a multi-sensory immersive experience cabin multi-sensory sensing data processing method and system. Background Technology

[0002] In the field of multi-sensory immersive experience cabins, to create an immersive interactive experience, it is usually necessary to equip them with multiple types of sensors, such as vision, posture, and touch sensors, as well as various special effects devices for motion and environment. The virtual scene and the cabin's movements are linked through the coordinated response of multiple devices. However, existing technologies have the following problems: (1) The deployment of multiple sensors and special effects devices lacks data support, which can easily lead to conflicts such as overlapping signal frequency bands and vibration interference. Furthermore, the collaborative communication protocol is not dynamically optimized in conjunction with the working conditions, resulting in uneven data transmission delay and low synchronization accuracy, which affects the continuity of interaction.

[0003] (2) The original multimodal data has temporal offset, systematic error and random error, lacks a targeted dynamic correction mechanism, and the data fusion accuracy is insufficient, which leads to inaccurate matching of the mapping relationship between user interaction actions and special effects device responses.

[0004] (3) The sensor acquisition frequency is fixed and is not dynamically adjusted according to the working conditions, load and error scenarios, resulting in data redundancy or insufficient accuracy. Furthermore, safety monitoring and interactive response are disconnected, and emergency response is mostly fixed shutdown or throttling, making it difficult to balance equipment safety and the integrity of the immersive experience.

[0005] Existing technologies have serious limitations on the interaction accuracy, operational stability, and user experience of multi-sensory immersive experience cabins. Therefore, there is an urgent need for an efficient and accurate multi-sensor sensing data processing solution to solve one or more of the above problems. Summary of the Invention

[0006] The purpose of this application is to provide a multi-sensor sensing data processing method and system for a multi-sensory immersive experience cabin. Through the precise coordination of multiple sensors and special effects devices, the coherence and accuracy of the multi-sensory immersive experience can be effectively improved while ensuring the safe operation of the equipment.

[0007] To achieve the above objectives, this application provides a multi-sensor sensing data processing method for a multi-sensor immersive experience cabin, comprising the following steps: S1: Collecting operational data from each device in the multi-sensor immersive experience cabin, and performing structured processing on the operational data to obtain structured data; wherein, each device in the multi-sensor immersive experience cabin includes: various sensors and various special effects devices; the structured data is a structured database table with multi-dimensional labels, including at least: device ID, data type, data value, acquisition timestamp, operating condition label, associated error label, and acquisition location information; S2: Optimizing the deployment scheme and collaborative communication protocol of sensors and special effects devices based on the structured data to obtain a multi-source sensing and special effects collaborative network; S3: Based on the multi-source sensing and special effects collaborative network, various sensors collect multimodal raw data according to a dynamically set acquisition frequency, and perform time-series calibration on the multimodal raw data according to the acquisition timestamp of the structured data to obtain calibrated data; S4: Based on the multi-source sensing and special effects collaborative network... The network calls the error sample set corresponding to the working condition label and associated error label in the structured data to dynamically correct the error of the calibrated data, obtain fused data, and synchronize it to all nodes of the multi-source sensing and special effects collaborative network; where the all nodes include various sensors and various special effects devices; S5: Based on the multi-source sensing and special effects collaborative network, the fused data is processed into a standardized training sample set and then input into a pre-set feature learning model for training to establish a feature-level mapping relationship between user interaction actions and special effects device responses; S6: Based on the feature-level mapping relationship between user interaction actions and special effects device responses called by the multi-source sensing and special effects collaborative network, mapping instructions for virtual scenes and cabin actions are pushed to drive various sensors and special effects devices to respond collaboratively to create an immersive experience; at the same time, the environment and equipment operation status of the multi-sensory immersive experience cabin are monitored in real time to obtain monitoring data, and emergency measures are taken for devices whose monitoring data exceeds the safety threshold to realize the linkage between interactive response and safety monitoring.

[0008] As described above, the sub-step of step S2 is as follows: S21: Retrieve structured data, filter records with the associated error labels being signal interference error and / or data drift error, and extract the corresponding record's device ID, operating condition label, data value, and acquisition location information to construct an information association matrix. Quantitative identification of conflicts; when Then the determination device Under working conditions Error type exists below Corresponding deployment conflicts include: overlapping sensor frequency bands and areas affected by vibration interference from special effects devices; when Then the determination device Under working conditions No error type exists below The corresponding deployment conflicts; where the expression for the information association matrix is: ; in, The devices are numbered, and the devices include various sensors and special effects devices; This refers to the operating condition number, which includes no-load, half-load, and full-load operating conditions. S22: Based on structured data, optimize the deployment scheme of each device using partition isolation and distance threshold optimization strategies to form an optimized deployment scheme; S23: Retrieve the acquisition timestamps from the structured data as a unified synchronization benchmark for the communication protocol, optimize the time synchronization mechanism and transmission priority rules of the original protocol to form an optimized collaborative communication protocol; S24: Complete the hardware installation of the sensors and special effects devices according to the optimized deployment scheme, and load the optimized collaborative communication protocol to complete the current networking of the sensors and special effects devices; S25: Utilize the error labels associated with no error in the structured data... The operating data is used to test the current network. If the signal interference rate between sensors in the current network is less than or equal to a preset first interference threshold, the signal interference rate between sensors and special effects devices in the current network is less than or equal to a preset second interference threshold, the response synchronization accuracy between sensors in the current network is greater than or equal to a preset first response accuracy threshold, and the response synchronization accuracy between sensors and special effects devices in the current network is greater than or equal to a preset second response accuracy threshold, then the current network is determined to be a multi-source sensing and special effects collaborative network. Otherwise, the multi-source sensing and special effects collaborative network is re-acquired.

[0009] As above, the sub-step of step S23 is: S231: Set the protocol clock synchronization period. Collection period obtained based on collection timestamps of structured data Consistent; S232: Calculate the transmission priority coefficient of each device under different operating conditions, and allocate the optimized cooperative communication protocol to the corresponding device according to the transmission priority coefficient; wherein, the higher the transmission priority coefficient, the lower the data transmission delay of the optimized cooperative communication protocol of the corresponding device.

[0010] As shown above, the expression for the transmission priority coefficient is: ; in, For device Under working conditions The transmission priority coefficient below; For device Device type weight; For device The importance weight of the device; For working conditions The weight of the working condition level; For working conditions The operating load weight.

[0011] As described above, the sub-step of allocating an optimized cooperative communication protocol to the corresponding device based on the transmission priority coefficient is as follows: S2321: Traverse the preset communication protocol mapping table based on the transmission priority coefficient, wherein the preset communication protocol mapping table includes: multiple standard transmission priorities, each standard transmission priority corresponding to a priority coefficient range, and at least one standard cooperative communication protocol; S2322: Take the standard transmission priority corresponding to the priority coefficient range to which the transmission priority coefficient belongs as the current priority, and take one of the standard cooperative communication protocols corresponding to the current priority as the optimized cooperative communication protocol.

[0012] As described above, the sub-steps of step S3 are as follows: S31: Based on the operating condition labels and associated error labels in the structured data synchronized by the multi-source sensing and special effects collaborative network, the acquisition frequencies of various sensors in the multi-source sensing and special effects collaborative network are dynamically configured to obtain dynamically set acquisition frequencies; S32: Various sensors acquire multimodal raw data according to the dynamically set acquisition frequencies; S33: Using the acquisition timestamps in the structured data synchronized by the multi-source sensing and special effects collaborative network as a unified benchmark, a timestamp alignment algorithm is used to perform time-series calibration on the multimodal raw data aggregated in the multi-source sensing and special effects collaborative network to eliminate time offset and obtain calibrated data.

[0013] As mentioned above, the rules for dynamically configuring the acquisition frequencies of various sensors within the multi-source sensing and special effects collaborative network are as follows: If the matching condition label is full-load condition or the associated error label is high-error scenario: the acquisition frequencies of visual sensors, attitude sensors, and tactile sensors within the multi-source sensing and special effects collaborative network are increased to the first acquisition frequency, while environmental sensors remain at the second acquisition frequency; if the matching condition label is no-load / half-load condition, or the associated error label is low-error scenario / no-error scenario: the acquisition frequencies of all sensors within the multi-source sensing and special effects collaborative network are uniformly configured to the third acquisition frequency; where the third acquisition frequency < the second acquisition frequency < the first acquisition frequency.

[0014] As described above, the sub-steps of step S4 are as follows: S41: Based on the structured data synchronized by the multi-source sensing and special effects collaborative network, extract the operating condition features and error features of the calibrated data, match the operating condition labels and associated error labels in the structured data, and retrieve the error sample set corresponding to the operating condition label and associated error label from the preset error database; wherein, the preset error database includes: multiple types of sensors, one type of sensor corresponds to multiple label combinations, each label combination includes an operating condition label and an associated error label; each label combination corresponds to an error sample set; S42: Use the error sample set as a correction benchmark, select according to sensor type Select the appropriate error correction algorithm to dynamically correct the errors in the calibrated data and obtain fused data. Specifically, for visual sensors, posture sensors, and tactile sensors, the error compensation method is used to correct the data offset error of the calibrated data based on the compensation value in the error sample set, and then the fitting correction method is used to correct the nonlinear error based on the fitting model in the error sample set, thereby obtaining fused data. For environmental sensors, the error compensation method is used to correct the basic error of the calibrated data based on the temperature drift / zero drift parameters in the error sample set, thereby obtaining fused data. S43: Synchronize the fused data to all nodes of the multi-source sensing and special effects collaborative network.

[0015] As described above, the sub-steps of step S6 are as follows: S61: Based on the feature-level mapping relationship between user interaction actions and special effects device responses, the collected user interaction actions are matched with the quantified user interaction action features in the feature-level mapping relationship. Based on the correspondence between the matched quantified user interaction action features and the quantified special effects device response features, a mapping instruction adapted to the interaction requirements of the current virtual scene is generated, and the mapping instruction is transmitted to the corresponding sensors and special effects devices. The mapping instruction includes the collected action parameters of various sensors and the execution action parameters of various special effects devices. S62: Various sensors and special effects devices synchronously execute collaborative actions according to the mapping instruction, realizing multi-sensory interactive linkage between the current virtual scene and the multi-sensory immersive experience cabin, creating an immersive experience. S63: During the multi-sensory interactive linkage process, monitoring data of the multi-sensory immersive experience cabin is continuously collected. The monitoring data includes: real-time environmental parameters inside the cabin, Real-time operating data of various sensors and real-time operating parameters of various special effects devices; synchronously retrieve pre-stored structured equipment operating condition safety thresholds and cabin environment safety thresholds in the multi-source sensing and special effects collaborative network; S64: Use the cabin environment safety threshold to perform safety threshold comparison analysis on the real-time environmental parameters inside the cabin, and use the structured equipment operating condition safety threshold to perform safety threshold comparison analysis on the real-time operating data of various sensors and the real-time operating parameters of various special effects devices. If the real-time environmental parameters inside the cabin, the real-time operating data of various sensors, and the real-time operating parameters of various special effects devices do not exceed the corresponding safety thresholds, then maintain the normal collaborative response of various sensors and various special effects devices; if one or more of the real-time environmental parameters inside the cabin, the real-time operating data of various sensors, and the real-time operating parameters of various special effects devices exceed the corresponding safety thresholds, then generate comparison results and immediately trigger the emergency response mechanism to carry out emergency response for objects whose monitored data exceed the safety thresholds.

[0016] This application also provides a multi-sensor sensing data processing system for a multi-sensor immersive experience cabin, including: a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it can control various sensors and special effects devices of the multi-sensor immersive experience cabin to realize the above-mentioned multi-sensor sensing data processing method for the multi-sensor immersive experience cabin. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1A schematic diagram of the structure of a multi-sensor sensing data processing system for a multi-sensory immersive experience cabin; Figure 2 A flowchart of one embodiment of a multi-sensor sensing data processing method for a multi-sensory immersive experience cabin. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 As shown, this application provides a multi-sensor sensing data processing system for a multi-sensor immersive experience cabin, including: a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it can control various sensors and special effects devices of the multi-sensor immersive experience cabin to realize the following multi-sensor sensing data processing method for the multi-sensor immersive experience cabin.

[0021] like Figure 2 As shown, this application provides a multi-sensor sensing data processing method for a multi-sensory immersive experience cabin, including the following steps: S1: Collect operational data from each device in the multi-sensory immersive experience cabin, perform structured processing on the operational data, and obtain structured data; among which, each device in the multi-sensory immersive experience cabin includes: various sensors and special effects devices; the structured data is a structured database table with multi-dimensional labels, which includes at least: device ID, data type, data value, collection timestamp, operating condition label, associated error label, and collection location information.

[0022] Specifically, the operational data includes: parameters, error messages, and log information of various sensors during operation, as well as parameters, error messages, and log information of various special effects devices during operation.

[0023] Furthermore, the sub-steps of step S1 are as follows: S11: Continuously collect operational data of each device in the multi-sensory immersive experience cabin under various typical operating conditions; among which, the various typical operating conditions include at least: no-load operating condition, half-load operating condition and full-load operating condition.

[0024] Specifically, the types of devices in the multi-sensory immersive experience cabin can be flexibly set according to actual needs. Among them, the various sensors include at least visual sensors (e.g., high-definition cameras, infrared depth cameras), attitude sensors (e.g., three-axis gyroscopes, accelerometers, angle sensors), environmental sensors (e.g., pressure sensors, temperature and humidity sensors, gas sensors), and tactile sensors (e.g., pressure sensors, vibration sensors); the special effects devices include at least dynamic special effects devices (e.g., six-degree-of-freedom platforms, horizontal rotating bases) and environmental special effects devices (e.g., water mist generators, odor releasers, sound effects equipment, and touch-sensitive actuators with spatial positioning).

[0025] Unloaded Condition: No users are inside the cabin. All sensors and special effects devices are in standby or low-power operation to collect baseline operating parameters and background error data when the equipment is unloaded. Half-Loaded Condition: The cabin carries 50% of the rated passenger capacity (or 50% of the rated load weight). The equipment operates at medium power to simulate collaborative operation data under normal usage scenarios. Full-Loaded Condition: The cabin carries the rated passenger capacity or rated load weight. All special effects devices (such as the six-degree-of-freedom platform, water mist generator, and touch-sensitive actuators with spatial positioning) operate at full power to simulate operating parameters and potential malfunctions under high-load extreme scenarios.

[0026] S12: Clean the running data, remove outliers and duplicates, and then use linear interpolation to fill in the missing data to obtain standardized raw data.

[0027] Specifically, step S12 can be achieved using existing commercial data analysis software (such as MATLAB data processing toolbox, SPSS statistical analysis software, and OriginPro data analysis software, all of which support graphical interface operation and can complete outlier removal, duplicate value removal, and linear interpolation completion of running data to obtain standardized raw data), industrial-grade hardware acquisition modules (such as Advantech DAQ series data acquisition units and Siemens S7-1200 PLC built-in data preprocessing firmware, which can complete data cleaning of running data in real time and directly output standardized raw data), or standardized algorithm frameworks (such as Scikit-learn data preprocessing module and OpenCV data cleaning toolset), but it is not limited to commercial data analysis software, industrial-grade hardware acquisition modules, and standardized algorithm frameworks.

[0028] S13: Based on the three-level naming rules, the standardized raw data is classified and labeled, and a millisecond-level collection timestamp is added to each data point in the standardized raw data. At the same time, error information and error type labels under the corresponding working conditions are bound to complete the structured processing and obtain structured data. The three-level naming rules include: equipment type, data attribute and working condition label.

[0029] Specifically, step S13 can be executed using commercial database tools (such as MySQL Workbench, Navicat Premium, and SQLiteStudio, which support graphical interfaces for creating data tables, batch importing data, and adding label fields), industrial-grade data annotation platforms (such as LabelStudio, LabelImg, and VGG Image Annotator (VIA), which allow for custom label templates, batch data annotation, and export to structured formats (such as CSV, JSON, and SQL)), and general office software (such as Microsoft Excel and WPS Spreadsheet, which can perform structured processing through custom column fields, data filtering, and batch filling functions, suitable for small-batch data scenarios). Therefore, it will not be elaborated further.

[0030] S2: Optimize the deployment scheme and collaborative communication protocol of sensors and special effects devices based on structured data to obtain a multi-source sensing and special effects collaborative network.

[0031] Furthermore, the sub-steps of step S2 are as follows: S21: Retrieve structured data, filter records with the associated error labels of signal interference error and / or data drift error, and extract the device ID, operating condition label, data value, and acquisition location information of the corresponding records to construct an information association matrix. Quantitative identification of conflicts; when Then the determination device Under working conditions Error type exists below Corresponding deployment conflicts include: overlapping sensor frequency bands and areas affected by vibration interference from special effects devices; when Then the determination device Under working conditions No error type exists below The corresponding deployment conflict.

[0032] The expression for the information association matrix is ​​as follows: ; in, The devices are numbered, and the devices include various sensors and special effects devices; This refers to the operating condition number, which includes no-load, half-load, and full-load operating conditions. This is the error type number, which includes signal interference and data drift.

[0033] S22: Based on structured data, the deployment scheme of each device is optimized by adopting partitioning isolation and distance threshold optimization strategies to form an optimized deployment scheme.

[0034] Furthermore, the sub-step of step S22 is as follows: For sensors with overlapping signal frequency bands, divide them into independent physical zones according to the frequency band differentiation principle, and arrange sensors of different frequency bands separately. The installation spacing of sensors of the same frequency band in the same zone shall not be less than the minimum safe spacing without error records in the structured data. To physically avoid signal crosstalk; for sensors in areas affected by vibrations from special effects devices, elastic shock-absorbing components are added, and the installation distance between the sensors and the special effects devices is adjusted to... , To create redundant spacing for vibration attenuation, an optimized deployment scheme is formed.

[0035] Specifically, The value is determined based on the fitting curve of vibration parameters and errors in the structured data.

[0036] S23: Retrieve the collection timestamp from the structured data as a unified synchronization benchmark for the communication protocol, optimize the time synchronization mechanism and transmission priority rules of the original protocol, and form an optimized collaborative communication protocol.

[0037] Furthermore, the sub-steps of step S23 are as follows: S231: Set the protocol clock synchronization period Collection period obtained based on collection timestamps of structured data Consistent.

[0038] Specifically, let This ensures that the timing of command transmission and reception and data acquisition of all devices is aligned.

[0039] S232: Calculate the transmission priority coefficient of each device under different operating conditions, and assign an optimized cooperative communication protocol to the corresponding device according to the transmission priority coefficient; wherein, the higher the transmission priority coefficient, the lower the data transmission delay of the corresponding device's optimized cooperative communication protocol.

[0040] The expression for the transmission priority coefficient is as follows: ; in, For device Under working conditions The transmission priority coefficient below; For device Device type weight; For device The importance weight of the device; For working conditions The weight of the working condition level; For working conditions The operating load weight.

[0041] Specifically, The specific value should be flexibly set according to the actual scene and specific equipment. This application prefers: special effects device. All values ​​are taken as 0.7, for the sensor. All values ​​are taken as 0.3. The specific value should be flexibly set according to the actual scenario and specific working conditions. This application prefers: the value under full load conditions. Take 0.8, for half-load conditions. Take 0.5, for no-load conditions. Take 0.3.

[0042] The device importance weight mapping is directly matched based on a preset device importance weight mapping table. This table is pre-configured in the collaborative communication protocol and can be updated in real time based on the structured data in step S1. The specific content of the device importance weight mapping table is flexibly set according to the actual scenario. For example, based on the core importance of the device to the immersive experience, core interactive sensors such as motion effects devices and motion capture / pressure sensors are matched with a weight of 0.8~1.0, while environmental auxiliary sensors such as temperature, humidity, and brightness are matched with a weight of 0.3~0.5. Normalize to the [0,1] interval.

[0043] The load weight mapping table is directly matched based on a preset load condition weight mapping table. This table is pre-configured in the collaborative communication protocol and can be updated in real time based on the structured data from step S1. The specific content of the load weight mapping table can be flexibly set according to the actual scenario. For example, it can be directly matched according to no-load, half-load, and full-load conditions, with full-load conditions matching 1.0, half-load conditions matching 0.5~0.8, no-load conditions matching 0.1~0.4, and so on. Normalized to the [0,1] interval, the matching values ​​corresponding to the operating conditions / half-load operating conditions / full-load operating conditions are comprehensively calibrated by the equipment operating load rate and data transmission volume in the structured data in step S1.

[0044] Furthermore, the sub-step of allocating optimized cooperative communication protocols to the corresponding devices based on the transmission priority coefficient is as follows: S2321: Traverse the preset communication protocol mapping table based on the transmission priority coefficient, wherein the preset communication protocol mapping table includes: multiple standard transmission priorities, each standard transmission priority corresponding to a priority coefficient range, and at least one standard cooperative communication protocol.

[0045] Specifically, the number of standard transmission priority levels in the preset communication protocol mapping table, the specific values ​​of the priority coefficient range corresponding to each standard transmission priority, and the specific content of the standard cooperative communication protocol can be flexibly set according to actual needs. For example, the preset communication protocol mapping table includes three standard transmission priorities: high priority, medium priority, and low priority. The priority coefficient range for high priority is [1.0, 1.5], for medium priority is [0.5, 1.0), and for low priority is [0, 0.5]. The standard cooperative communication protocol corresponding to high priority is a TSN (Time-Sensitive Networking) based communication protocol, configured with a deterministic transmission delay of less than 5ms and a maximum bandwidth reservation priority; the standard cooperative communication protocol corresponding to medium priority is an MQTT-SN based communication protocol, configured with a transmission delay of less than 10ms and a medium bandwidth reservation; the standard cooperative communication protocol corresponding to low priority is a CoAP based communication protocol, configured with a transmission delay of less than 50ms and a basic bandwidth allocation.

[0046] S2322: Take the standard transmission priority corresponding to the priority coefficient range to which the transmission priority coefficient belongs as the current priority, and take one of the standard cooperative communication protocols corresponding to the current priority as the optimized cooperative communication protocol.

[0047] Furthermore, the preset communication protocol mapping table can be updated in real time based on the structured data in step S1 to adapt to the dynamic operating status of the multi-sensory immersive experience cabin.

[0048] S24: Complete the hardware installation of sensors and special effects devices according to the optimized deployment plan, and load the optimized collaborative communication protocol to complete the current networking of sensors and special effects devices.

[0049] S25: Test the current network using the working condition data with no error associated with the error label in the structured data. If the signal interference rate between sensors in the current network is less than or equal to the preset first interference threshold, the signal interference rate between sensors and special effects devices in the current network is less than or equal to the preset second interference threshold, the response synchronization accuracy between sensors in the current network is greater than or equal to the preset first response accuracy threshold, and the response synchronization accuracy between sensors and special effects devices in the current network is greater than or equal to the preset second response accuracy threshold, then the current network is determined to be a multi-source sensing and special effects collaborative network. Otherwise, re-acquire the multi-source sensing and special effects collaborative network.

[0050] Specifically, the values ​​of the first interference threshold, the second interference threshold, the first response accuracy threshold, and the second response accuracy threshold can be flexibly set according to the actual scenario.

[0051] S3: Based on a multi-source sensing and special effects collaborative network, various sensors collect multimodal raw data according to a dynamically set acquisition frequency, and perform time-series calibration on the multimodal raw data based on the acquisition timestamp of the structured data to obtain calibrated data.

[0052] Furthermore, the sub-steps of step S3 are as follows: S31: Based on the operating condition labels and associated error labels in the structured data synchronized by the multi-source sensing and special effects collaborative network, the acquisition frequency of various sensors in the multi-source sensing and special effects collaborative network is dynamically configured in a differentiated manner to obtain the dynamically set acquisition frequency.

[0053] Furthermore, the rules for dynamically configuring the acquisition frequencies of various sensors within the multi-source sensing and special effects collaborative network are as follows: If the matching condition label is full-load condition or the associated error label is high-error scenario: the acquisition frequencies of visual sensors, attitude sensors, and tactile sensors within the multi-source sensing and special effects collaborative network are increased to the first acquisition frequency, while environmental sensors remain at the second acquisition frequency; if the matching condition label is no-load / half-load condition, or the associated error label is low-error scenario / no-error scenario: the acquisition frequencies of all sensors within the multi-source sensing and special effects collaborative network are uniformly configured to the third acquisition frequency; wherein, the third acquisition frequency < the second acquisition frequency < the first acquisition frequency.

[0054] Specifically, by dynamically configuring the acquisition frequency according to the differences of various sensors, it is possible to balance data acquisition accuracy, device power consumption, and network transmission efficiency. The specific values ​​of the third, second, and first acquisition frequencies can be flexibly set according to the actual scenario. For example, the third acquisition frequency is 10Hz, the second acquisition frequency is 20Hz, and the first acquisition frequency is 60Hz.

[0055] S32: Various sensors acquire multimodal raw data according to dynamically set acquisition frequencies.

[0056] S33: Using the acquisition timestamps in the structured data synchronized by the multi-source sensing and special effects collaborative network as a unified benchmark, a timestamp alignment algorithm is used to perform time-series calibration on the multimodal raw data aggregated in the multi-source sensing and special effects collaborative network to eliminate time offset and obtain calibrated data.

[0057] Specifically, timestamp alignment algorithms can be implemented using existing technologies, so they will not be elaborated further, such as linear interpolation or bias correction.

[0058] S4: Based on the multi-source sensing and special effects collaborative network, the error sample set corresponding to the working condition label and associated error label in the structured data is called to perform dynamic error correction on the calibrated data, obtain fused data, and synchronize it to all nodes of the multi-source sensing and special effects collaborative network; among which, the all nodes include various types of sensors and various types of special effects devices.

[0059] Specifically, the sub-steps of step S4 are as follows: S41: Based on the structured data synchronized by multi-source sensing and special effects collaborative network, extract the operating condition features and error features of the calibrated data, match the operating condition labels and associated error labels in the structured data, and retrieve the error sample set corresponding to the operating condition label and associated error label from the preset error database; wherein, the preset error database includes: multiple types of sensors, one type of sensor corresponds to multiple label combinations, each label combination includes an operating condition label and an associated error label; each label combination corresponds to an error sample set.

[0060] Specifically, the content of the error sample set is flexibly set according to the actual scenario. This application preferably includes the following: when the operating condition label is full-load operating condition and the associated error label is high-error scenario, the error sample set includes motion blur error sample set of visual sensors, drift error sample set of attitude sensors, pressure offset error sample set of tactile sensors, and temperature drift error sample set of environmental sensors; when the operating condition label is no-load operating condition and the associated error label is low-error scenario, the error sample set includes noise error sample set of visual sensors, slight jitter error sample set of attitude sensors, contact error sample set of tactile sensors, and zero-drift error sample set of environmental sensors; when the operating condition label is half-load operating condition and the associated error label is error-free scenario, the error sample set includes basic calibration error sample set of various sensors.

[0061] S42: Using the error sample set as a correction benchmark, select the corresponding error correction algorithm according to the sensor type, and perform dynamic error correction on the calibrated data to obtain fused data. Specifically, for vision sensors, posture sensors, and tactile sensors, the error compensation method is used to correct the data offset error of the calibrated data based on the compensation value in the error sample set, and then the fitting correction method is used to correct the nonlinear error based on the fitting model in the error sample set to obtain fused data. For environmental sensors, the error compensation method is used to correct the basic error of the calibrated data based on the temperature drift / zero drift parameters in the error sample set to obtain fused data.

[0062] Specifically, in step S42, different label combinations correspond to different correction benchmarks and correction weights, and the algorithm parameters are not fixed. The obtained fused data is a spatiotemporally consistent, error-controllable structured multimodal data, which retains the original multimodal features and eliminates the systematic errors, random errors and scene adaptability errors of various sensors.

[0063] Furthermore, the specific type of error correction algorithm can be flexibly selected according to the actual scenario, and is not limited to error compensation and fitting correction methods; Kalman filtering can also be used. Error compensation, fitting correction, and Kalman filtering are all existing technologies, and therefore will not be elaborated upon further.

[0064] S43: Synchronize the fused data to all nodes of the multi-source sensing and special effects collaborative network.

[0065] Specifically, step S43 achieves global data unification, providing accurate data support for the subsequent collaborative operation of the multi-source sensing and special effects collaborative network.

[0066] S5: Based on a multi-source sensing and special effects collaborative network, the fused data is processed into a standardized training sample set and then input into a pre-set feature learning model for training, establishing a feature-level mapping relationship between user interaction actions and special effects device responses.

[0067] Furthermore, the sub-steps of step S5 are as follows: S51: Retrieve fusion data from all nodes of the multi-source sensing and special effects collaborative network, extract user interaction action features and special effects device response features based on the fusion data, complete the association labeling of user interaction action features and special effects device response features according to sensor type, and form a standardized training sample set.

[0068] Specifically, existing feature extraction methods can be used to extract user interaction features and special effects device response features from fused data based on sensor type. For example, visual sensors collect image / video data, and existing methods such as CNN convolutional layers, HOG, and SIFT can be used to extract visual interaction features such as user limb movements and gaze changes as user interaction features. Posture sensors collect temporal motion data, and existing methods such as LSTM, joint coordinate extraction, and temporal feature difference can be used to extract temporal motion features such as user limb posture changes and motion trajectories as user interaction features. Tactile sensors collect pressure / contact numerical data, and existing methods such as statistical feature extraction (e.g., mean, extreme values, rate of change) and threshold segmentation can be used to extract tactile interaction features such as user touch and pressing as user interaction features. Special effects device response features correspond to equipment operating parameter data, and existing numerical feature extraction methods can be used to extract response features such as device start / stop, power, and operating angle as special effects device response features.

[0069] Existing data label matching methods or feature association annotation methods can be used to achieve the association annotation of user interaction action features and special effect device response features according to sensor type, but it is not limited to existing data label matching methods or feature association annotation methods.

[0070] S52: The standardized training sample set is divided into three branches according to sensor type: vision, posture and touch. The samples are then input into the sub-models adapted to the pre-set feature learning model. The sub-model corresponding to the standardized training sample set of vision sensors is a CNN sub-model, while the sub-models corresponding to posture sensors and touch sensors are both LSTM sub-models.

[0071] Specifically, both the CNN (Convolutional Neural Network) sub-model and the LSTM (Long Short-Term Memory Network) sub-model can use existing feature learning models in the field of artificial intelligence, so they will not be elaborated further.

[0072] S53: Each sub-model independently pre-trains and reinforces the corresponding standardized training sample set of the branch input. Each sub-model uses mini-batch gradient descent as the training method, mean squared error (MSE) as the loss function, and goodness of fit ≥ preset goodness of fit threshold as the convergence criterion. Iteratively optimizes its own weight parameters through backpropagation until each sub-model learns the feature-level correspondence between user interaction action features and special effect device response under the corresponding sensor type, and obtains the sub-model after training convergence.

[0073] Specifically, the preset fit threshold values ​​for each sub-model can be flexibly set according to the actual scenario. In this application, it is preferred that the preset fit threshold for each sub-model is 95%.

[0074] S54: Based on the sub-model output after training convergence, the feature-level quantization correspondence dataset is structured and organized according to sensor type, constructing a feature-level mapping relationship between user interaction actions and special effects device responses; among them, the feature-level mapping relationship is divided into three feature mapping subsets according to sensor type: vision, posture, and touch. Each feature mapping subset stores a one-to-one correspondence between the quantized features of user interaction actions and the quantized features of special effects device responses under the corresponding sensor type.

[0075] S6: Based on the feature-level mapping relationship between user interaction actions and special effects device responses called by multi-source sensing and special effects collaborative network, it pushes mapping instructions between virtual scenes and cabin actions, driving various sensors and special effects devices to respond collaboratively to create an immersive experience; at the same time, it monitors the environment and equipment operation status of the multi-sensory immersive experience cabin in real time, obtains monitoring data, and takes emergency measures for devices whose monitoring data exceeds the safety threshold, realizing the linkage between interactive response and safety monitoring.

[0076] Furthermore, the sub-steps of step S6 are as follows: S61: Based on the feature-level mapping relationship between user interaction actions and special effects device responses, the collected user interaction actions are matched with the quantized user interaction action features in the feature-level mapping relationship. According to the correspondence between the matched user interaction action quantized features and the special effects device response quantized features, a mapping instruction adapted to the interaction requirements of the current virtual scene is generated, and the mapping instruction is transmitted to the corresponding sensors and special effects devices. The mapping instruction includes the action parameters collected by various sensors and the execution action parameters of various special effects devices.

[0077] Specifically, as an example, the execution parameters of the special effects device include at least: the positioning accuracy parameters and transmission response speed parameters of the touch transmission device with spatial positioning.

[0078] S62: Various sensors and special effects devices execute coordinated actions synchronously according to the mapping instructions, realizing multi-sensory interaction and linkage between the current virtual scene and the multi-sensory immersive experience cabin, creating an immersive experience.

[0079] S63: During the multi-sensory interactive linkage process, continuously collect monitoring data of the multi-sensory immersive experience cabin. The monitoring data includes: real-time environmental parameters inside the cabin, real-time operating data of various sensors, and real-time operating parameters of various special effects devices; and simultaneously retrieve the pre-stored structured equipment operating condition safety thresholds and cabin environment safety thresholds in the multi-source sensing and special effects collaborative network.

[0080] Specifically, the specific monitoring items for real-time environmental parameters inside the cabin are set according to the actual situation, such as temperature, humidity, and air pressure. The specific monitoring items for real-time operating data of various sensors and real-time operating parameters of special effects devices are set according to the actual situation, such as the power supply and operating current of sensors, the operating power, operating angle, and operating current of special effects devices, and the operating power, positioning error, and transmission mechanism temperature of touch transmission devices with spatial positioning.

[0081] S64: Utilize cabin environment safety thresholds to perform safety threshold comparison analysis on real-time cabin environmental parameters, and utilize structured equipment operating condition safety thresholds to perform safety threshold comparison analysis on real-time operating data of various sensors and real-time operating parameters of various special effects devices. If the real-time cabin environmental parameters, real-time operating data of various sensors, and real-time operating parameters of various special effects devices do not exceed the corresponding safety thresholds, then maintain the normal coordinated response of various sensors and special effects devices; if one or more of the real-time cabin environmental parameters, real-time operating data of various sensors, and real-time operating parameters of various special effects devices exceed the corresponding safety thresholds, then generate comparison results and immediately trigger the emergency response mechanism to carry out emergency response for objects whose monitored data exceed the safety thresholds.

[0082] Specifically, the safety thresholds for structured equipment include safety thresholds for various sensors and special effects devices. The cabin environment safety thresholds include the safety threshold for each monitored item in the real-time cabin environmental parameters. The specific values ​​for each safety threshold are set according to the actual situation.

[0083] Furthermore, the sub-steps for triggering the emergency response mechanism and taking emergency measures for objects whose monitoring data exceeds the safety threshold are as follows: S651: Determine the handling type for objects whose monitoring data exceeds the safety threshold based on the comparison results. The handling types include: cabin environment, sensor, and special effects device.

[0084] S652: If the treatment type is a special effects device, calculate the target safe power of the special effects device. If the target safe power is less than or equal to the maximum operating power of the special effects device and greater than or equal to the minimum operating power of the special effects device, then adjust the special effects device to the target safe power and continue to operate. If the target safe power is less than the minimum operating power of the object or higher than the maximum operating power of the special effects device, then directly suspend the special effects device.

[0085] Furthermore, the expression for the target safe power of the special effects device is: ; in, The target safe power for special effects devices; When the limit is exceeded, the actual operating power in the real-time operating parameters of the special effects device; This is the power regulation coefficient; When the value exceeds the standard, the real-time monitoring parameter value in the real-time operating parameters of the special effects device; The safety threshold corresponding to the special effects device; This represents the sensor type matching coefficient.

[0086] Specifically, The specific value is set comprehensively based on the equipment performance, load characteristics and immersive experience requirements of the special effects device. In this application, it is preferably 0.2 to 0.8. Used to quantify the power linkage compatibility between different types of sensors and special effects devices. The specific values ​​are set based on the experience weighting and response sensitivity of different types of sensors corresponding to special effects devices. This application prefers: visual sensors. For 0.9, attitude sensors For 1.0, tactile sensors The value is 1.

[0087] This application achieves dynamic and quantitative power adjustment by using the target safe power of the special effects device. While ensuring the safe operation of the special effects device, it preserves the integrity of the multi-sensory immersive experience to the greatest extent and improves the accuracy and feasibility of emergency response.

[0088] S653: If the disposal type is sensor type, then directly cut off the power supply to the sensor.

[0089] S654: If the handling type is cabin environment, the corresponding environmental control device will be triggered to make corrections until the monitoring data does not exceed the corresponding safety threshold.

[0090] Specifically, the type of environmental control device is selected based on the actual application scenario of the immersive experience cabin, such as air conditioning, fresh air system, dehumidifier, humidifier, air pressure regulator and air purification device.

[0091] Furthermore, step S6 also includes S66: when the monitoring data of the object exceeding the standard returns to the corresponding safety threshold range, the emergency response state is lifted, the mapping instructions for virtual scenes and cabin actions are pushed to the corresponding devices, and various sensors and special effects devices return to normal multi-sensory interactive linkage state, completing the full-process linkage of this interactive response and safety monitoring.

[0092] The beneficial effects achieved by this application are as follows: (1) The multi-sensor sensing data processing method and system of the multi-sensory immersive experience cabin of this application effectively solves the problems of signal interference and synchronization deviation by optimizing the deployment scheme and collaborative communication protocol of each device, and realizes the precise collaboration of multiple sensors and special effects devices.

[0093] (2) The multi-sensor sensing data processing method and system of the multi-sensory immersive experience cabin of this application improves the accuracy of multimodal data fusion by dynamically adapting the acquisition frequency and correcting data errors, so that the matching between user interaction actions and special effects response is more accurate.

[0094] (3) The multi-sensor sensing data processing method and system of the multi-sensory immersive experience cabin of this application can realize the linkage of safety monitoring and interactive response. Through differentiated emergency handling, it can ensure the safe operation of the equipment without destroying the integrity of the immersive experience.

[0095] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the scope of protection of this application is intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application. Obviously, those skilled in the art can make various alterations and variations to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of protection of this application and its equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for processing multi-sensor sensing data from a multi-sensory immersive experience cabin, characterized in that, Includes the following steps: S1: Collect operational data from each device in the multi-sensory immersive experience cabin, perform structured processing on the operational data, and obtain structured data; among which, each device in the multi-sensory immersive experience cabin includes: various sensors and various special effects devices; the structured data is a structured database table with multi-dimensional labels, which includes at least: device ID, data type, data value, collection timestamp, operating condition label, associated error label, and collection location information; S2: Optimize the deployment scheme and collaborative communication protocol of sensors and special effects devices based on structured data to obtain a multi-source sensing and special effects collaborative network; S3: Based on a multi-source sensing and special effects collaborative network, various sensors collect multimodal raw data according to a dynamically set acquisition frequency, and perform time-series calibration on the multimodal raw data according to the acquisition timestamp of the structured data to obtain calibrated data; S4: Based on the multi-source sensing and special effects collaborative network, the error sample set corresponding to the working condition label and associated error label in the structured data is called to perform dynamic error correction on the calibrated data, obtain fused data, and synchronize it to all nodes of the multi-source sensing and special effects collaborative network; where the all nodes include various types of sensors and various types of special effects devices; S5: Based on the multi-source sensing and special effects collaborative network, the fused data is processed into a standardized training sample set and then input into a pre-set feature learning model for training to establish a feature-level mapping relationship between user interaction actions and special effects device responses. S6: Based on the feature-level mapping relationship between user interaction actions and special effects device responses called by multi-source sensing and special effects collaborative network, it pushes mapping instructions between virtual scenes and cabin actions, driving various sensors and special effects devices to respond collaboratively to create an immersive experience; at the same time, it monitors the environment and equipment operation status of the multi-sensory immersive experience cabin in real time, obtains monitoring data, and takes emergency measures for devices whose monitoring data exceeds the safety threshold, realizing the linkage between interactive response and safety monitoring.

2. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 1, characterized in that, The sub-steps of step S2 are: S21: Retrieve structured data, filter records with the associated error labels of signal interference error and / or data drift error, and extract the device ID, operating condition label, data value, and acquisition location information of the corresponding records to construct an information association matrix. Quantitative identification of conflicts; when Then the determination device Under working conditions Error type exists below Corresponding deployment conflicts include: overlapping sensor frequency bands and areas affected by vibration interference from special effects devices; when Then the determination device Under working conditions No error type exists below Corresponding deployment conflicts; The expression for the information association matrix is ​​as follows: ； in, The devices are numbered, and the devices include various sensors and special effects devices; This refers to the operating condition number, which includes no-load, half-load, and full-load operating conditions. This is a number representing the error type, which includes signal interference and data drift. S22: Based on structured data, the deployment scheme of each device is optimized by adopting partitioning isolation and distance threshold optimization strategies to form an optimized deployment scheme; S23: Retrieve the collection timestamp from the structured data as a unified synchronization benchmark for the communication protocol, optimize the time synchronization mechanism and transmission priority rules of the original protocol, and form an optimized collaborative communication protocol; S24: Complete the hardware installation of sensors and special effects devices according to the optimized deployment plan, and load the optimized collaborative communication protocol to complete the current networking of sensors and special effects devices; S25: Test the current network using the working condition data with no error associated with the error label in the structured data. If the signal interference rate between sensors in the current network is less than or equal to the preset first interference threshold, the signal interference rate between sensors and special effects devices in the current network is less than or equal to the preset second interference threshold, the response synchronization accuracy between sensors in the current network is greater than or equal to the preset first response accuracy threshold, and the response synchronization accuracy between sensors and special effects devices in the current network is greater than or equal to the preset second response accuracy threshold, then the current network is determined to be a multi-source sensing and special effects collaborative network. Otherwise, re-acquire the multi-source sensing and special effects collaborative network.

3. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 2, characterized in that, The sub-steps of step S23 are as follows: S231: Set the protocol clock synchronization period Collection period obtained based on collection timestamps of structured data Consistent; S232: Calculate the transmission priority coefficient of each device under different operating conditions, and assign an optimized cooperative communication protocol to the corresponding device according to the transmission priority coefficient; wherein, the higher the transmission priority coefficient, the lower the data transmission delay of the corresponding device's optimized cooperative communication protocol.

4. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 3, characterized in that, The expression for the transmission priority coefficient is: ； in, For device Under working conditions The transmission priority coefficient below; For device Device type weight; For device The importance weight of the device; For working conditions The weight of the working condition level; For working conditions The operating load weight.

5. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 3, characterized in that, The sub-steps for allocating optimized cooperative communication protocols to the corresponding devices based on transmission priority coefficients are as follows: S2321: Traverse the preset communication protocol mapping table based on the transmission priority coefficient, wherein the preset communication protocol mapping table includes: multiple standard transmission priorities, each standard transmission priority corresponds to a priority coefficient range, and at least one standard cooperative communication protocol. S2322: Take the standard transmission priority corresponding to the priority coefficient range to which the transmission priority coefficient belongs as the current priority, and take one of the standard cooperative communication protocols corresponding to the current priority as the optimized cooperative communication protocol.

6. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 1, characterized in that, The sub-steps of step S3 are as follows: S31: Based on the working condition labels and associated error labels in the structured data synchronized by the multi-source sensing and special effects collaborative network, the acquisition frequency of various sensors in the multi-source sensing and special effects collaborative network is dynamically configured in a differentiated manner to obtain the dynamically set acquisition frequency. S32: Various sensors acquire multimodal raw data according to dynamically set acquisition frequencies; S33: Using the acquisition timestamps in the structured data synchronized by the multi-source sensing and special effects collaborative network as a unified benchmark, a timestamp alignment algorithm is used to perform time-series calibration on the multimodal raw data aggregated in the multi-source sensing and special effects collaborative network to eliminate time offset and obtain calibrated data.

7. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 6, characterized in that, The rules for dynamically configuring the acquisition frequencies of various sensors within the multi-source sensing and special effects collaborative network are as follows: If the matching condition label is full-load condition or the associated error label is high-error scenario: the acquisition frequencies of visual sensors, attitude sensors, and tactile sensors within the multi-source sensing and special effects collaborative network are increased to the first acquisition frequency, while environmental sensors remain at the second acquisition frequency; if the matching condition label is no-load / half-load condition, or the associated error label is low-error scenario / no-error scenario: the acquisition frequencies of all sensors within the multi-source sensing and special effects collaborative network are uniformly configured to the third acquisition frequency; wherein, the third acquisition frequency < the second acquisition frequency < the first acquisition frequency.

8. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 1, characterized in that, The sub-steps of step S4 are as follows: S41: Based on the structured data synchronized by multi-source sensing and special effects collaborative network, extract the operating condition features and error features of the calibrated data, match the operating condition labels and associated error labels in the structured data, and retrieve the error sample set corresponding to the operating condition label and associated error label from the preset error database; wherein, the preset error database includes: multiple types of sensors, one type of sensor corresponds to multiple label combinations, each label combination includes an operating condition label and an associated error label; each label combination corresponds to an error sample set; S42: Using the error sample set as a correction benchmark, select the corresponding error correction algorithm according to the sensor type, and perform dynamic error correction on the calibrated data to obtain fused data. Specifically, for vision sensors, posture sensors, and tactile sensors, the error compensation method is used to correct the data offset error of the calibrated data based on the compensation value in the error sample set, and then the fitting correction method is used to correct the nonlinear error based on the fitting model in the error sample set to obtain fused data. For environmental sensors, the error compensation method is used to correct the basic error of the calibrated data based on the temperature drift / zero drift parameters in the error sample set to obtain fused data. S43: Synchronize the fused data to all nodes of the multi-source sensing and special effects collaborative network.

9. The multi-sensor sensing data processing method for the multi-sensory immersive experience cabin according to claim 1, characterized in that, The sub-steps of step S6 are as follows: S61: Based on the feature-level mapping relationship between user interaction actions and special effects device responses, the collected user interaction actions are matched with the quantized user interaction action features in the feature-level mapping relationship. According to the correspondence between the matched user interaction action quantized features and the special effects device response quantized features, a mapping instruction adapted to the interaction requirements of the current virtual scene is generated, and the mapping instruction is transmitted to the corresponding sensors and special effects devices. The mapping instruction includes the action parameters collected by various sensors and the execution action parameters of various special effects devices. S62: Various sensors and special effects devices execute coordinated actions synchronously according to the mapping instructions, realizing multi-sensory interaction and linkage between the current virtual scene and the multi-sensory immersive experience cabin, creating an immersive experience; S63: During the multi-sensory interactive linkage process, continuously collect monitoring data of the multi-sensory immersive experience cabin. The monitoring data includes: real-time environmental parameters inside the cabin, real-time operating data of various sensors, and real-time operating parameters of various special effects devices; and simultaneously retrieve the pre-stored structured equipment operating condition safety thresholds and cabin environment safety thresholds in the multi-source sensing and special effects collaborative network. S64: Utilize cabin environment safety thresholds to perform safety threshold comparison analysis on real-time cabin environmental parameters, and utilize structured equipment operating condition safety thresholds to perform safety threshold comparison analysis on real-time operating data of various sensors and real-time operating parameters of various special effects devices. If the real-time cabin environmental parameters, real-time operating data of various sensors, and real-time operating parameters of various special effects devices do not exceed the corresponding safety thresholds, then maintain the normal coordinated response of various sensors and special effects devices; if one or more of the real-time cabin environmental parameters, real-time operating data of various sensors, and real-time operating parameters of various special effects devices exceed the corresponding safety thresholds, then generate comparison results and immediately trigger the emergency response mechanism to carry out emergency response for objects whose monitored data exceed the safety thresholds.

10. A multi-sensor sensing data processing system for a multi-sensory immersive experience cabin, characterized in that, include: The system includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it can control various sensors and special effects devices of the multi-sensory immersive experience cabin to realize the multi-sensory sensing data processing method of the multi-sensory immersive experience cabin as described in any one of claims 1-9.

Images