Immersive interactive sand pool system based on wireless radio frequency and visual interaction

Abstract

The invention discloses an immersive interactive sand pool system based on wireless radio frequency and visual interaction, and particularly relates to the field of electric digital interactive processing, which comprises a server, a projection module, sound equipment, a wireless radio frequency communication protocol device and a man-machine interaction module, and the server is used as a central controller of the system, outputs a picture display signal to the projection module through an HDMI interface, is connected with the sound equipment through an audio cable to output an audio signal, and sends a WIFI and EV1527 coding wireless communication signal and a machine switch signal to the wireless radio frequency communication protocol device. By performing electrical digital data processing on multi-user behavior information and location data and detecting individual low-interaction signs during group interaction, avoidance users are identified and contextual guidance is output to help them restore normal social behavior.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital interactive processing, and more specifically, to an immersive interactive sand pool system based on wireless radio frequency and visual interaction. Background Art

[0002] At present, most interactive sand pool systems use projection equipment or simple sensors to obtain basic location information of users, and realize entertainment scenes through conventional screen switching and audio output. However, in general sand pool interactions, the system often only recognizes static or single actions, lacks a perception and feedback mechanism for deeper behavioral associations, and is difficult to fully adapt to the complex needs of various groups of people. The existing technology has a defect when applied to group scenarios, namely, there is a lack of effective identification and flexible auxiliary measures for users in a low-interaction or avoidance state, which makes it difficult for them to obtain a normal interactive experience in a multi-person interactive environment, and also makes the overall interactive effect easily disturbed by such negative factors. Summary of the invention

[0003] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an immersive interactive sand pool system based on wireless radio frequency and visual interaction. By constructing a behavior synchronization observation and deviation prompt mechanism, the behavior trajectories of multiple users are dynamically analyzed and deviations are identified, so as to actively identify low-interaction users during group interaction and generate contextual guidance feedback, thereby solving the shortcomings of the existing system in adapting to complex crowd behaviors and maintaining interaction quality.

[0004] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: an immersive interactive sand pool system based on wireless radio frequency and visual interaction, comprising a server, a projection module, an audio device, a wireless radio frequency communication protocol device, and a human-computer interaction module; The server is used as the central controller of the system. The server outputs the screen display signal to the projection module through the HDMI interface. The server connects the audio equipment through the audio cable to output the audio signal. At the same time, it sends WIFI and EV1527 coded wireless communication signals and machine switch signals to the wireless radio frequency communication protocol device. Features: The human-computer interaction module is used to connect the sensing camera and infrared sensor through the USB interface to obtain the user's behavior information and location data, and analyze the original sensing signal through electronic digital data processing, and output the positioning signal and audio signal to the server for triggering the game content; The hardware interaction module is used to receive interaction control instructions from the server; The hardware interaction module includes connecting the lighting interaction module through the DMX512 protocol to realize the lighting scene response control; the hardware interaction module is connected to the joystick interaction module through the USB interface to realize the physical operation signal input and interactive control; The human-computer interaction module also includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electronic digital data processing operations on user behavior information and location data, and constructs a dual behavior modeling path based on multi-user behavior trajectories. It drives the model switching process through behavior deviation identification to generate situational guidance feedback for abnormal users.

[0005] In a preferred embodiment, the projection module includes a wall projection, a floor projection 1 and a floor projection 2, and the projection module is used to receive a picture display signal from a server to display image content, wherein the floor projection 1 and the floor projection 2 receive a machine switch signal from a wireless radio frequency communication protocol device through an RS232 serial port; The wireless radio frequency communication protocol device is used to receive WIFI and EV1527 encoded wireless communication signals from the server, and output the machine switch signals therein to the floor projection 1 and the floor projection 2 through the RS232 serial port; The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and provide feedback on the execution of the lighting effects; the joystick interaction module is connected to the hardware interaction module through USB to receive user input instructions for controlling the game interaction response; It also includes a game software module, which is used to receive the positioning signal output by the human-computer interaction module and the interaction signal fed back by the hardware interaction module, and trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

[0006] In a preferred embodiment, the behavior synchronization observation and deviation prompting mechanism includes a behavior acquisition stage, a synchronization comparison stage, a model construction stage, a model switching stage, and a guidance feedback stage; In the behavior collection phase, the user's spatial movement vector sequence in each frame is extracted from the sensing camera, and a behavior trajectory vector function is constructed to describe the position data; ; in For users In time The moving trajectory vector function, in units of ; For users collected by the sensor camera In frame time The image intensity value function of , the unit is dimensionless; Represents the image in two dimensions Gradient extraction is performed on ; is the frame time integral unit, in units of ; The infrared sensor array is used to extract the heat source density change function of the user's body parts per unit time: ; in Indicates user In time The heat source density fluctuation energy, in units of , describing the behavior information through the energy of heat source density fluctuation; For infrared sensor The sensing points at time The temperature flux on ; For the The weight of each sensing point; is the total number of sensing points in the infrared sensor array; is the time difference window, in units of ; The behavior trajectory vector function is combined with the heat source density change function to construct the user's full-dimensional behavior state vector: ; in For users In time The behavior state vector is in units of ; is the instantaneous modulus of the user's moving path, in units of ; It is the scalar representation of the user's mobile acceleration, in units of .

[0007] In a preferred embodiment, in the synchronous comparison phase, a user behavior state map is constructed, and the user behavior state map collected in the collection phase is converted into a user behavior state map. Constructed as a behavior graph structure: ; in Indicates at time The generated behavior graph structure is in the form of a "state relationship graph"; Represents a user set, no unit; is the edge set; for A collection of, the unit is state strength / second, ; Based on the user behavior state graph, a Laplace spectral kernel propagation model is constructed to capture the multi-order graph diffusion effect of synchronous behavior; the Laplace spectral kernel propagation model is expressed as: ;in For users exist The synchronization eigenvector under order neighborhood propagation, the unit is state strength / second; For the The order propagation weight is a dimensionless coefficient; is the graph Laplacian matrix, unitless; Next, a local synchronization stability deviation score is generated to determine whether there is structural deviation or synchronization break. Based on this, a local Laplace graph stability index is constructed: ; in For users The synchronization stability deviation score of , in (state strength / second)²; For users of The set of neighbors of order, unitless; Indicates user A user in the neighborhood user set of ; For users exist Synchronous eigenvectors under order neighborhood propagation; Perform time series convolution matching on the behavior propagation direction, construct frequency domain propagation difference index, and measure the difference of cross-user propagation mode through frequency domain transformation: ; in is the frequency domain propagation difference index, the unit is (state intensity)²; is Fourier transform; represents the frequency variable after Fourier transformation; is the average response of all neighboring node spectra, in units of state intensity; , Indicates the upper and lower limits of the frequency interval of the integral; Indicates user In the channel The original behavioral state signal sequence on ; Fusion synchronization structure deviation and frequency domain error, output the final synchronization deviation factor as the next stage model switching criterion, comprehensive and , output synchronization anomaly score: ; in Synchronize anomaly scores for users; Combining weights for structural deviation and frequency domain differences; is the Sigmoid normalization function.

[0008] In a preferred embodiment, in the model building stage, the behavior state vector set outputted in the acquisition stage or the synchronous comparison stage is used. Synchronous anomaly score collection , respectively construct normal behavior response models , build an abnormal behavior response model , and generate a difference map , the final output behavior response judgment index ; The user's behavior state vector Map to feature subspace and cluster encode to build normal behavior response model : ; exist In the formula, the normal behavior feature map is constructed through nonlinear projection and cluster alignment to capture the stable mode of most users; in represents the feature projection weight matrix; is the dimension of the original behavior state vector, The feature dimension of the target mapping space; represents the bias vector; is the Swish activation function; represents the cluster center; Represents a set of model parameters; is the regularization weight term; Select the synchronization deviation function Greater than threshold of users, extract disturbance behaviors and build abnormal behavior response models : ; in is the synchronization deviation function, and the calculation formula is: ; in: is the total amount of spatial deviation, unit: m; is the behavioral synchronization variance, unit: ; Perturbation spread function, unit: dimensionless; Time standardization operations; is the perturbation feature extraction function; Represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is a sparse decoding network; in the abnormal behavior response model is the synchronization abnormality threshold, unit: m; By calculating the distribution deviation and state topological distance between models, the overall difference between normal and abnormal models at the probability and path layers is quantified to assist in determining the degree of deviation and generate a difference map. : ; in represents JensenShannon divergence; is the normal behavior state transition diagram, It is the current user behavior state transition diagram; ; is the trajectory distance function; By comparing the cosine similarity between the behavior state vector and the two models, the behavior response judgment index is generated. : ; in Represents the cosine similarity function.

[0009] In a preferred embodiment, in the model building stage, according to The fluctuation degree of the switching perception weight function is constructed : ; in Indicates user At the moment Behavior change vector; Neighborhood users At the moment Behavior change vector; represents the two-norm measure of the behavior fluctuation energy, Indicates user At the moment The squared norm of the behavior change vector; For users The set of neighboring users refers to the participating users in the same interaction scenario in practical applications; It is a sensitive factor for behavioral differences; is the stability offset buffer constant; is the Sigmoid function; by Construct a probability distribution function for the input , used to determine whether to trigger the model switching process: ; in represents the probability of model switching decision; To determine the intensity control factor, the unit is dimensionless; Constructing a dual model selection operator for behavioral model selection , the behavior trajectories are sent to two types of models respectively and State prediction is performed in: ; in Indicates the length is A window into historical behavior; It is a standard behavior model; It is an intervention-type compensation model; represents the model switching probability threshold; Finally, the model tag status of the current user at the current moment is output , used for calling in the boot feedback phase: .

[0010] In a preferred embodiment, in the guidance feedback stage, according to and the current , construct the feedback allocation function , determine the type and intensity of guided feedback: ; Its purpose is to achieve a differentiated situational guidance mechanism by generating feedback intensity and content adjustment methods through different functions between normal behavior states and deviant behavior states; is a vector consisting of feedback type and feedback intensity; is the feedback intensity coefficient; represents the feedback regulation function constructed based on the output of the standard behavior model; It represents the feedback construction function based on the intervention compensation model state combined with the abnormal probability; Feedback distribution function Deconstructed into a multi-channel context-guided instruction set , and allocate execution resources to the output channel: ; In the formula The audio feedback strength required by the audio device. Output visual feedback brightness / color level for projection module; To guide the task trigger intensity; Assign a matrix to the feedback; based on Construct time scene linkage control function , used to determine the subsequent situational intervention process: ; It builds a multi-level plot guidance plan based on the feedback instruction intensity and time interval to achieve dynamic intervention and adjustment in the time dimension; in Indicates the time interval Internal users The generated situation content control instructions are in plot identifiers; The number of preset scenario plots; is the indicator function; The trigger weight for each situation module; For the The effective time period of each situation module, unit: seconds; For the Situational plot control instructions; According to the output , sending control instructions to the corresponding projection module or audio equipment through the server; ; in For the server at time The set of control instructions issued; Represents a function used to generate visual feedback instructions. Represents a function used to generate feedback commands for a sound device.

[0011] Technical effects and advantages of the present invention: By performing electronic digital data processing on multi-user behavior information and location data and detecting individual low interaction signs during group interaction, avoidant users are identified and situational guidance is output to help them restore normal social behavior; With the help of "behavior synchronization observation and deviation prompt mechanism", the system realizes deep aggregation analysis of multi-source information such as location data and action paths, which can not only capture multi-user collaboration patterns, but also detect individual abnormal deviations; Combining the cross-collection approach of infrared sensors and cameras, this system tracks the user's action activity in multiple dimensions, making the subsequent judgment results relatively more accurate and improving the adaptability to complex group environments; Through the linkage management of wireless radio frequency communication protocol devices and servers, each projector and interactive device can switch independently and automatically adjust the display content to provide differentiated services to normal interactive users and low interactive individuals; A phased model switching process is adopted to instantly enter the deviation handling strategy after capturing abnormal signals, providing gentle prompts and auxiliary scenarios for users who show avoidance tendencies, thereby steadily improving the overall interaction quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a system module diagram of the present invention.

[0013] Figure 2It is a system architecture diagram of the present invention.

[0014] Figure 3 This is a physical diagram of the wireless radio frequency communication protocol device of the present invention.

[0015] Figure 4 This is a physical picture of the "plate" interactive hardware in the joystick interaction module of the present invention.

[0016] Figure 5 This is a physical picture of the "joystick wave-making" interactive hardware in the joystick interaction module of the present invention.

[0017] Figure 6 This is a physical picture of the sensing camera and infrared sensor in the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] Refer to the instruction manual Figure 1-6 , an immersive interactive sand pool system based on wireless radio frequency and visual interaction according to an embodiment of the present invention includes a server, a projection module, an audio device, a wireless radio frequency communication protocol device, and a human-computer interaction module; The server is used as the central controller of the system. The server outputs the screen display signal to the projection module through the HDMI interface. The server connects the audio equipment through the audio cable to output the audio signal. At the same time, it sends WIFI and EV1527 coded wireless communication signals and machine switch signals to the wireless radio frequency communication protocol device. The human-computer interaction module is used to connect the sensing camera and infrared sensor through the USB interface to obtain the user's behavior information and location data, and analyze the original sensing signal through electronic digital data processing, and output the positioning signal and audio signal to the server for triggering the game content; The hardware interaction module is used to receive interaction control instructions from the server; The hardware interaction module includes connecting the lighting interaction module through the DMX512 protocol to realize the lighting scene response control; the hardware interaction module is connected to the joystick interaction module through the USB interface to realize the physical operation signal input and interactive control; The human-computer interaction module also includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electronic digital data processing operations on user behavior information and location data, and constructs a dual behavior modeling path based on multi-user behavior trajectories. It drives the model switching process through behavior deviation identification to generate situational guidance feedback for abnormal users.

[0020] The projection module includes a wall projection, a ground projection 1 and a ground projection 2. The projection module is used to receive a picture display signal from a server to display image content, wherein the ground projection 1 and the ground projection 2 further receive a machine switch signal from a wireless radio frequency communication protocol device through an RS232 serial port; wherein the projection module is used to achieve a multi-dimensional immersive display effect, the wall projection is used to construct a depth visual scene and enhance the sense of spatial immersion, while the ground projection 1 and the ground projection 2 are responsible for the projection task of the interactive main interface, and receive a machine switch signal transmitted by a wireless radio frequency communication protocol device through an RS232 serial port, so as to independently control its opening and closing, and realize the functional separation of partition control and energy-saving management; and The purpose of setting up the external ground projection 1 and the ground projection 2 is to divide the entire sand pool area into independent but coordinated display areas, so that the system can simultaneously carry the interactive needs of multiple users or multiple tasks. On the one hand, the ground projection 1 can be used for real-time presentation of the main task content, such as target guidance, character animation or interactive prompts. On the other hand, the ground projection 2 can carry auxiliary layers, such as environmental special effects, path feedback or regional linkage information. This dual-channel structure not only improves the interaction fluency and content expression, but also can realize the differentiated start and stop of some areas through independent control signals, thereby reducing energy consumption, extending the life of the equipment, and providing a clear spatial segmentation basis for the subsequent access to expansion modules such as emotion recognition or behavior monitoring. The wireless radio frequency communication protocol device is used to receive WIFI and EV1527 encoded wireless communication signals from the server, and output the machine switch signals therein to the floor projection 1 and the floor projection 2 through the RS232 serial port; The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and provide feedback on the execution of the lighting effects; the joystick interaction module is connected to the hardware interaction module through USB to receive user input instructions for controlling the game interaction response; It also includes a game software module, which is used to receive the positioning signal output by the human-computer interaction module and the interaction signal fed back by the hardware interaction module, and trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

[0021] The behavior synchronization observation and deviation prompting mechanism includes the behavior acquisition stage, synchronization comparison stage, model construction stage, model switching stage, and guidance feedback stage; In the behavior collection phase, the user's spatial movement vector sequence in each frame is extracted from the sensing camera, and a behavior trajectory vector function is constructed to describe the position data; ; in For users In time The moving trajectory vector function, in units of ; For users collected by the sensor camera In frame time The image intensity value function, unit dimensionless, is used to normalize the image intensity value between 0 and 1; Represents the image in two dimensions Gradient extraction is performed on ; is the frame time integral unit, in units of ; The behavior trajectory vector function is used to extract the spatial changes in the original image frame into a movement trajectory vector through gradients. The purpose is to construct a continuous expression of the user's movement path as the geometric basis for subsequent behavior modeling. The image gradient constitutes an integral model to ensure that the trajectory has temporal continuity and spatial resolution. The infrared sensor array is used to extract the heat source density change function of the user's body parts per unit time: ; in Indicates user In time The heat source density fluctuation energy, in units of , describing the behavior information through the energy of heat source density fluctuation; For infrared sensor The sensing points at time The temperature flux on ; For the The weights of the sensing points can be defined according to the importance of the user's body parts. Dimensionless but normalized; is the total number of sensing points in the infrared sensor array; is the time difference window, in units of ; The heat source density change function is used to capture the heat source fluctuation pattern and to reflect the movement activity of different parts of the user's body to describe the behavior information. The square error structure in the formula combines the sensor position confidence information to form a relatively sensitive modeling of the motion characteristics. The behavior trajectory vector function is combined with the heat source density change function to construct the user's full-dimensional behavior state vector: ; in For users In time The behavior state vector is in units of ; is the instantaneous modulus of the user's moving path, in units of ; It is the scalar representation of the user's mobile acceleration, in units of ; represents a user behavior feature function generator for constructing a user's full-dimensional behavior state vector; exist In the formula, the position vector and the heat source dynamics are uniformly encoded into a state vector, supporting the subsequent ternary input structure for dynamic behavior modeling; this vector captures the velocity change trend through the modulus and first-order derivative while maintaining a coordinated mapping with the body's thermal activity.

[0022] In the synchronization comparison phase, a user behavior state map is constructed, and the data collected in the collection phase is Constructed as a behavior graph structure: ; in Indicates at time The generated behavior graph structure is in the form of a "state relationship graph"; Represents a user set, no unit; is an edge set, which is based on spatial proximity (Unit: m) Similarity to interaction rhythm (Unit: Hz) calculated; for A collection of, the unit is state strength / second, ; is a real matrix, Represents a users, each user contains dimensional real matrix of behavioral characteristics; By aggregating the data collected in the collection phase , combined with spatial proximity Similarity to interaction rhythm , build a dynamic graph containing nodes, edge weights and attributes , as a structural carrier for subsequent behavior synchronization propagation calculation; in addition, spatial proximity Indicates user In time The position change distance is used to reflect its movement in the interaction area, and each component is defined as: ;in For users In time The two-dimensional coordinate position of (unit: meter); is the time interval (unit: seconds); is the Euclidean distance calculation function; Interaction rhythm similarity Indicates user In time The continuous behavior trigger frequency sequence is used to describe the user's action density and rhythm state per unit time: ;in For time window Internal User The interaction trigger frequency (unit: Hertz, i.e. "times / second"); In is the number of window segments of the rhythm vector; Based on the user behavior state graph, a Laplace spectral kernel propagation model is constructed to capture the multi-order graph diffusion effect of synchronous behavior; the Laplace spectral kernel propagation model is expressed as: ;in For users exist The synchronization eigenvector under order neighborhood propagation, the unit is state strength / second; For the The order propagation weight is a dimensionless coefficient; is the graph Laplacian matrix, unitless; The Laplace spectral kernel propagation model performs high-order kernel diffusion operations on the user behavior state graph. Propagate in multi-order neighborhoods to obtain the synchronized characteristic response vector of each user , used to model the potential collaborative interaction relationship between it and neighboring users and capture the coordination of group behaviors with multi-step dependencies; Next, a local synchronization stability deviation score is generated to determine whether there is structural deviation or synchronization break. Based on this, a local Laplace graph stability index is constructed: ; in For users The synchronization stability deviation score of , in (state strength / second)²; For users of The set of neighbors of order, unitless; Indicates user A user in the neighborhood user set of ; For users exist The synchronous eigenvector under the order neighborhood propagation; Indicated in Time under order neighborhood propagation ,user With users The square of the Euclidean distance between the behavioral trajectories; It should be noted that the use of With neighbor nodes The mean of the squared sum of the Euclidean distances is used to calculate the synchronization stability index , to evaluate the user's Whether there is behavioral disjunction or structural deviation in the order-interaction propagation, providing highly sensitive structural early warning signals for model switching; Perform time series convolution matching on the behavior propagation direction, construct frequency domain propagation difference index, and measure the difference of cross-user propagation mode through frequency domain transformation: ; in is the frequency domain propagation difference index, the unit is (state intensity)²; is Fourier transform; represents the frequency variable after Fourier transformation; is the average response of all neighboring node spectra, in units of state intensity; , Indicates the upper and lower limits of the frequency interval of the integral; Indicates user In the channel The original behavioral state signal sequence on ; The unit of is state intensity; Comparing Users in the Fourier Domain and the neighborhood group spectrum mean The difference in output , used to explore different steps of user interaction rhythm, which is a frequency domain supplement to structural difference analysis, and is especially suitable for identifying hidden interaction avoidance patterns; Fusion synchronization structure deviation and frequency domain error, output the final synchronization deviation factor as the next stage model switching criterion, comprehensive and , output synchronization anomaly score: ; in Score the user synchronization anomaly in dimensionless probability scores; It is the combined weight of structural deviation and frequency domain difference, unitless; is the Sigmoid normalization function, which is used to normalize to probability distribution; Will and Weighted fusion, input normalization function , outputs a synchronous anomaly score with a unified probability dimension ,This score is used to determine whether to execute the model switching mechanism and is a key node in the behavior deviation diagnosis and feedback process.

[0023] In the model building phase, the behavior state vector set outputted in the acquisition phase or the synchronization comparison phase is used. Synchronous anomaly score collection , respectively construct normal behavior response models , build an abnormal behavior response model , and generate a difference map , the final output behavior response judgment index ;in is the total number of users, For specific users; The user's behavior state vector Map to low-dimensional feature subspace and cluster encoding to build a normal behavior response model : ; exist In the formula, the normal behavior feature map is constructed through nonlinear projection and cluster alignment to capture the stable mode of most users; in Represents the feature projection weight matrix, unit: dimensionless; is the dimension of the original behavior state vector, The feature dimension of the target mapping space; Represents the bias vector, with the same unit as Consistency; is the Swish activation function, defined as ; Represents the cluster center, the mean vector of the cluster to which it belongs, and the unit is consistent with the embedding; Represents a set of model parameters; is the regularization weight term, unit: dimensionless; Select the synchronization deviation function Greater than threshold of users, extract disturbance behaviors and build abnormal behavior response models : ; in is the synchronization deviation function, and the calculation formula is: ; in: is the total amount of spatial deviation, unit: m; is the behavioral synchronization variance, unit: ; Perturbation spread function, unit: dimensionless; Time standardization operation: Time standardization operation is used to normalize the behavior time axis; is the perturbation feature extraction function, the perturbation feature extraction function is such as spline fitting ),in is the output value of the perturbation feature extraction function, For the Spline basis function (B-spline) is used to fit the local feature segments of the behavior trajectory. Corresponding to The weight coefficients of the spline basis functions; Represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is a sparse decoding network, which is used to output a sparse feature matrix; is the synchronization abnormality threshold, unit: m; By calculating the distribution deviation and state topological distance between models, the overall difference between normal and abnormal models at the probability and path layers is quantified to assist in determining the degree of deviation and generate a difference map. : ; in Represents JensenShannon divergence, which is used to measure the difference in response distribution. Unit: bit; is the normal behavior state transition diagram, is the current user behavior state transition diagram, Used to represent the jump frequency matrix between state nodes, unit: Hz; ; is the trajectory distance function, which is defined as: ; in is the smoothing factor, unit: Hz; in practical applications Including reference , Including reference ; is the Frobenius norm, unit: Hz, the Frobenius norm is used to take the square root of the sum of the squares of the matrix differences, reflecting the degree of structural difference of the overall graph. The result is a single real number representing the intensity of the structural difference between the two graphs; is a matrix logarithmic function, which is used to amplify small structural differences and avoid dominance of large values; By comparing the cosine similarity between the behavior state vector and the two models, the behavior response judgment index is generated. : ; Behavioral response determination index It is used to judge whether the overall behavior is biased towards a normal or abnormal model, and is called by the context switching module; Represents the cosine similarity function, unit: dimensionless; The unit of is dimensionless; positive output values ​​in the above formula indicate that the behavior is closer to the normal model, and negative values ​​indicate a deviation towards an abnormal structure.

[0024] In the model building phase, according to The fluctuation degree of the switching perception weight function is constructed : ; in It is used to characterize the deviation of the current user's behavior change amplitude from the neighboring user's behavior change amplitude, which serves as the sensitivity basis for model switching judgment; Indicates user At the moment The behavior change vector, in "behavior unit / second", such as the number of operations / second; Neighborhood users At the moment The behavior change vector, with the same unit as ; represents the two-norm measure of the behavior fluctuation energy, Indicates user At the moment The squared norm of the behavior change vector, Indicates the energy intensity of its behavior fluctuations, Indicates user With neighboring users The Euclidean distance of the difference in behavior change at the current moment; For users The set of neighboring users refers to the participating users in the same interaction scenario in practical applications; is the sensitivity factor of behavioral differences, which is used to adjust the decay rate of similarity. The unit is ; is the stability offset buffer constant, with the unit of “behavior energy squared”; is the Sigmoid function, where the Sigmoid function is used to map the nonlinear weight to the interval (0, 1) to form a switching bias score; by Construct a probability distribution function for the input , used to determine whether to trigger the model switching process: ; Probability distribution function By normalizing and comparing the sensitive weight values ​​of behavioral fluctuations, a model switching probability decision standard is established to ensure a flexible switching mechanism under critical deviations. represents the model switching decision probability, which can be understood as The closer the value is to 1, the more switching is needed, which can be determined based on the application environment and requirements; To determine the intensity control factor, the unit is dimensionless; Constructing a dual model selection operator for behavioral model selection , the behavior trajectories are sent to two types of models respectively and State prediction is performed in: ; According to the probability distribution function As the model switching judgment value, the basic normal behavior response model or the intervention-type abnormal behavior response model is selected to perform behavior intention reasoning to ensure that the abnormal state can be responded to at the moment of switching; in Indicates the length The historical behavior window; It is a standard behavior model. In practical applications, the standard behavior model is oriented to synchronous users. The standard behavior model includes selecting a normal behavior response model. It is an intervention compensation model, which is aimed at deviant users and includes the selection of abnormal behavior response model; represents the model switching probability threshold; Finally, the model tag status of the current user at the current moment is output , used for calling in the boot feedback phase: ; Model tag status As the intermediate decision state after model selection, it is used for reference when executing the targeted feedback mechanism in the next stage and for performing differentiated response control on abnormal behaviors.

[0025] In the guidance feedback stage, according to and the current , construct the feedback allocation function , determine the type and intensity of guided feedback: ; Its purpose is to achieve a differentiated situational guidance mechanism by generating feedback intensity and content adjustment methods through different functions between normal behavior states and deviant behavior states; It is a vector consisting of feedback type and feedback intensity. The unit is set according to the specific situation, such as the level of sound and light feedback. is the feedback strength coefficient, and its unit is feedback weight; represents the feedback regulation function constructed based on the output of the standard behavior model; It represents the feedback construction function based on the intervention compensation model state combined with the abnormal probability; Feedback distribution function Deconstructed into a multi-channel context-guided instruction set , and allocate execution resources to the output channel: ; The feedback allocation function is used to convert the feedback vector into the instruction output of a specific execution channel (sound, vision, and plot guidance) for the server to send to the hardware for execution; In the formula The audio feedback intensity required by the audio equipment, unit: dB. Output visual feedback brightness / color level for the projection module in lumens or chromaticity values; To guide the task trigger strength, the unit is the scene control number; is the feedback allocation matrix, and the units are set according to the feedback channel mapping; based on Construct time scene linkage control function , used to determine the subsequent situational intervention process: ; It builds a multi-level plot guidance plan based on the feedback instruction intensity and time interval to achieve dynamic intervention and adjustment in the time dimension; in Indicates the time interval Internal users The generated situation content control instructions are in plot identifiers; The number of preset scenario plots; is an indicator function, which takes the value of 1 when the time period meets the condition, otherwise it takes the value of 0; The trigger weight of each situation module. represents the set of positive real numbers; For the The effective time period of each situation module, unit: seconds; For the Situational plot control instructions; According to the output , through the server to send control instructions to the corresponding projection module or audio equipment, forming a "guidance-response" closed-loop mechanism; ; Build a complete response chain from user behavior to feedback execution, so that deviant behaviors can be guided interactively in an immediate, accurate, and non-intrusive manner, improving the level of scene intelligence; in For the server at time The control command set issued, unit: device command set; Represents a function used to generate visual feedback instructions. Represents a function used to generate feedback commands for a sound device.

[0026] It should be generally explained that this system is based on the organic integration of wireless radio frequency communication protocol devices and visual interaction components (sensing cameras, infrared sensors), aiming to create a multi-user, multi-channel immersive interactive scene in the sand pool environment; first, the server acts as a central controller, and uniformly outputs audio and video signals through audio cables and HDMI interfaces, which can not only make the wall projection and ground projection simultaneously present ocean themes and other content, but also cooperate with audio equipment to achieve a strong sense of space sound effects; at the same time, the server can also transmit WIFI and EV1527 encoded wireless communication signals to the wireless radio frequency communication protocol device, so as to control the projection module in different areas or trigger game content; In terms of acquiring user behavior, the human-computer interaction module connects the sensing camera and infrared sensor through the USB interface, so that the user's movement trajectory, body heat source fluctuations, plate and net scooping movements and other information can be captured in real time and transmitted back to the server; unlike ordinary sand pools that only support single camera recognition, the sensing camera of this system is combined with the infrared fill light to detect the reflective film circle attached to the edge of the plate, thereby realizing a variety of interesting interactive functions such as dragging the plate in the ground projection and "releasing fish" when aiming at the net cage; at the same time, the recognition of the wall projection also uses the laser light curtain to improve the coordinate capture accuracy, allowing users to trigger the corresponding popular science introduction when touching the wall to swim fish, forming a truly "touchable" depth scene; In this multi-sensor environment, in order to avoid users interfering with each other and slow movements, the system specially introduces a "behavior synchronization observation and deviation prompt mechanism": the server uses the collected multi-user behavior information to extract each person's movement pattern and heat source activity. If someone shows obvious social avoidance or abnormal movements, it will be judged as a deviation state, and a specially designed guidance scene will be designed through "situational guidance feedback" so that this user can be intervened and assisted under the influence of lighting, projected wave special effects or sound prompts; for example, when the ground plate user does not move with the group or stagnates for a long time, the system can be judged as a low-interaction state, and then an auxiliary prompt will pop up at the ground projection 2 or a guiding prompt voice will be played on the audio equipment to help him re-engage in interaction; It is worth mentioning that the "wave-making joystick" is connected to the hardware interaction module in this system, and sends information such as speed to the server through the USB interface to simulate different special effects from small waves to big waves, forming a strong immersive atmosphere of ocean style; common problems with traditional joysticks are single resistance or inaccurate speed detection. This system combines PWM pulse width modulation technology to dynamically adjust the joystick resistance, allowing users to not only "make big waves" or "small waves", but also feel more realistic gravity feedback when shaking the joystick; at the same time, the server is connected to the lighting interaction module through the DMX512 protocol, which makes it possible for the lighting changes to be synchronized with the size of the waves: when the user shakes the joystick quickly, the ground projection and the light color will be linked to switch to a brighter glare state, accelerating the scene rendering, allowing participants to intuitively feel the rapid response of "waves getting bigger, light getting glare"; As for the wireless radio frequency communication protocol device, this project has been optimized to make it more sensitive when controlling the switches of multiple projectors and server hosts, so as to better switch the upper and lower game content scenes as a whole; it can not only send switch signals, but also send RS232 serial port commands to ground projection 1 and ground projection 2 respectively, so that different areas can be started or paused separately, thus realizing partition management in the game; compared with the disadvantage that ordinary sand pools can only turn on and off all devices at once, this independent control ensures the energy saving and service life of the equipment, and also allows the main task area and the auxiliary special effect area to be flexibly combined for use; for example, the main task area shows users the game content of "fish catching" or "wave forming", and the auxiliary area can carry additional special effects or instructions without the need for the entire scene to light up simultaneously; Through this hierarchical software and hardware linkage design, this system allows every user to "lift the plate to catch fish" or "shake the joystick to make waves" in the sand pool area; if it is detected that some users have a significant lag in responding to the plate or joystick operation, the system can also output special prompts based on the "behavior synchronization observation and deviation prompt mechanism" and even automatically generate a slow-down interaction link; with the robust hardware control of the wireless radio frequency communication protocol device and the data processing process of the server, all screen display effects, sound effects, and game logic can be realized under a unified data processing and control system; thus, a sand pool system with high interaction efficiency, strong immersion, and the ability to take into account special needs (such as plate drag enhancement, wave-making joystick speed distinction, light segmentation control, etc.) is formed; precisely because of its multiple sensor fusion and flexible intervention capabilities for abnormal behaviors, it is obviously different from the common entertainment devices that are only "image projection + human hand swinging", and it is these "multi-user behavior tracking" and "partition projection management" characteristics that make the system suitable for diversified children's entertainment or science popularization interaction fields.

[0027] Our interactive hardware includes: Ground plate recognition interaction: The infrared camera uses the algorithm for plate feature recognition to interact, and the infrared fill light is used to enhance the accuracy of camera recognition. The virtual scoop net formed by the plate triggers the fishing interaction for fish swimming in the sea water on the ground. When the plate (virtual scoop net) is moved to the net cage area, the fish in the scoop net is released into the net cage. The icons and numbers of the caught fish species are displayed around the net cage, and the total number of fish caught by the red and blue teams are displayed on the wall to distinguish the winners and losers. Wave-making joystick interaction: Through the resistance controller, set the resistance parameters to simulate the feedback experience of wave making by the remote control rod; Use the joystick speed detection function to create the intensity of surging waves.

[0028] In addition, the behavior synchronization observation and deviation prompt mechanism in the human-computer interaction module can be used to improve children's social skills. By capturing the movement trajectories and response frequencies of multiple children when interacting in the sand pool, when it is determined that some individuals have obvious interaction delays or avoidance tendencies, the system will automatically adjust the game content or lighting and sound effects to guide the low-interaction target individuals to rejoin the group communication in a gentler way. In this process, the "behavior synchronization observation and deviation prompt mechanism" not only monitors the children's collaborative participation, but also sends personalized "prompt or focus situation" signals through the projection module or audio equipment after identifying abnormalities, providing individuals who may have social problems with a more friendly social interaction environment, thereby helping them gradually form more stable interactive behaviors in shared activities.

[0029] In addition, the wireless radio frequency communication protocol device in the scheme transmits the wireless communication signal encoded by EV1527 at the frequency of RF315MHz / 433.92MHz to the receiving card, and the receiving card is connected to the USB port of the server host to realize communication; Figure 3 As shown, the device is equipped with 4 buttons, and the button functions are equivalent to the keyboard. By setting the keyboard button corresponding to each button through the preset program, the return, up and down switching and play functions of the game can be controlled; the wireless radio frequency communication protocol device realizes the communication between the mainboard and the server through WIFI to control the power on and off commands of the server, and controls the power on and off commands of the projector through the RS232 serial port signal; In the generation and configuration of the product, the algorithm for plate feature recognition can be used to interact with the induction camera and the infrared sensor, and the accuracy of induction camera recognition can be enhanced through the infrared sensor; In actual applications, the virtual scoop net formed by the plate can trigger the fishing interaction for the fish swimming in the sea water on the ground. When the plate (virtual scoop net) is moved to the net cage area, the fish in the scoop net will be dragged and dropped into the net cage. The icons and numbers of the caught fish species are displayed around the net cage, and the total number of fish caught by the red and blue teams are displayed on the wall to distinguish the winners and losers. The specific interactive recognition optimization of the plate is to stick a reflective film circle with the same inner and outer diameters on the plate. The recognition algorithm detects the light spot under the camera and detects that the light spot is an inner and outer circle, then it is identified as an interactive plate, which can greatly reduce the recognition error; Induction camera and infrared sensor: You can choose to use 3 groups of induction cameras and infrared sensors, 2 groups to illuminate the ground, and 1 group to illuminate the wall. When installing, you need to ensure that the projection screen is within the camera's illumination range. You can also add a fill light, which acts on the projection on the ground and is installed next to the projector to enhance the light intensity of the induction camera's illumination screen. This will make the reflective effect of the plate under the camera more obvious, better capture the reflective effect of the plate, and help enhance the recognition function; for Figure 5 It should be noted that the "joystick wave making" can set the resistance parameters through the resistance controller to simulate the feedback experience of joystick wave making; the speed detection function of the joystick can be used to create the intensity of the surging waves; 1. Resistance adjustment mechanism: PWM pulse width modulation technology can be used to control the resistance. The photoeye on the joystick can detect the speed of the joystick shaking. Through the IO input high and low level detection function, the two functions can communicate, so that the resistance can be controlled in the software to affect the joystick speed. 2. Control method: We fix a resistance value so that the joystick has a little resistance when it is shaken but does not affect the rotation, imitating the rotation speed effect under the influence of sea water; based on this resistance value, we divide the joystick rotation speed into three intervals. In the UNITY software production, the speed of the three intervals corresponds to the three special effects of small waves, medium waves and large waves, forming a game experience effect that the greater the joystick shaking speed value, the larger the waves; In the formula structure involved in this scheme, dimensionless terms can be used as proportional or structural adjustment factors. When combined with quantities with units, they only play a role in numerical scaling without introducing new physical dimensions, so they will not change or confuse the unit system of the overall expression. This combination of "dimensionless terms and units" can be understood as a composite structural expression commonly used in mathematical and physical modeling, which conforms to the principle of dimensional consistency and has a clear physical interpretation basis. Secondly, in the formula structure of this scheme, if multiple variables with different physical units are involved, including but not limited to time, mass or energy variables, their joint appearance is to express the collaborative modeling relationship of multiple physical mechanisms. Each variable forms a unified structure through function mapping, ratio combination or normalization adjustment, with clear units and meanings, and the overall expression conforms to the principle of dimensional consistency and the common formula of engineering modeling; The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An immersive interactive sand pool system based on wireless radio frequency and visual interaction, including a server, a projection module, an audio device, a wireless radio frequency communication protocol device, and a human-computer interaction module; The server is used as the central controller of the system. The server outputs the screen display signal to the projection module through the HDMI interface. The server connects the audio equipment through the audio cable to output the audio signal. At the same time, it sends WIFI and EV1527 coded wireless communication signals and machine switch signals to the wireless radio frequency communication protocol device. Features: The human-computer interaction module is used to connect the sensing camera and infrared sensor through the USB interface to obtain the user's behavior information and location data, and analyze the original sensing signal through electronic digital data processing, and output the positioning signal and audio signal to the server for triggering the game content; The hardware interaction module is used to receive interaction control instructions from the server; The hardware interaction module includes connecting the lighting interaction module through the DMX512 protocol to realize the lighting scene response control; the hardware interaction module is connected to the joystick interaction module through the USB interface to realize the physical operation signal input and interactive control; The human-computer interaction module also includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electronic digital data processing operations on user behavior information and location data, and constructs a dual behavior modeling path based on multi-user behavior trajectories. It drives the model switching process through behavior deviation identification to generate situational guidance feedback for abnormal users.

2. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 1 is characterized in that: The projection module includes a wall projection, a floor projection 1 and a floor projection 2. The projection module is used to receive a picture display signal from a server to display image content, wherein the floor projection 1 and the floor projection 2 receive a machine switch signal from a wireless radio frequency communication protocol device through an RS232 serial port; The wireless radio frequency communication protocol device is used to receive WIFI and EV1527 encoded wireless communication signals from the server, and output the machine switch signals therein to the floor projection 1 and the floor projection 2 through the RS232 serial port; The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and provide feedback on the execution of the lighting effects; the joystick interaction module is connected to the hardware interaction module through USB to receive user input instructions for controlling the game interaction response; It also includes a game software module, which is used to receive the positioning signal output by the human-computer interaction module and the interaction signal fed back by the hardware interaction module, and trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

3. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 2 is characterized in that: The behavior synchronization observation and deviation prompting mechanism includes the behavior acquisition stage, synchronization comparison stage, model construction stage, model switching stage, and guidance feedback stage; In the behavior collection phase, the user's spatial movement vector sequence in each frame is extracted from the sensing camera, and a behavior trajectory vector function is constructed to describe the position data; ； in For users In time The moving trajectory vector function, in units of ; For users collected by the sensor camera In frame time The image intensity value function of , the unit is dimensionless; Represents the image in two dimensions Gradient extraction is performed on ; is the frame time integral unit, in units of ; The infrared sensor array is used to extract the heat source density change function of the user's body parts per unit time: ; in Indicates user In time The heat source density fluctuation energy, in units of , describing the behavior information through the energy of heat source density fluctuation; For infrared sensor The sensing points at time The temperature flux on ; For the The weight of each sensing point; is the total number of sensing points in the infrared sensor array; is the time difference window, in units of ; The behavior trajectory vector function is combined with the heat source density change function to construct the user's full-dimensional behavior state vector: ; in For users In time The behavior state vector is in units of ; is the instantaneous modulus length of the user's moving path, in units of ; It is the scalar representation of the user's mobile acceleration, in units of .

4. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 3 is characterized by: In the synchronization comparison phase, a user behavior state map is constructed, and the data collected in the collection phase is Constructed as a behavior graph structure: ; in Indicates at time The generated behavior graph structure is in the form of "state relationship graph"; Represents a user set, no unit; is the edge set; for A collection of, the unit is state strength / second, ; Based on the user behavior state graph, a Laplace spectral kernel propagation model is constructed to capture the multi-order graph diffusion effect of synchronous behavior; the Laplace spectral kernel propagation model is expressed as: ;in For users exist The synchronization eigenvector under order neighborhood propagation, the unit is state strength / second; For the The order propagation weight is a dimensionless coefficient; is the graph Laplacian matrix, unitless; Next, a local synchronization stability deviation score is generated to determine whether there is structural deviation or synchronization break. Based on this, a local Laplace graph stability index is constructed: ; in For users The synchronization stability deviation score of , in (state strength / second)²; For users of The set of neighbors of order, unitless; Indicates user A user in the neighborhood user set of ; For users exist Synchronous eigenvectors under order neighborhood propagation; Perform time-series convolution matching on the behavior propagation direction, construct frequency domain propagation difference index, and measure the difference in cross-user propagation patterns through frequency domain transformation: ; in is the frequency domain propagation difference index, the unit is (state intensity)²; is Fourier transform; represents the frequency variable after Fourier transformation; is the average response of all neighboring node spectra, in units of state intensity; , Indicates the upper and lower limits of the frequency interval of the integral; Indicates user In the channel The original behavioral state signal sequence on ; Fusion synchronization structure deviation and frequency domain error, output the final synchronization deviation factor as the next stage model switching criterion, comprehensive and , output synchronization anomaly score: ; in Synchronize anomaly scores for users; Combining weights for structural deviation and frequency domain differences; is the Sigmoid normalization function.

5. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 4 is characterized in that: In the model building phase, the behavior state vector set outputted in the acquisition phase or the synchronization comparison phase is used. Synchronous anomaly score collection , respectively construct normal behavior response models , build an abnormal behavior response model , and generate a difference map , and finally output the behavior response judgment index ; The user's behavior state vector Map to feature subspace and cluster encode to build normal behavior response model : ； exist In the formula, the normal behavior feature map is constructed through nonlinear projection and cluster alignment to capture the stable mode of most users; in represents the feature projection weight matrix; is the dimension of the original behavior state vector, The feature dimension of the target mapping space; represents the bias vector; is the Swish activation function; represents the cluster center; Represents a set of model parameters; is the regularization weight term; Select the synchronization deviation function Greater than threshold of users, extract disturbance behaviors and build abnormal behavior response models : ; in is the synchronization deviation function, and the calculation formula is: ; in: is the total amount of spatial deviation, unit: m; is the behavioral synchronization variance, unit: ; Perturbation spread function, unit: dimensionless; Time standardization operations; is the perturbation feature extraction function; Represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is a sparse decoding network; in the abnormal behavior response model is the synchronization abnormality threshold, unit: m; By calculating the distribution deviation and state topological distance between models, the overall difference between normal and abnormal models at the probability and path layers is quantified to assist in determining the degree of deviation and generate a difference map. : ； in represents JensenShannon divergence; is the normal behavior state transition diagram, It is the current user behavior state transition diagram; ; is the trajectory distance function; By comparing the cosine similarity between the behavior state vector and the two models, the behavior response judgment index is generated. : ； in Represents the cosine similarity function.

6. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 5 is characterized by: In the model building phase, according to The fluctuation degree of the switching perception weight function is constructed : ； in Indicates user At the moment Behavior change vector; Neighborhood users At the moment Behavior change vector; represents the two-norm measure of the behavior fluctuation energy, Indicates user At the moment The squared norm of the behavior change vector; For users The set of neighboring users refers to the participating users in the same interaction scenario in practical applications; It is a sensitive factor for behavioral differences; is the stability offset buffer constant; is the Sigmoid function; by Construct a probability distribution function for the input , used to determine whether to trigger the model switching process: ; in represents the probability of model switching decision; To determine the intensity control factor, the unit is dimensionless; Constructing a dual model selection operator for behavioral model selection , the behavior trajectories are sent to two types of models respectively and State prediction is performed in: ； in Indicates the length A window into historical behavior; It is a standard behavior model; It is an intervention-type compensation model; represents the model switching probability threshold; Finally, the model tag status of the current user at the current moment is output , used for calling in the boot feedback phase: 。 7. The immersive interactive sand pool system based on wireless radio frequency and visual interaction according to claim 6 is characterized by: In the guidance feedback stage, according to and the current , construct the feedback allocation function , determine the type and intensity of guided feedback: ； Its purpose is to achieve a differentiated situational guidance mechanism by generating feedback intensity and content adjustment methods through different functions between normal behavior states and deviant behavior states; is a vector consisting of feedback type and feedback intensity; is the feedback intensity coefficient; represents the feedback regulation function constructed based on the output of the standard behavior model; It represents the feedback construction function based on the intervention compensation model state combined with the abnormal probability; Feedback distribution function Deconstructed into a multi-channel context-guided instruction set , and allocate execution resources to the output channel: ； In the formula The audio feedback strength required for the audio equipment; Output visual feedback brightness / color level for projection module; To guide the task trigger intensity; Assign a matrix to the feedback; based on Construct time scene linkage control function , used to determine the subsequent situational intervention process: ; It builds a multi-level plot guidance plan based on the feedback instruction intensity and time interval to achieve dynamic intervention and adjustment in the time dimension; in Indicates the time interval Internal users The generated situation content control instructions are in plot identifiers; The number of preset scenario plots; is the indicator function; The trigger weight for each situation module; For the The effective time period of each situation module, unit: seconds; For the Situational plot control instructions; According to the output , sending control instructions to the corresponding projection module or audio equipment through the server; ； in For the server at time The set of control instructions issued; Represents a function used to generate visual feedback instructions; Represents a function used to generate feedback commands for a sound device.

Images