An immersive interactive sand pool system based on radio frequency and visual interaction

Abstract

The present invention discloses an immersive interactive sand pool system based on radio frequency and visual interaction, specifically related to the field of electro-digital interactive processing, including a server, a projection module, a sound device, a radio frequency communication protocol device, and a human-computer interaction module; the server is used as the central controller of the system. The server outputs a picture display signal to the projection module through an HDMI interface, connects the sound device through an audio cable to output an audio signal, and simultaneously sends WIFI and EV1527 encoded wireless communication signals and machine switch signals to the radio frequency communication protocol device. By performing electro-digital data processing on multi-user behavior information and position data, and detecting individual low-interaction signs during group interaction, so as to identify avoiding users and output situational guidance to help them restore normal social behavior.

Description

Technical Field

[0001] The present invention relates to the technical field of digital interactive processing, and more specifically, to an immersive interactive sand pool system based on radio frequency and visual interaction. Background Art

[0002] Currently, most interactive sand pool systems use projection devices or simple sensors to obtain the basic position information of users, and achieve entertainment scenarios through conventional screen switching and audio output. However, in general sand pool interactions, the system often only recognizes static or single actions, lacking a perception and feedback mechanism for deeper behavior associations, and it is difficult to fully meet the complex needs of various groups of people.

[0003] There is a defect in the existing technology when applied to group scenarios, that is, there is a lack of effective identification and flexible assistance measures for users in a low-interaction or avoidance state, resulting in their difficulty in obtaining a normal interaction experience in a multi-user interaction environment, and also making the overall interaction effect vulnerable to interference from such negative factors. Summary of the Invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an immersive interactive sand pool system based on radio frequency and visual interaction. By constructing a behavior synchronization observation and deviation prompt mechanism, dynamic analysis and deviation identification of multi-user behavior trajectories are carried out to actively identify low-interaction users during group interaction and generate contextualized guiding feedback, thereby solving the deficiencies of the existing system in adapting to complex population behaviors and maintaining interaction quality.

[0005] To achieve the above object, the present invention provides the following technical solution: An immersive interactive sand pool system based on radio frequency and visual interaction, including a server, a projection module, a sound device, a radio frequency communication protocol device, and a human-computer interaction module;

[0006] The server is used as the central controller of the system. The server outputs a video display signal to the projection module through an HDMI interface, connects to the sound device through an audio cable to output an audio signal, and simultaneously sends WIFI and EV1527 encoded wireless communication signals and machine switch signals to the radio frequency communication protocol device;

[0007] It is characterized in that:

[0008] The human-computer interaction module is used to connect an induction camera and an infrared sensor through a USB interface, obtain the behavior information and position data of the user, and parse the original induction signal by means of digital data processing, and output a positioning signal and an audio signal to the server for triggering game content;

[0009] The hardware interaction module is used to receive the interaction control instruction from the server;

[0010] The hardware interaction module includes connecting the lighting interaction module through the DMX512 protocol to achieve lighting scene response control; the hardware interaction module connects to the joystick interaction module through the USB interface to achieve physical operation signal input and interactive control;

[0011] The human-computer interaction module further includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electro-digital data processing operations on user behavior information and position data, and constructs a dual behavior modeling path based on multi-user behavior trajectories. It drives the model switching process through behavior deviation identification to generate situational guidance feedback for abnormal users.

[0012] In a preferred embodiment, the projection module includes wall projection, floor projection 1, and floor projection 2. The projection module is used to receive the picture display signal from the server for image content display. Among them, floor projection 1 and floor projection 2 receive the machine switch signal from the wireless radio frequency communication protocol device through the RS232 serial port;

[0013] The wireless radio frequency communication protocol device is used to receive the WIFI and EV1527 encoded wireless communication signals from the server, and output the machine switch signal to floor projection 1 and floor projection 2 through the RS232 serial port;

[0014] The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and perform the execution feedback of the lighting effect; the joystick interaction module is connected to the hardware interaction module through USB, receives user input instructions, and is used to control the game interaction response;

[0015] It further includes a game software module. The game software module is used to receive the positioning signal output by the human-computer interaction module and the interaction signal feedback by the hardware interaction module to trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

[0016] In a preferred embodiment, the behavior synchronization observation and deviation prompt mechanism includes a behavior collection stage, a synchronization comparison stage, a model construction stage, a model switching stage, and a guidance feedback stage;

[0017] In the behavior collection stage, extract the spatial movement vector sequence of the user in each frame from the induction camera, construct a behavior trajectory vector function, and describe the position data through the behavior trajectory vector function;

[0018] ;

[0019] Wherein For the user at time the movement trajectory vector function, with the unit of ; is the image intensity value function of the user collected by the induction camera at the frame time , with the unit being dimensionless; represents gradient extraction on the image at the two-dimensional position , with the unit of ; is the frame time integration unit, with the unit of ;

[0020] Extract the function of the change in the heat source density of the user's body part within a unit time through the infrared sensor array: ;

[0021] Among them represents the heat source density fluctuation energy of the user at time , with the unit of , and the behavior information is described through the heat source density fluctuation energy; is the temperature flux of the th sensing point of the infrared sensor at time , with the unit of ; is the weight of the th sensing point; is the total number of sensing points in the infrared sensor array; is the time difference window, with the unit of ;

[0022] Fuse the behavior trajectory vector function and the heat source density change function to construct the user's full-dimensional behavior state vector: ;

[0023] Among them is the behavior state vector of the user at time , with the unit of ; is the instantaneous modulus length of the user's movement path, with the unit of ; is the scalar representation of the user's movement acceleration, with the unit of .

[0024] In a preferred embodiment, in the synchronous comparison stage, construct a user behavior state map, and construct the collected in the acquisition stage into a behavior map structure: ;

[0025] Among them represents the behavior graph structure generated at time , with the unit of "state relation graph"; represents the user set, without unit; is the edge set; is 's set, with the unit of state intensity / second, ;

[0026] Based on the user behavior state graph, a Laplacian spectral kernel propagation model is constructed to capture the multi-order graph diffusion influence of synchronous behaviors; the Laplacian spectral kernel propagation model is expressed as: ; Among them is the synchronous eigenvector of user under the -order neighborhood propagation, with the unit of state intensity / second; is the -order propagation weight, with the unit of dimensionless coefficient; is the graph Laplacian matrix, without unit;

[0027] Immediately afterwards, a local synchronous stability deviation score is generated to judge whether there is structural deviation or synchronous break. Based on this, a local Laplacian graph stability index is constructed: ;

[0028] Among them is the local synchronous stability deviation score of user , with the unit of (state intensity / second)²; is the neighborhood user set of user , without unit; represents a certain user in the neighborhood user set of user ; is the synchronous eigenvector of user under the -order neighborhood propagation;

[0029] Perform temporal convolutional matching on the behavior propagation direction, construct a frequency-domain propagation difference index, and measure the cross-user propagation pattern difference through frequency-domain transformation: ;

[0030] Among them is the frequency-domain propagation difference index, with the unit of (state intensity)²; is the Fourier transform; represents the frequency variable after Fourier transform; is the average response of all neighbor node spectra, with the unit of state intensity; , represents the upper and lower limits of the frequency interval of the integral; represents user The original behavior status signal sequence on the channel ;

[0031] Fuse the synchronization structure deviation and the frequency domain error, output the final synchronization deviation factor as the criterion for model switching in the next stage, and comprehensively and , output the synchronization anomaly score: ;

[0032] where is the user synchronization anomaly score; is the combined weight of the structure deviation and the frequency domain difference; is the Sigmoid normalization function.

[0033] In a preferred embodiment, in the model construction stage, use the set of behavior status vectors output by the acquisition stage and the synchronization comparison stage and the set of synchronization anomaly scores to construct a normal behavior response model and construct an abnormal behavior response model , and generate a difference map ;

[0034] Map the user's behavior status vector to the feature subspace and cluster code it to construct a normal behavior response model :

[0035] ;

[0036] In the formula, construct a normal behavior feature map through non-linear projection and clustering alignment to capture the stable patterns of most users;

[0037] where represents the feature projection weight matrix; is the dimension of the original behavior status vector, is the feature dimension of the target mapping space; represents the bias vector; is the Swish activation function; represents the cluster center; represents the set of model parameters; is the regularization weight term;

[0038] Select users with the synchronization deviation function greater than the threshold , extract the perturbed behavior and construct an abnormal behavior response model : ;

[0039] wherein is the synchronization deviation function, and the calculation formula is: ;

[0040] where: is the total spatial deviation, unit: m; is the behavior synchronization variance, unit: ; is the perturbation expansion function, unit: dimensionless; is the time normalization operation; is the perturbation feature extraction function; represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is the sparse decoding network; the threshold in the abnormal behavior response model is the synchronization anomaly threshold, unit: m;

[0041] By calculating the distribution deviation degree and state topology distance between models, the overall differences between the normal model and the abnormal model in the probability and path layers are quantified, the deviation degree is assisted in judgment, and a difference map is generated :

[0042] ;

[0043] where represents the JensenShannon divergence; is the normal behavior state transition diagram, is the current user behavior state transition diagram; ; is the trajectory distance function;

[0044] By comparing the cosine similarity between the behavior state vector and the two models, a behavior response determination index is generated :

[0045] ;

[0046] where represents the cosine similarity function.

[0047] In a preferred embodiment, in the model construction stage, according to the fluctuation degree of, a switching perception weight function is constructed:

[0048] ;

[0049] where represents the user at the moment behavior change vector; Indicates neighboring users At time Behavior change vector; Indicates the two-norm metric behavior fluctuation energy, Indicates the user At time The squared two-norm of the behavior change vector; For the user Set of neighboring users, which refers to the participating users in the same interaction scenario in practical applications; Is the sensitivity factor of behavior difference; Is the stability offset buffer constant; Is the Sigmoid function;

[0050] With Construct a probability distribution function as the input , used to determine whether to trigger the model switching process: ;

[0051] Where Indicates the model switching decision probability; Is the determination intensity regulation factor, unitless;

[0052] Construct a dual model selection operator for behavior model selection , and send the behavior trajectory into two types of models And For state prediction:

[0053] ;

[0054] Where Indicates the historical behavior window of length ; Is the standard behavior model; Is the intervention compensation model; Indicates the model switching probability threshold;

[0055] Finally, output the model label state of the current user at the current moment , for use in the guidance feedback stage:

[0056] .

[0057] In a preferred embodiment, in the guidance feedback stage, according to And the current , construct a feedback distribution function , to determine the type and intensity of the guidance feedback:

[0058] ;

[0059] Its purpose is to generate feedback intensity and content adjustment methods through different functions between the normal behavior state and the deviation behavior state, so as to achieve a differential situation guidance mechanism; among them is a vector composed of feedback type and feedback intensity; is the feedback intensity coefficient; represents a feedback adjustment function constructed based on the output of the standard behavior model; represents a feedback construction function based on the state of the intervention compensation model combined with the abnormal probability;

[0060] Decompose the feedback distribution function into a multi-channel situation guidance instruction set and allocate execution resources to the output channels:

[0061] ;

[0062] In the formula is the audio feedback intensity required by the audio device. is the visual feedback brightness / color level output by the projection module; is the triggering intensity of the guidance task; is the feedback distribution matrix;

[0063] Based on Construct a time-scene linkage control function for determining the subsequent scenario intervention process: ; It constructs a multi-level plot guidance plan according to the feedback instruction intensity and time interval, and realizes dynamic intervention adjustment in the time dimension;

[0064] Among them represents the situation content control instruction generated for the user within the time interval , and the unit is the plot identifier; is the preset number of situation plots; is the indicator function; is the triggering weight of each situation module; is the th effective time period of the situation module, unit: second; is the th situation plot control instruction;

[0065] According to the already output , send control instructions to the corresponding projection module or audio device through the server;

[0066] ;

[0067] Among them is the server at the moment Set of control instructions issued; Function representing the generation of visual feedback instructions. Function representing the generation of audio device feedback instructions.

[0068] Technical effects and advantages of the present invention:

[0069] By performing electro-digital data processing on multi-user behavior information and location data, and detecting individual low-interaction signs during group interaction, avoiding users are identified and situational guidance is output to help them resume normal social behavior;

[0070] With the help of the "behavior synchronization observation and deviation prompt mechanism", the system realizes in-depth aggregation analysis of multi-source information such as location data and action paths, can capture multi-user cooperation patterns, and can also discover individual abnormal deviations;

[0071] Combined with the cross-acquisition method of infrared sensors and cameras, the system tracks the action activity of users in multiple dimensions, making the subsequent judgment results relatively more accurate and improving the adaptability to complex group environments;

[0072] Through the linkage management of the radio frequency communication protocol device and the server, each projector and interactive device can be independently switched and the display content can be automatically adjusted to differentially serve normal interactive users and low-interaction individuals;

[0073] Adopting a phased model switching process, when an abnormal signal is captured, it instantly enters the deviation processing strategy, providing gentle prompts and auxiliary scenarios for users showing avoidance tendencies, thereby steadily improving the overall interaction quality. Description of the drawings

[0074] Figure 1 System module diagram of the present invention.

[0075] Figure 2 System architecture diagram of the present invention.

[0076] Figure 3 Physical diagram of the radio frequency communication protocol device of the present invention.

[0077] Figure 4 Physical diagram of the "plate" interactive hardware in the joystick interaction module of the present invention.

[0078] Figure 5 Physical diagram of the "joystick wave-making" interactive hardware in the joystick interaction module of the present invention.

[0079] Figure 6 Physical diagram of the induction camera and infrared sensor in the present invention. Detailed implementation manners

[0080] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0081] Refer to the attached drawings of the specification Figures 1-6 A kind of immersive interactive sand pool system based on radio frequency and visual interaction in an embodiment of the present invention includes a server, a projection module, a sound device, a radio frequency communication protocol device, and a human-computer interaction module;

[0082] The server is used as the central controller of the system. The server outputs a picture display signal to the projection module through the HDMI interface, connects the sound device through an audio cable to output an audio signal, and simultaneously sends WIFI and EV1527 encoded wireless communication signals and machine switch signals to the radio frequency communication protocol device;

[0083] The human-computer interaction module is used to connect an induction camera and an infrared sensor through the USB interface, obtain the user's behavior information and position data, and parse the original induction signal by means of electro-digital data processing, and output a positioning signal and an audio signal to the server for game content triggering;

[0084] The hardware interaction module is used to receive the interaction control instruction from the server;

[0085] The hardware interaction module includes connecting a lighting interaction module through the DMX512 protocol to realize lighting scene response control; the hardware interaction module connects a joystick interaction module through the USB interface to realize physical operation signal input and interaction control;

[0086] The human-computer interaction module further includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electro-digital data processing operations on the user's behavior information and position data, and constructs a dual behavior modeling path based on the multi-user behavior trajectory, and drives the model switching process through behavior deviation recognition to generate a situation guidance feedback for abnormal users.

[0087] The projection module includes wall projection, floor projection 1, and floor projection 2. The projection module is used to receive the picture display signal from the server for image content display. Among them, floor projection 1 and floor projection 2 further receive the machine switch signal from the wireless radio frequency communication protocol device through the RS232 serial port. The projection module is used to achieve a multi-dimensional immersive display effect. The wall projection is used to construct a depth visual scene to enhance the sense of space immersion, while floor projection 1 and floor projection 2 undertake the projection task of the interactive main interface and receive the machine switch signal transmitted by the wireless radio frequency communication protocol device through the RS232 serial port to independently control their on and off, realizing the functional separation of zoning control and energy-saving management. In addition, the purpose of setting floor projection 1 and floor projection 2 is to divide the entire sand pool area into independent but coordinated display areas, enabling the system to simultaneously carry the interactive needs of multiple users or multiple tasks. On the one hand, floor projection 1 can be used for the real-time presentation of the main task content, such as target guidance, character animation, or interactive prompts. On the other hand, floor projection 2 can carry auxiliary layers, such as environmental special effects, path feedback, or area linkage information. This dual-channel structure not only improves the interactive fluency and content expressiveness but also enables the differential start and stop of some areas through independent control signals, thereby reducing energy consumption, extending the equipment life, and providing a clear spatial segmentation basis for subsequent access to expansion modules such as emotion recognition or behavior monitoring.

[0088] The wireless radio frequency communication protocol device is used to receive the WIFI and EV1527 encoded wireless communication signals from the server and output the machine switch signal to floor projection 1 and floor projection 2 through the RS232 serial port.

[0089] The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and perform the execution feedback of the lighting effect. The joystick interaction module is connected to the hardware interaction module through USB and receives the user input instructions to control the game interaction response.

[0090] It also includes a game software module. The game software module is used to receive the positioning signal output by the human-computer interaction module and the interaction signal feedback by the hardware interaction module to trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

[0091] The behavior synchronization observation and deviation prompt mechanism includes a behavior collection stage, a synchronization comparison stage, a model construction stage, a model switching stage, and a guidance feedback stage.

[0092] In the behavior acquisition stage, a sequence of spatial movement vectors of the user in each frame is extracted from the induction camera, and a behavior trajectory vector function is constructed to describe the position data through the behavior trajectory vector function.

[0093] ;

[0094] where is the movement trajectory vector function of the user at time , with the unit of ; is the image intensity value function of the user collected by the induction camera at the frame time , with the unit being dimensionless and used to normalize the image intensity value between 0 - 1; represents gradient extraction of the image at the two - dimensional position , with the unit of ; is the frame - time integration unit, with the unit of ;

[0095] The behavior trajectory vector function is used to extract the spatial changes in the original image frame as movement trajectory vectors through gradient extraction, aiming to construct a continuous expression of the user's movement path as the geometric basis for subsequent behavior modeling; an integral model is formed by the image gradient to ensure that the trajectory has temporal continuity and spatial resolution;

[0096] The change function of the heat source density of the user's body parts within a unit time is extracted through an infrared sensor array: ;

[0097] where represents the heat source density fluctuation energy of the user at time , with the unit of , and the behavior information is described through the heat source density fluctuation energy; is the temperature flux of the th sensing point of the infrared sensor at time , with the unit of ; is the weight of the th sensing point, which can be defined according to the importance of the user's body parts, dimensionless but normalized; is the total number of sensing points in the infrared sensor array; is the time - difference window, with the unit of ;

[0098] The heat source density variation function is used to capture the heat source fluctuation pattern, reflect the movement activity of different body parts of the user, and describe the behavior information; the squared difference structure in the formula combines the sensor position confidence information to form a relatively sensitive modeling of the motion characteristics;

[0099] Fuse the behavior trajectory vector function with the heat source density variation function to construct the user's full-dimensional behavior state vector: ;

[0100] Where is the user's at time behavior state vector, with the unit of ; is the instantaneous modulus length of the user's movement path, with the unit of ; is the scalar representation of the user's movement acceleration, with the unit of ; represents the user behavior feature function generator used to construct the user's full-dimensional behavior state vector;

[0101] In the formula of , the position vector and the heat source dynamics are uniformly encoded into a state vector, supporting the ternary input structure for subsequent dynamic behavior modeling; this vector captures the speed change trend through the modulus length and the first derivative, while maintaining the coordinated mapping with the body's heat activity.

[0102] In the synchronous comparison stage, construct the user behavior state map, and construct the collected in the collection stage into a behavior map structure: ;

[0103] Where represents the behavior map structure generated at time , with the unit of "state relationship diagram"; represents the user set, without unit; is the edge set, and the edge set is calculated based on the spatial proximity (unit: m) and the interaction rhythm similarity (unit: Hz); is the set, with the unit of state intensity / second, ; is a real number matrix, represents a real number matrix with users, and each user contains dimensional behavior characteristics;

[0104] By aggregating the collected in the collection stage, combined with the spatial proximity and the interaction rhythm similarity , construct a dynamic graph containing nodes, edge weights, and attributes , as a structural carrier for subsequent behavior synchronization propagation calculation; in addition, the spatial proximity represents the user 's position change distance in time , used to reflect their movement within the interaction area, and each of its components is defined as: ; where is the user 's two-dimensional coordinate position at time (unit: meter); is the time interval (unit: second); is the Euclidean distance calculation function;

[0105] Interaction rhythm similarity represents the sequence of continuous behavior trigger frequencies of the user at time , used to describe the action density and rhythm state of the user per unit time: ; where is the interaction trigger frequency of the user within the time window (unit: Hertz, i.e., "times / second"); in of is the number of window segments of the rhythm vector;

[0106] Based on the user behavior state graph, construct a Laplacian spectral kernel propagation model to capture the multi-order graph diffusion influence of synchronous behaviors; the Laplacian spectral kernel propagation model is expressed as: ; where is the synchronous eigenvector of the user under the -order neighborhood propagation, with the unit of state intensity / second; is the -order propagation weight, with the unit of dimensionless coefficient; is the graph Laplacian matrix, without unit;

[0107] Among them, the Laplacian spectral kernel propagation model performs high-order graph kernel diffusion operations on the user behavior state graph, and propagates in the multi-order neighborhood to obtain the synchronous feature response vector of each user, used to model the potential collaborative interaction relationship between it and neighboring users, and capture the group behavior coordination of multi-step dependencies;

[0108] Immediately generate a local synchronization stability deviation score to judge whether there is structural deviation or synchronization break. Based on this, construct a local Laplacian graph stability index: ;

[0109] where is the local synchronization stability deviation score of the user, with the unit of (state intensity / second)²; is the local synchronization stability deviation score of the user, with the unit of (state intensity / second)²; is the set of neighboring users of the user, without unit; is the set of neighboring users of the user, without unit; represents a certain user in the set of neighboring users of the user is a certain user in the set of neighboring users of the user is the synchronization eigenvector of the user under the -order neighborhood propagation; in the formula represents the time under the -order neighborhood propagation, and the square of the Euclidean distance between the behavior trajectories of the user and the user ;

[0110] It should be noted that by using the mean of the sum of the squares of the Euclidean distances between and the neighbor nodes , the synchronization stability index is calculated to evaluate whether there is a behavior disconnection or structural deviation in the user's -order interactive propagation, providing a high-sensitivity structural warning signal for model switching;

[0111] Perform temporal convolutional matching on the behavior propagation direction, construct a frequency-domain propagation difference index, and measure the cross-user propagation pattern difference through frequency-domain transformation: ;

[0112] where is the frequency-domain propagation difference index, with the unit of (state intensity)²; is the Fourier transform; represents the frequency variable after Fourier transform; is the average response of the spectra of all neighbor nodes, with the unit of state intensity; , represents the upper and lower limits of the frequency interval of the integral; represents the user on the channel the original behavior state signal sequence; has the unit of state intensity;

[0113] In the Fourier domain, compare the difference between the user and the neighborhood group spectrum mean , and output , which is used to mine the asynchronous behavior in the user interaction rhythm, is a frequency-domain supplement to the structural difference analysis, and is especially suitable for identifying hidden interaction avoidance patterns;

[0114] Fuse the synchronization structure deviation and the frequency domain error, and output the final synchronization deviation factor as the model switching criterion for the next stage. Synthesize with , and output the synchronization anomaly score: ;

[0115] where is the user synchronization anomaly score, with the unit of dimensionless probability score; is the combined weight of the structure deviation and the frequency domain difference, dimensionless; is the Sigmoid normalization function, used to normalize to a probability distribution;

[0116] Weightedly fuse with , input the normalization function , and output the synchronization anomaly score with a unified probability dimension. This score is used to determine whether to execute the model switching mechanism, which is a key node in the behavior deviation diagnosis and feedback process.

[0117] In the model construction stage, use the set of behavior state vectors output in the acquisition stage and the synchronization comparison stage and the set of synchronization anomaly scores to construct the normal behavior response model and the abnormal behavior response model respectively, and generate the difference map ; where is the total number of users, is a specific user;

[0118] Map the user's behavior state vector to a low-dimensional feature subspace and cluster code it to construct the normal behavior response model :

[0119] ;

[0120] In formula, construct the normal behavior feature map through non-linear projection and clustering alignment to capture the stable patterns of most users;

[0121] where represents the feature projection weight matrix, unit: dimensionless; is the dimension of the original behavior state vector, is the feature dimension of the target mapping space; represents the bias vector, with the unit consistent with ; is the Swish activation function, defined as ; represents the clustering center, the mean vector of the cluster it belongs to, with the unit consistent with the embedding; represents the set of model parameters; is the regularization weight term, unit: dimensionless;

[0122] Select the synchronization deviation function greater than the threshold users, extract the perturbation behavior and construct an abnormal behavior response model : ;

[0123] where is the synchronization deviation function, and the calculation formula is: ;

[0124] where: is the total spatial deviation, unit: m; is the behavior synchronization variance, unit: ; perturbation expansion function, unit: dimensionless; time normalization operation, which is used to normalize the behavior time axis; is the perturbation feature extraction function, and the perturbation feature extraction function is like spline fitting ), where is the output value of the perturbation feature extraction function, is the spline basis function (B-spline), which is used to fit the local feature segments of the behavior trajectory, corresponding to the weight coefficient of the represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is the sparse decoding network, and the sparse decoding network is used to output the sparse feature matrix; the threshold in the abnormal behavior response model is the synchronization anomaly threshold, unit: m;

[0125] By calculating the distribution deviation degree and state topology distance between models, quantify the overall differences between the normal model and the abnormal model at the probability and path levels, assist in judging the deviation degree, and generate a difference map :

[0126] ;

[0127] where represents the Jensen-Shannon divergence, and the Jensen-Shannon divergence is used to measure the response distribution difference, unit: bit; is the normal behavior state transition diagram, is the current user behavior state transition diagram, Used to represent the jump frequency matrix between state nodes, unit: Hz; ; is the trajectory distance function, which is defined as:

[0128] ;

[0129] in is the smoothing factor, unit: Hz; in practical applications Including reference , Including reference ; is the Frobenius norm, unit: Hz, the Frobenius norm is used to take the square root of the sum of the squares of the matrix differences, reflecting the degree of structural difference of the overall graph. The result is a single real number representing the intensity of the structural difference between the two graphs; is a matrix logarithmic function, which is used to amplify small structural differences and avoid dominance of large values;

[0130] By comparing the cosine similarity between the behavior state vector and the two models, the behavior response judgment index is generated. :

[0131] ;

[0132] Behavioral response determination index It is used to judge whether the overall behavior is biased towards a normal or abnormal model, and is called by the context switching module; Represents the cosine similarity function, unit: dimensionless; The unit of is dimensionless; positive output values in the above formula indicate that the behavior is closer to the normal model, and negative values indicate a deviation towards an abnormal structure.

[0133] In the model building phase, according to The fluctuation degree of the switching perception weight function is constructed :

[0134] ;

[0135] in It is used to characterize the deviation of the current user's behavior change amplitude from the neighboring user's behavior change amplitude, which serves as the sensitivity basis for model switching judgment; Indicates user At the moment The behavior change vector, in "behavior unit / second", such as the number of operations / second; Neighborhood users At the moment The behavior change vector, with the same unit as ; represents the two-norm metric of the behavior fluctuation energy, represents the user at the moment the squared two-norm of the behavior change vector, represents the intensity of its behavior fluctuation energy, represents the user and the neighboring user the Euclidean distance of the behavior change difference at the current moment; is the set of neighboring users of the user , which refers to the participating users in the same interaction scenario in practical applications; is the sensitivity factor of the behavior difference, and the sensitivity factor of the behavior difference is used to adjust the attenuation speed of the similarity, with the unit of ; is the stability offset buffer constant, with the unit of "behavior energy squared"; is the Sigmoid function, and the Sigmoid function in the formula is used to map the non-linear weight to the interval (0, 1) to form the switching deviation score;

[0136] With as the input, construct the probability distribution function , which is used to judge whether to trigger the model switching process: ;

[0137] The probability distribution function establishes the model switching probability decision criterion by normalizing and comparing the behavior fluctuation sensitive weight values to ensure a flexible switching mechanism under the critical deviation; where represents the model switching decision probability, which can be understood as the closer the value of is to 1, the more switching is required, and it can be specifically determined according to the application environment and requirements;

[0138] Construct a dual model selection operator for behavior model selection , and send the behavior trajectory into two types of models and respectively for state prediction:

[0139] ;

[0140] According to the probability distribution function as the model switching decision value, select the basic normal behavior response model or the intervention-based abnormal behavior response model to perform behavior intention reasoning to ensure that the abnormal state can be responded to instantly at the moment of switching;

[0141] wherein represents a historical behavior window of length ; is a standard behavior model. In practical applications, the standard behavior model is for synchronous users and includes selecting a normal behavior response model; is an intervention compensation model. The intervention compensation model is for deviated users and includes selecting an abnormal behavior response model; represents a model switching probability threshold;

[0142] Finally, output the model label status of the current user at the current moment , which is used to be called in the guiding feedback stage:

[0143] ;

[0144] The model label status As the intermediate decision-making status after model selection, it is referenced when implementing the targeted feedback mechanism in the next stage and is used to perform differential response control on abnormal behaviors.

[0145] In the guiding feedback stage, according to and the current , construct a feedback distribution function , which determines the type and intensity of the guiding feedback:

[0146] ;

[0147] Its purpose is to generate a feedback intensity and content adjustment method through different functions between the normal behavior state and the deviated behavior state to achieve a differential situation guiding mechanism; wherein is a vector composed of the feedback type and feedback intensity, and the unit is set according to the specific situation, such as the sound and light feedback level, etc.; is the feedback intensity coefficient, and the unit is the feedback weight; represents a feedback adjustment function constructed based on the output of the standard behavior model; represents a feedback construction function based on the intervention compensation model state combined with the abnormal probability;

[0148] Decompose the feedback distribution function into a multi-channel situation guiding instruction set , and allocate execution resources to the output channels:

[0149] ;

[0150] The feedback distribution function is used to convert the feedback vector into the instruction output of specific execution channels (sound, vision, guiding plot) for the server to send to the hardware for execution;

[0151] In the formula is the required audio feedback intensity of the audio device, unit: dB. is the output visual feedback brightness / color level of the projection module, unit: lumen or chromaticity value; is the triggering intensity of the guiding task, unit: scene control label; is the feedback distribution matrix, and the unit is set according to the feedback channel mapping;

[0152] Based on Construct a time-scene linkage control function , which is used to determine the subsequent scenario intervention process: ; It constructs a multi-level plot guiding scheme according to the feedback instruction intensity and time interval section, and realizes dynamic intervention adjustment in the time dimension;

[0153] Among them represents the situation content control instruction generated for the user within the time interval , unit: plot identifier; is the preset number of scenario plots; is an indicator function, and the value of the indicator function is 1 when the time period meets the condition, otherwise it is 0; is the triggering weight of each scenario module, represents the set of positive real numbers; is the th effective time period of the scenario module, unit: second; is the th scenario plot control instruction;

[0154] According to the already output , send control instructions to the corresponding projection module or audio device through the server to form a "guidance-response" closed-loop mechanism;

[0155] ;

[0156] Construct a complete response chain from user behavior to feedback execution, so that deviant behaviors can be instantaneously, accurately and non-intrusively interactively guided, and the scene intelligence level can be improved;

[0157] Among them is the set of control instructions sent by the server at time , unit: device instruction set; represents a function used to generate visual feedback instructions. represents a function used to generate audio device feedback instructions.

[0158] Generally speaking, this system is based on the organic integration of a wireless radio frequency communication protocol device and visual interaction components (inductive cameras, infrared sensors), aiming to create a multi-user and multi-channel immersive interaction scenario in a sand pool environment. First, as the central controller, the server outputs audio and video signals uniformly through audio cables and HDMI interfaces, enabling both wall projections and floor projections to display content such as an ocean theme simultaneously, and also capable of achieving a highly spatial sound effect in cooperation with audio equipment. At the same time, the server can also transmit WIFI and EV1527 encoded wireless communication signals to the wireless radio frequency communication protocol device for zoning control of the projection module or triggering game content.

[0159] In terms of obtaining user behavior, the human-computer interaction module is connected to the inductive camera and infrared sensor through a USB interface, enabling real-time capture and transmission of information such as the user's movement trajectory, body heat source fluctuations, and actions with plates and scoop nets back to the server. Different from ordinary sand pools that only support single camera recognition, the inductive camera of this system combined with an infrared fill light can detect the reflective film circle that fits the edge of the plate, thus realizing various interesting interaction functions such as dragging the plate in the floor projection and "releasing fish" when aligning with the net cage. At the same time, the recognition of the wall projection also uses a laser light curtain to improve the accuracy of coordinate capture, allowing the user to trigger corresponding popular science introductions when touching the swimming fish on the wall, forming a truly "tangible and perceptible" depth scenario.

[0160] In such a multi-sensor environment, to avoid interference and slow actions among users, the system specifically introduces a "behavior synchronization observation and deviation prompt mechanism": The server uses the multi-user behavior information collected to extract the movement patterns and heat source activity levels of each person. If someone shows obvious social avoidance or abnormal actions, it will be determined as a deviation state, and a dedicated guiding scenario will be designed through "situational guidance feedback", enabling this user to be intervened and assisted under the influence of lighting, projection wave special effects, or audio prompts. For example, when the user with the plate on the ground does not follow the group's movement or stays still for a long time, the system can determine it as a low-interaction state, and then pop up an auxiliary prompt at the second floor of the floor projection or play a guiding prompt voice on the audio equipment to help them re-engage in the interaction.

[0161] It is worth mentioning that the "wave-making joystick" is connected to the hardware interaction module in this system, and sends information such as speed to the server through the USB interface to simulate different special effects from small waves to big waves, forming a strong immersive atmosphere of ocean style; common problems with traditional joysticks are single resistance or inaccurate speed detection. This system combines PWM pulse width modulation technology to dynamically adjust the joystick resistance, allowing users to not only "make big waves" or "small waves", but also feel more realistic gravity feedback when shaking the joystick; at the same time, the server is connected to the lighting interaction module through the DMX512 protocol, which makes it possible for the lighting changes to be synchronized with the size of the waves: when the user shakes the joystick quickly, the ground projection and the light color will be linked to switch to a brighter glare state, accelerating the scene rendering, allowing participants to intuitively feel the rapid response of "waves getting bigger, light getting glare";

[0162] As for the wireless radio frequency communication protocol device, this project has been optimized to make it more sensitive when controlling the switches of multiple projectors and server hosts, so as to better switch the upper and lower game content scenes as a whole; it can not only send switch signals, but also send RS232 serial port commands to ground projection 1 and ground projection 2 respectively, so that different areas can be started or paused separately, thus realizing partition management in the game; compared with the disadvantage that ordinary sand pools can only turn on and off all devices at once, this independent control ensures the energy saving and service life of the equipment, and also allows the main task area and the auxiliary special effect area to be flexibly combined for use; for example, the main task area shows users the game content of "fish catching" or "wave forming", and the auxiliary area can carry additional special effects or instructions without the need for the entire scene to light up simultaneously;

[0163] Through this hierarchical software and hardware linkage design, this system allows every user to "lift the plate to catch fish" or "shake the joystick to make waves" in the sand pool area; if it is detected that some users have a significant lag in responding to the plate or joystick operation, the system can also output special prompts based on the "behavior synchronization observation and deviation prompt mechanism" and even automatically generate a slow-down interaction link; with the robust hardware control of the wireless radio frequency communication protocol device and the data processing process of the server, all screen display effects, sound effects, and game logic can be realized under a unified data processing and control system; thus, a sand pool system with high interaction efficiency, strong immersion, and the ability to take into account special needs (such as plate drag enhancement, wave-making joystick speed distinction, light segmentation control, etc.) is formed; precisely because of its multiple sensor fusion and flexible intervention capabilities for abnormal behaviors, it is obviously different from the common entertainment devices that are only "image projection + human hand swinging", and it is these "multi-user behavior tracking" and "partition projection management" characteristics that make the system suitable for diversified children's entertainment or science popularization interaction fields.

[0164] Our interactive hardware includes:

[0165] Ground plate recognition interaction:

[0166] Use the algorithm for recognizing plate features with an infrared camera for interaction, and enhance the accuracy of camera recognition through an infrared fill light; trigger a fishing interaction for the fish swimming in the seawater on the ground through the virtual dip net formed by the plate. When moving the plate (virtual dip net) to the net cage area, trigger the action of putting the fish in the dip net into the net cage. Display the icons and quantities of the types of fish caught around the net cage, and separately display the total quantities of fish caught by the red and blue teams on the wall to distinguish the winning and losing situations;

[0167] Surge joystick interaction:

[0168] Set resistance parameters through a resistance controller to simulate the feedback experience of joystick-generated surges;

[0169] Utilize the function of detecting the speed of the joystick to create the intensity of surging waves.

[0170] In addition, when the behavior synchronization observation and deviation prompt mechanism in the human-computer interaction module is applied to the improvement of children's social skills, by capturing the movement trajectories and response frequencies of multiple children during sandpool interactions, when it is determined that some individuals show obvious interaction delays or avoidance tendencies, the system will automatically adjust the game content or lighting and sound effects to gently guide the target individuals with low interaction to rejoin the group communication; in this process, the "behavior synchronization observation and deviation prompt mechanism" not only monitors the collaborative participation of children, but also, after identifying abnormalities, sends personalized "prompt or focus scenario" signals through the projection module or audio equipment to provide a more friendly social interaction environment for individuals who may have social problems, thereby helping them gradually form more stable interaction behaviors during shared activities.

[0171] In addition, the wireless radio frequency communication protocol device in the solution uses a wireless communication signal encoded by EV1527 with frequencies RF315MHz / 433.92MHz and sends it to the receiving card, and the receiving card is connected to the USB port of the server host to achieve communication; as Figure 3 shown, 4 buttons are configured on the device, and the button functions are equivalent to those of a keyboard. By setting the keyboard buttons corresponding to each button through a preset program, the return, up and down switching, and play functions of the game can be controlled; the wireless radio frequency communication protocol device controls the on / off commands of the server through WIFI to realize communication between the main board and the server, and controls the on / off commands of the projector through RS232 serial port signals;

[0172] In the production and configuration of the product, the algorithm for recognizing plate features can be used with an induction camera and an infrared sensor for interaction, and the accuracy of the induction camera recognition can be enhanced through the infrared sensor;

[0173] In practical applications, a virtual dip net formed by a plate triggers a fishing interaction with fish swimming in the seawater on the ground. When the plate (virtual dip net) is moved to the net cage area, it triggers the dragging of the fish in the dip net and dropping them into the net cage. Icons and quantities indicating the types of fish caught are displayed around the net cage, and the total quantities of fish caught by the red and blue teams are separately displayed on the wall to distinguish the winning and losing situations;

[0174] Specific interaction recognition optimization of the plate: A reflective film circle with the same inner and outer diameters is pasted on the plate. The recognition algorithm detects a light point under the capture of a camera and determines that the light point is an inner and outer circle, then it is recognized and judged as an interactive plate, which can greatly reduce the recognition error;

[0175] Inductive camera and infrared sensor: Three groups of inductive cameras and infrared sensors can be selected. Two groups irradiate the ground and one group irradiates the wall. During installation, it is necessary to ensure that the projection images are within the irradiation range of the camera. Supplementary lights can also be added and installed beside the projector, acting on the projection on the ground to enhance the light intensity of the image irradiated by the inductive camera. In this way, the reflective effect of the plate under the camera will be more obvious, and it is better able to capture the reflective effect of the plate, which is conducive to enhancing the recognition function;

[0176] Regarding Figure 5 the "joystick wave making" mentioned above, it should be noted that the "joystick wave making" can set resistance parameters through a resistance controller to simulate the feedback experience of joystick wave making; by using the detection function of the speed of the joystick, the intensity of the surging waves can be created;

[0177] 1. Resistance adjustment mechanism: The PWM pulse width modulation technology can be used to control the resistance size. The light eye on the joystick can detect the speed of the joystick shaking. Through the IO input high and low level detection function, the two functions are communicated, so that the resistance size can be controlled on the software to affect the joystick speed;

[0178] 2. Control method: When we fix a resistance value, the joystick shakes with a little resistance without affecting rotation, imitating the rotation effect under the influence of seawater; on the basis of this resistance value, the speed of the joystick rotation is divided into three speed intervals. In the production of UNITY software, the three speed intervals respectively correspond to three special effects of small waves, medium waves, and large waves, forming a game experience effect where the greater the speed value of the joystick shaking, the larger the waves;

[0179] In the formula structure involved in this solution, dimensionless terms can serve as proportional or structural adjustment factors. When combined with quantities having units, they only play a role in numerical scaling and do not introduce new physical dimensions. Therefore, they do not change or confuse the unit system of the overall expression. Such combinations of "dimensionless terms and terms with units" can be understood as the composite structure expression forms commonly used in mathematical and physical modeling, conform to the principle of dimensional consistency, and have a clear physical interpretation basis.

[0180] Secondly, in the formula structure of this solution, if there are multiple variable terms with different physical units, including but not limited to time, mass, or energy variables, their combined appearance is to express the collaborative modeling relationship of multiple physical mechanisms. Each variable forms a unified structure through function mapping, ratio combination, or normalization adjustment, with clear units and clear meanings. The overall expression conforms to the principle of dimensional consistency and the common norms of engineering modeling.

[0181] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An immersive interactive sand pool system based on radio frequency and visual interaction, comprising a server, a projection module, a sound device, a radio frequency communication protocol device, and a human-computer interaction module; The server is used as the central controller of the system. The server outputs a picture display signal to the projection module through an HDMI interface, connects to the sound device through an audio cable to output an audio signal, and simultaneously sends WIFI and EV1527 encoded wireless communication signals and machine switch signals to the radio frequency communication protocol device; It is characterized in that: The human-computer interaction module is used to connect an induction camera and an infrared sensor through a USB interface, obtain the behavior information and position data of the user, and parse the original induction signal in the way of electro-digital data processing, and output a positioning signal and an audio signal to the server for game content triggering; The hardware interaction module is used to receive the interactive control instructions from the server; The hardware interaction module includes connecting a lighting interaction module through the DMX512 protocol to achieve lighting scene response control; the hardware interaction module connects a rocker interaction module through a USB interface to achieve physical operation signal input and interactive control; The human-computer interaction module further includes a behavior synchronization observation and deviation prompt mechanism. The behavior synchronization observation and deviation prompt mechanism performs electro-digital data processing operations on the user behavior information and position data, and constructs a dual behavior modeling path based on the multi-user behavior trajectory, and drives the model switching process through behavior deviation recognition to generate situation guidance feedback for abnormal users; The behavior synchronization observation and deviation prompt mechanism includes a behavior collection stage, a synchronization comparison stage, a model construction stage, a model switching stage, and a guidance feedback stage; In the behavior collection stage, a spatial movement vector sequence of the user in each frame is extracted from the induction camera, a behavior trajectory vector function is constructed, and the position data is described by the behavior trajectory vector function; ； wherein is the moving trajectory vector function of the user at time , with the unit of ; is the image intensity value function of the user acquired by the induction camera at frame time , with the unit being dimensionless; represents gradient extraction on the image at two-dimensional position , with the unit of ; is the frame time integration unit, with the unit of ; Extract the heat source density change function of the user's body part within a unit time through an infrared sensor array: ; Among them represents the heat source density fluctuation energy of the user at time , with the unit of , and the behavior information is described by the heat source density fluctuation energy; is the temperature flux of the th sensing point of the infrared sensor at time , with the unit of ; is the weight of the th sensing point; is the total number of sensing points in the infrared sensor array; is the time difference window, with the unit of ; Fuse the behavioral trajectory vector function with the heat source density change function to construct the user's full-dimensional behavioral state vector: ; wherein is the behavior state vector of the user at time , with the unit of ; is the instantaneous modulus length of the user's movement path, with the unit of ; is the scalar representation of the user's movement acceleration, with the unit of .

2. The immersive interactive sand pool system based on radio frequency and visual interaction according to claim 1, characterized in that: The projection module includes a wall projection, a ground projection 1, and a ground projection 2. The projection module is used to receive the picture display signal from the server for image content display, wherein the ground projection 1 and the ground projection 2 receive the machine switch signal from the radio frequency communication protocol device through an RS232 serial port; The radio frequency communication protocol device is used to receive the WIFI and EV1527 encoded wireless communication signals from the server, and output the machine switch signal to the ground projection 1 and the ground projection 2 through an RS232 serial port; The lighting interaction module is used to receive the instructions sent by the hardware interaction module through the DMX512 protocol and perform execution feedback of lighting effects; the rocker interaction module is connected to the hardware interaction module through a USB to receive user input instructions for controlling game interaction response; It further includes a game software module, which is used to receive the positioning signal output by the human-computer interaction module and the interaction signal feedback by the hardware interaction module, and trigger the game content logic. The game content of the game software module includes: triggering game interaction based on the positioning signal; triggering game wave special effect interaction based on the interaction signal; triggering plate recognition interaction based on the positioning signal; triggering quantum recognition interaction based on the positioning signal.

3. The immersive interactive sand pool system based on radio frequency and visual interaction according to claim 2, wherein: In the synchronous comparison stage, a user behavior status graph is constructed, and the collected in the collection stage is constructed into a behavior graph structure: ; Among them represents the behavior graph structure generated at time , with the unit of "state relation graph"; represents the user set, without a unit; is the edge set; is a set of, with the unit of state intensity / second, ; Construct a Laplacian spectral kernel propagation model based on the user behavior state graph to capture the multi-order graph diffusion influence of synchronous behaviors; the Laplacian spectral kernel propagation model is expressed as: ; where is the synchronous eigenvector of user under the propagation of the -th order neighborhood, with the unit of state intensity / second; is the propagation weight of the -th order, with the unit of dimensionless coefficient; is the graph Laplacian matrix, without unit; Subsequently, a local synchronous stability deviation score is generated to determine whether there is a structural deviation or a synchronization break. Based on this, a local Laplacian spectrum stability index is constructed: ; where is the local synchronization stability deviation score of the user, with the unit of (state strength / second)²; is the local synchronization stability deviation score of the user, with the unit of (state strength / second)²; is the set of neighboring users of the user without unit; represents a certain user in the set of neighboring users of the user ; is the synchronization eigenvector of the user under the -order neighborhood propagation; Perform temporal convolutional matching on the behavior propagation direction, construct a frequency-domain propagation difference index, and measure the cross-user propagation pattern difference through frequency-domain transformation: ; Among them is the frequency-domain propagation difference index, with the unit of (state intensity)²; is the Fourier transform; represents the frequency variable after Fourier transform; is the average response of the spectra of all neighbor nodes, with the unit of state intensity; , represents the upper and lower limits of the frequency interval for integration; represents the user on the channel the original behavioral state signal sequence; Fuse the synchronous structure deviation and the frequency domain error, output the final synchronous deviation factor as the criterion for model switching in the next stage, and comprehensively and , output the synchronous anomaly score: ; wherein is the abnormal score synchronized for the user; is the combined weight of the structural deviation and the frequency-domain difference; is the Sigmoid normalization function.

4. The immersive interactive sand pool system based on radio frequency and visual interaction according to claim 3, wherein: In the model construction stage, the set of behavioral state vectors output in the acquisition stage and the synchronization comparison stage is used and the set of synchronization anomaly scores , respectively, to construct a normal behavior response model , construct an abnormal behavior response model , and generate a difference map , and finally output the behavioral response determination index ; Map the user's behavior state vector to the feature subspace and cluster-encode it to construct a normal behavior response model : ； In wherein, a normal behavior feature map is constructed by aligning non-linear projection and clustering to capture the stable patterns of most users; where represents the feature projection weight matrix; is the dimension of the original behavior state vector, and is the feature dimension of the target mapping space; represents the bias vector; is the Swish activation function; represents the cluster center; represents the set of model parameters; is the regularization weight term; Select the synchronization deviation function Greater than the threshold users, extract the perturbation behavior and construct an abnormal behavior response model : ; wherein is the synchronization deviation function, and the calculation formula is: ; Wherein: is the total spatial deviation, unit: m; is the behavior synchronization variance, unit: ; perturbation expansion function, unit: dimensionless; time normalization operation; is the perturbation feature extraction function; represents the convolution operation; is the one-dimensional convolution kernel, unit: dimensionless; is the sparse decoding network; the threshold in the abnormal behavior response model is the synchronization anomaly threshold, unit: m; Quantify the overall differences between normal models and abnormal models at the probability and path levels by calculating the distribution deviation degree and state topology distance between models, assist in judging the deviation degree, and generate a difference map : ； where represents the Jensen-Shannon divergence; is the normal behavior state transition diagram, is the current user behavior state transition diagram; ; is the trajectory distance function; Generate a behavioral response determination index by comparing the cosine similarity between the behavioral state vector and the two models : ； where represents the cosine similarity function.

5. The immersive interactive sand pool system based on radio frequency and visual interaction according to claim 4, wherein: In the model construction phase, according to the degree of fluctuation, construct a switching perception weight function : ； where represents the behavior change vector of the user at time ; represents the behavior change vector of the neighboring user at time ; represents the two - norm metric of the behavior fluctuation energy, represents the squared two - norm of the behavior change vector of the user at time ; is the set of neighboring users of the user , which refers to the participating users in the same interaction scenario in practical applications; is the sensitivity factor of the behavior difference; is the stability offset buffer constant; is the Sigmoid function; Construct a probability distribution function with as the input to determine whether to trigger the model switching process: ; ; Among them represents the model switching decision probability; is the determination intensity regulation factor, with the unit of dimensionless; Construct a dual model selection operator for behavior model selection , and send the behavior trajectories into two types of models respectively and for state prediction: ； Among them represents a historical behavior window with a length of ; is a standard behavior model; is an intervention-based compensation model; represents the model switching probability threshold; Finally, output the model label status of the current user at the current moment , for use in the guided feedback stage: 。 6. The immersive interactive sand pool system based on radio frequency and visual interaction according to claim 5, wherein: In the guided feedback stage, according to and the current , construct the feedback allocation function , and determine the type and intensity of the guided feedback: ； Its purpose is to achieve a differential situation guidance mechanism by generating feedback intensity and content adjustment methods through different functions between the normal behavior state and the deviation behavior state; among them is a vector composed of feedback type and feedback intensity; is the feedback intensity coefficient; represents a feedback adjustment function constructed based on the output of the standard behavior model; represents a feedback construction function based on the state of the intervention compensation model combined with the abnormal probability; Decompose the feedback distribution function into a multi-channel context-guided instruction set , and allocate execution resources to the output channels: ； In the formula is the audio feedback intensity required by the audio device; is the visual feedback brightness / color level output by the projection module; is the guiding task trigger intensity; is the feedback distribution matrix; Based on Construct a time-scenario linkage control function , which is used to determine the subsequent scenario intervention process: ; According to the feedback instruction intensity and time interval, it constructs a multi-level plot guidance scheme to achieve dynamic intervention adjustment in the time dimension; wherein represents the situational content control instructions generated for the user within the time interval , with the unit being the plot identifier; is the preset number of situational plots; is the indicator function; is the trigger weight for each situational module; is the th effective time period of the situational module, unit: second; is the th situational plot control instruction; According to the already output , send control instructions to the corresponding projection module or audio device through the server; ； wherein is the set of control instructions issued by the server at time ; represents a function for generating visual feedback instructions; represents a function for generating audio device feedback instructions.

Images