Immersive arc display space interaction control method and system

Abstract

The invention relates to the technical field of display space interaction, and discloses an immersive arc-shaped display space interaction control method and system.The method comprises the steps that depth image data of continuous actions of a user are collected through a depth camera arranged in an arc-shaped display screen, and effective interaction action fragments are recognized; extracting local features and global features, and outputting an action recognition result; carrying out three-dimensional coordinate extraction, carrying out two-stage pulse response filtering fusion on the three-dimensional coordinates and inertial sensor data of the user, and outputting real-time space coordinates; performing man-machine interaction state analysis based on the action recognition result and the real-time space coordinates, and generating system state prediction data; and performing anti-disturbance prediction and dynamic adjustment on the system state prediction data, and generating an interaction control instruction used for driving the arc-shaped display screen to display contents, thereby realizing accurate identification of effective interaction action fragments, and improving the overall performance and user experience of the immersive man-machine interaction system.

Description

Technical Field

[0001] The present invention relates to the field of display space interaction technology, and in particular to an immersive arc display space interaction control method and system. Background Art

[0002] With the rapid development of virtual reality and augmented reality technologies, curved displays have been widely used in the field of human-computer interaction due to their unique immersive visual experience and wide field of view. However, traditional curved display interaction systems mainly rely on fixed interactive devices and preset interaction modes, which makes it difficult to accurately identify and respond to users' natural movements, severely limiting the smoothness and naturalness of the interactive experience.

[0003] At present, although the introduction of devices such as depth cameras and inertial sensors provides a technical basis for user action recognition, it still faces many challenges in practical applications: the user's actions in the curved display space are diverse and non-periodic, resulting in low accuracy in action recognition; secondly, due to the curved surface characteristics of the curved display screen, user position estimation is easily affected by occlusion and environmental interference, affecting the accuracy of interactive positioning; the system response delay and control strategy are not optimized enough, making it difficult to ensure the real-time and stability of the interactive process. Summary of the invention

[0004] The present invention provides an immersive arc display space interactive control method and system, which realizes accurate recognition of effective interactive action fragments and improves the overall performance and user experience of the immersive human-computer interaction system.

[0005] In a first aspect, the present invention provides an immersive curved display space interactive control method, the immersive curved display space interactive control method comprising: The depth camera set in the curved display screen collects the depth image data of the user's continuous actions and identifies the effective interactive action segments; Performing continuous wavelet transform on the effective interactive action fragments to generate a multi-scale time-frequency feature map, extracting local features and global features, and outputting action recognition results; Extracting three-dimensional coordinates of key points of the human skeleton in the depth image data, and fusing them with the user's inertial sensor data by two-stage impulse response filtering to output the user's real-time spatial coordinates in the coordinate system of the curved display screen; Performing human-computer interaction state analysis based on the action recognition result and the real-time spatial coordinates to generate system state prediction data; Anti-disturbance prediction and dynamic adjustment are performed on the system state prediction data to generate interactive control instructions for driving the curved display screen to display content.

[0006] In a second aspect, the present invention provides an immersive curved display space interactive control system, the immersive curved display space interactive control system comprising: A collection module, used to collect depth image data of the user's continuous actions through a depth camera set in the curved display screen, and identify effective interactive action segments; A feature extraction module is used to perform continuous wavelet transform on the effective interactive action fragments, generate a multi-scale time-frequency feature map, extract local features and global features, and output action recognition results; A filtering and fusion module, used to extract three-dimensional coordinates of key points of the human skeleton in the depth image data, and perform two-stage impulse response filtering and fusion with the user's inertial sensor data to output the user's real-time spatial coordinates in the coordinate system of the curved display screen; A state analysis module, used to perform human-computer interaction state analysis based on the action recognition result and the real-time spatial coordinates, and generate system state prediction data; A generation module is used to perform anti-disturbance prediction and dynamic adjustment on the system state prediction data, and generate interactive control instructions for driving the curved display screen to display content.

[0007] In the technical solution provided by the present invention, the present invention collects the user's continuous action data through a depth camera, combines the action energy change value calculation and adaptive threshold segmentation, realizes the accurate recognition of effective interactive action fragments, and effectively reduces the interference of invalid actions on system performance. The continuous wavelet transform and multi-scale time-frequency analysis method are adopted, combined with the deep separable convolutional network and the bidirectional linear attention mechanism, to improve the recognition ability of actions of different durations and complexities, and enhance the robustness of the system. The deep image data and inertial sensor data are fused by the two-stage impulse response filtering algorithm, to achieve high-precision real-time position tracking of the user in the coordinate system of the curved display screen, and to improve the accuracy of spatial positioning. The human-computer interaction state equation based on the extended state observer is designed, and the system response delay problem is effectively solved through the system delay matrix and the state prediction model, and the real-time performance of the interaction process is enhanced. The anti-disturbance model predictive control strategy is adopted, combined with the adaptive gain matrix and fuzzy dynamic compensation, to achieve the smooth generation and accurate execution of interactive control instructions, and improve the stability and reliability of the system. The overall solution fully considers the particularity of the curved display space and significantly improves the overall performance and user experience of the immersive human-computer interaction system through the coordinated optimization of multi-source data fusion and multi-level control strategies.

[0008] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0009] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of an embodiment of an immersive arc display space interactive control method in an embodiment of the present invention; Figure 2 It is a schematic diagram of an embodiment of an immersive curved display space interactive control system in an embodiment of the present invention. DETAILED DESCRIPTION

[0011] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0012] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0013] To facilitate understanding of this embodiment, an immersive arc display space interactive control method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method comprises the following steps: 101. Collecting depth image data of the user's continuous actions through a depth camera disposed in the curved display screen, and identifying effective interactive action segments; It is understandable that the execution subject of the present invention may be an immersive arc display space interactive control system, or a terminal or a server, which is not limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0014] Specifically, the user's continuous movements are collected in real time by the depth camera in the curved display screen to obtain the initial depth image data. The depth image data is subjected to bilateral filtering and morphological operations. The bilateral filtering technology is used to effectively retain the edge details of the image and smooth the noise in the area, thereby achieving preliminary optimization of the depth image. On this basis, morphological operations are introduced to perform corrosion and expansion operations on the image to eliminate small-area noise points and enhance the coherence of the main area to generate denoised depth image data. The depth value of the denoised depth image data is supplemented and calibrated. The lost depth value is repaired by the interpolation algorithm to ensure that the depth information of each pixel in the image is complete and accurate. At the same time, the calibration algorithm is used to correct the image coordinates for the optical distortion caused by the depth camera to ensure that the generated pre-processed depth image data has high accuracy and consistency. On this basis, the pre-processed depth image data is processed, the human body area is extracted by segmentation technology, the background information is removed, and the skeleton features of the human body are extracted to obtain skeleton feature data, which contains the three-dimensional position information of each main joint point of the user. Based on human skeleton data, the three-dimensional spatial displacement of the skeletal joints between adjacent frames is calculated to obtain the inter-frame motion displacement data, which reflects the changing trend of the user's motion. In order to capture the temporal characteristics of the motion, the temporal differential operation is used to process the inter-frame motion displacement data and extract the change rate of the joint displacement. At the same time, the change in the angle of the joint is taken as another important indicator. These characteristics are combined by polynomial combination to calculate the energy change value of the motion. The motion energy change value is segmented by adaptive threshold. By analyzing the distribution characteristics of the motion energy change value, the segmentation threshold is dynamically adjusted to determine the optimal segmentation threshold. Based on the threshold, the motion energy change value is detected by peak, and the starting point of the energy data segment that is continuously greater than the optimal segmentation threshold is marked as the motion start point. This process can accurately identify the moment when the user's motion starts, thereby avoiding interference caused by noise or false motion. At the same time, the sliding window analysis is performed on the motion energy change value, and the end point of the data segment that is continuously less than the optimal segmentation threshold and the window length exceeds the preset time length is marked as the motion end point. According to the motion start point and the motion end point, the original motion sequence in the human motion data is segmented to obtain each independent motion segment. The segmented actions are similar to the predefined curved display screen interactive action templates, and the feature matching algorithm is used to determine whether the user action is a valid interactive action. The valid interactive action fragments are output based on the similarity analysis results.

[0015] 102. Perform continuous wavelet transform on the effective interactive action fragments to generate a multi-scale time-frequency feature map, extract local features and global features, and output the action recognition result; Specifically, spectrum analysis is performed on the effective interactive action fragments to calculate the main frequency characteristics of the action signal. The frequency distribution of the signal is obtained by Fourier transforming the signal, and the main frequency range of the signal is determined by calculating the autocorrelation function. Based on the main frequency range, a suitable continuous wavelet basis function is selected in a targeted manner to ensure that the wavelet transform can efficiently capture the signal characteristics. The scale parameters of the wavelet basis function are optimized. The information entropy minimization criterion is introduced, in which the information entropy is used as an indicator to measure the complexity of the action feature signal at different decomposition scales. By gradually adjusting the scale parameters of the wavelet basis function and calculating the corresponding information entropy value, the parameters that can minimize the information entropy are selected to obtain the optimal decomposition scale sequence. The optimized wavelet basis function is convolved with the effective interactive action fragment to realize the wavelet transform and generate the original time-frequency coefficient matrix. The original time-frequency coefficient matrix is ​​subjected to singular value decomposition to separate the main component and the noise component of the signal, and the denoised time-frequency coefficient matrix is ​​constructed. The denoising matrix is ​​sub-band divided according to the preset frequency interval, the entire spectrum range is decomposed into several frequency bands, and the data corresponding to each frequency band is extracted as an independent frequency band sub-matrix. The frequency band sub-matrices are subjected to energy normalization processing to eliminate the problem of uneven energy distribution caused by differences in frequency band ranges, so that the characteristics of each frequency band have better comparability in subsequent calculations, and the normalized time-frequency coefficients are obtained. The energy distribution characteristics of each frequency band are calculated according to the normalized time-frequency coefficients to generate a time-frequency energy distribution matrix. The time-frequency energy distribution matrix is ​​processed by multi-scale fusion technology, and the feature information at multiple scales is combined to generate a multi-scale time-frequency feature spectrum. Feature extraction is performed on the multi-scale time-frequency feature spectrum to obtain local features and global features respectively. The extraction of local features focuses on analyzing the significant changes in the spectrum in a short period of time, such as the energy peak position and mutation characteristics, while the global features capture the overall characteristics of the action signal by statistically analyzing the distribution pattern of the entire spectrum, such as the degree of energy concentration and the proportion of frequency components. By combining local features and global features, comprehensive recognition and classification of action signals are achieved, and the action recognition results are finally output.

[0016] The multi-scale time-frequency feature map is input into the depthwise separable convolution layer. Through pointwise convolution and depthwise convolution decomposition operations, the feature channels and spatial features are gradually separated, which effectively reduces the number of model parameters and speeds up the calculation speed while retaining the feature expression ability. After the depthwise separable convolution processing, the initial feature map is obtained. The initial feature map is subjected to multi-scale spatial pyramid pooling. Multi-scale spatial pyramid pooling extracts local features at multiple scales by designing pooling kernels of different sizes to capture action patterns in different spatial ranges. In this process, small-scale pooling kernels can extract fine-grained local features, while large-scale pooling kernels can capture a wider range of feature contexts and form multi-level feature representations. Channel attention calculation is performed on multi-level local features to enhance the weights of important features. By calculating the importance of each channel, a channel weight coefficient is generated to reflect the contribution of a specific channel in expressing the action pattern. According to the channel weight coefficient, the feature channel is recalibrated, that is, the weight of the feature channel is adjusted, so that the feature expression of the high-contribution channel is strengthened, while the feature expression of the low-contribution channel is appropriately suppressed, and enhanced local features are obtained. The enhanced local feature sequence is input into the bidirectional linear self-attention module, and the attention weight of the feature sequence is constructed by linear projection operation of the query matrix and the key value matrix. The query matrix is ​​used to represent the dependency of the current feature on other features, while the key value matrix generates attention weights through inner product operation to quantify the correlation between features. On this basis, the feature sequence is weighted and summed according to the attention weights to obtain the global context feature. The global context feature can capture the long-range dependency in the feature sequence, so that the model can not only focus on local features, but also understand the overall temporal dynamics of the action pattern. The global context feature and the enhanced local feature are fused into a hybrid feature through residual connection. The hybrid feature is reorganized for feature channels. By introducing an adaptive weight network, the feature importance of each channel is recalculated to obtain the feature importance coefficient. Based on the importance coefficient, the hybrid feature is weightedly fused to generate a weighted fused feature. The weighted fused feature is input into the fully connected classifier, and the probability distribution of each action category is calculated by the softmax function. The softmax function maps the weighted feature to a probability space, so that the probability of each category can intuitively reflect the model's confidence value for the action category. In this way, the action recognition results are finally output to achieve accurate prediction of the action category.

[0017] 103. Extract the three-dimensional coordinates of the key points of the human skeleton in the depth image data, and perform two-stage impulse response filtering fusion with the user's inertial sensor data to output the real-time spatial coordinates of the user in the curved display screen coordinate system; Specifically, the human body region is segmented based on the depth image data, the human body contour is extracted by the image segmentation algorithm, and a binary mask representing the human body region is generated to ensure that the subsequent processing is only for the target region. The human body region mask is subjected to skeleton refinement processing, and the human body region is simplified into a skeleton contour map with a single pixel width by using the refinement algorithm. On the basis of the skeleton contour map, the key points of the joint area are located in combination with the anatomical characteristics of the human body, and the two-dimensional skeleton key points representing the characteristics of the human body movement are extracted. These key points include the main joint positions, such as the head, shoulders, elbows, knees and ankles, which constitute the skeleton framework of the user's movement. According to the extracted two-dimensional skeleton key points and depth image data, the three-dimensional coordinates are calculated by projection transformation. The two-dimensional plane coordinates are mapped to the three-dimensional space in combination with the depth data to generate the initial three-dimensional coordinate data of the skeleton key points. Kinematic constraint optimization is applied to the initial coordinate data. By constructing a motion constraint model of the human joint, such as the range of joint movement and the consistency of the connection length, the optimized three-dimensional coordinate data is more in line with the actual kinematic characteristics. At the same time, the inertial sensor data worn by the user is preprocessed to ensure the reliability and accuracy of the data. The acceleration and angular velocity data collected by the sensor are compensated for zero drift to eliminate the error caused by sensor drift. At the same time, temperature calibration is performed to correct the impact of temperature changes on the data. The calibrated acceleration and angular velocity data are integrated to obtain the user's relative displacement data. In order to analyze the user's posture information, the relative displacement data is processed by the posture solution algorithm, and the user's posture angle data is calculated by quaternion operation. On this basis, the relative displacement data is converted into a coordinate system according to the posture angle data, and the displacement data under the local coordinates is converted into global displacement data, thereby reflecting the user's position change in the entire interactive space. The three-dimensional coordinate data and the global displacement data are input into the first-stage finite impulse response filter. The first-stage filter smoothes the input data through weighted average operation, eliminates high-frequency noise, retains the motion trend, and outputs the initial fusion data. The initial fusion data is input into the second-stage finite impulse response filter for deep fusion processing. The second-stage filter adopts the complementary filtering principle and realizes the dynamic weight adjustment of the inertial sensor data and the three-dimensional coordinate data through the complementary characteristics in the frequency domain. The low-frequency part is mainly provided by the three-dimensional coordinate data of the depth camera to ensure the global stability of the spatial position, while the high-frequency part is provided by the inertial sensor data to capture the rapid changes of the action. After complementary filtering operation, the fused user real-time spatial coordinates are output. Through the above steps, the user's real-time spatial coordinates in the curved display screen coordinate system are obtained, reflecting the user's position and posture.

[0018] 104. Perform human-computer interaction state analysis based on action recognition results and real-time spatial coordinates to generate system state prediction data; Specifically, the action recognition results are aligned with the real-time spatial coordinates of the user in the coordinate system of the curved display screen to ensure that the action recognition results are consistent with the spatial coordinates in the time dimension, and the basic data of the system state is generated. After obtaining the basic data of the system state, the operation delay factor of the system itself is considered. The curved display screen will be affected by data acquisition delay, data transmission delay and display refresh delay during the interaction process, and these delays are quantified. By measuring the response time of each module, the system delay matrix is ​​constructed, including the input delay of the acquisition device, the data transmission delay in network communication, and the refresh delay of the display hardware. The delay quantification results and the basic data of the system state are combined with parameters to establish a state space expression to comprehensively describe the impact of user actions, spatial positions and system delays on the current interaction state. Taylor series expansion and linear approximation calculation are performed on the state space expression to simplify complex nonlinear relationships. Through Taylor expansion, the state space expression is decomposed into computable linear terms and high-order nonlinear terms, and the linear part is retained for approximation, ignoring the small impact of high-order terms on the system state. The generated human-computer interaction state equation describes the dynamic characteristics of the system in a linear form, which is convenient for subsequent optimization and solution. The human-computer interaction state equation is expanded to expand the state variables to increase the system's adaptability to external disturbances. The expanded state equation is input into the disturbance observation module, and the state expression of the system is corrected by dynamically monitoring the changes in external disturbance factors. The disturbance observation module can identify the impact of the external environment on the interactive system, such as sudden changes in user behavior or short-term fluctuations in device performance, and integrate these disturbances into the expanded state variables to improve the robustness of the model. In order to realize the predictive analysis of the system state, the expanded state variables are substituted into the Gaussian probability distribution model, and the probability calculation is performed to generate the state probability distribution parameters. Through the Gaussian distribution, the probability of occurrence of different states is quantified, and the future state of the system is inferred based on these probability values. On this basis, combined with the historical state sequence, multiple sets of predicted state values ​​are calculated through the recursive prediction algorithm. The recursive prediction algorithm continuously iterates in the time dimension, and uses the current state information and historical state characteristics to gradually update the future state to form a set of multi-step prediction results. Error correction is performed on multiple sets of predicted state values ​​to improve the accuracy of the prediction results. By comparing the deviation between the predicted value and the actual historical data, the model parameters are adjusted, the calculation error is corrected, and more accurate system status prediction data is output.

[0019] 105. Perform anti-disturbance prediction and dynamic adjustment on the system state prediction data, and generate interactive control instructions for driving the curved display screen to display content.

[0020] Specifically, the system state prediction data is input into the disturbance decoupling module, and the data is decomposed using nonlinear separation operations to obtain the internal state sequence and the external disturbance sequence. The internal state sequence represents the dynamic change trend of the system itself, while the external disturbance sequence reflects the impact of external factors on the system. The internal state sequence and the external disturbance sequence are respectively optimized and calculated in the rolling time domain. Based on the rolling optimization idea, a predictive control cost function is constructed to measure the advantages and disadvantages of different control strategies in the future time domain. The design of the predictive control cost function needs to comprehensively consider factors such as system performance indicators, energy consumption, and interactive fluency. The predictive control cost function is solved using the quadratic programming method to generate a set of optimized control input sequences. These control input sequences ensure that the system can always maintain the best control performance during dynamic changes by minimizing the cost function. Constraint processing is performed on the control input optimization sequence. The goal of constraint processing is to ensure that the control input sequence meets the physical and safety boundaries of the actual system. For example, the display range, content switching speed, and dynamic effects of the curved display screen need to meet the preset physical limitations. At the same time, real-time feedback data is introduced into the constraint processing process, and the control boundary is corrected in real time according to the current system operation state to generate initial control instructions that meet actual needs. The initial control command is input into the adaptive gain matrix for control force distribution. The role of the adaptive gain matrix is ​​to dynamically adjust the components of the control command in multiple dimensions according to the needs of different interactive tasks. For example, for scenarios involving simultaneous interaction of multiple users, different interaction priorities and display resources are assigned to each user, so as to achieve reasonable control distribution as a whole. After generating the multi-dimensional control quantity, in order to improve the accuracy and stability of the control, fuzzy dynamic compensation calculation is performed on the multi-dimensional control quantity and the real-time feedback data. The fuzzy logic system is used to process uncertain factors, such as sudden changes in user actions or interference from the external environment, to generate compensation control quantities. The compensation control quantity mapping is converted into a smooth control sequence to eliminate mutation points in the control command and avoid jitter or incoherence when the display content is updated. By optimizing the smoothness of the control signal, the user's interactive experience can be significantly improved. The smooth control sequence is converted into display parameters of the curved display screen, including specific display attributes such as content position, size, color, and dynamic special effects. These display parameters are passed to the curved display screen driver module through the interface, and finally the interactive instructions for controlling the display content are output.

[0021] In the embodiment of the present invention, the present invention collects the user's continuous action data through a depth camera, combines the action energy change value calculation and adaptive threshold segmentation, realizes the accurate recognition of effective interactive action segments, and effectively reduces the interference of invalid actions on system performance. The continuous wavelet transform and multi-scale time-frequency analysis method are adopted, combined with the deep separable convolutional network and the bidirectional linear attention mechanism, to improve the recognition ability of actions of different durations and complexities, and enhance the robustness of the system. The deep image data and the inertial sensor data are fused by the two-stage impulse response filtering algorithm, to realize the high-precision real-time position tracking of the user in the coordinate system of the curved display screen, and to improve the accuracy of spatial positioning. The human-computer interaction state equation based on the extended state observer is designed, and the system response delay problem is effectively solved through the system delay matrix and the state prediction model, and the real-time performance of the interaction process is enhanced. The anti-disturbance model predictive control strategy is adopted, combined with the adaptive gain matrix and fuzzy dynamic compensation, to realize the smooth generation and accurate execution of interactive control instructions, and improve the stability and reliability of the system. The overall solution fully considers the particularity of the curved display space and significantly improves the overall performance and user experience of the immersive human-computer interaction system through the coordinated optimization of multi-source data fusion and multi-level control strategies.

[0022] In a specific embodiment, the process of executing step 101 may specifically include the following steps: The depth image data of the user's continuous actions are collected by a depth camera arranged in the curved display screen, and bilateral filtering and morphological operations are performed on the depth image data to obtain noise-reduced depth image data; Performing depth value completion and calibration on the denoised depth image data to obtain pre-processed depth image data, and performing human body region segmentation and skeleton feature extraction on the pre-processed depth image data to obtain human body motion data; The three-dimensional spatial displacement of the skeletal joints of adjacent frames in the human motion data is calculated to obtain the inter-frame motion displacement data, the inter-frame motion displacement data is subjected to temporal differentiation operation, and a polynomial combination is performed with the joint angle change to obtain the motion energy change value; Adaptive threshold segmentation is performed on the action energy change value to obtain the optimal segmentation threshold, and peak detection is performed on the action energy change value according to the optimal segmentation threshold, and the starting point of the data segment that is continuously greater than the optimal segmentation threshold is marked as the action starting point; Perform sliding window analysis on the action energy change value, and mark the end point of the data segment that is continuously less than the optimal segmentation threshold and whose window length is greater than the preset time as the action end point; According to the action start point and action end point, the original action sequence in the human motion data is segmented, and the similarity with the predefined curved display screen interactive action template is calculated to output the effective interactive action fragments.

[0023] Specifically, the depth camera continuously collects depth image data of the user's actions in the form of frames, and each frame of data contains the depth value of each pixel in the scene. ,in and Represents the horizontal and vertical coordinates of the pixel in the image. Indicates the distance from the pixel to the depth camera. The original depth image data is affected by noise and environmental interference, and noise reduction is performed to extract effective information. In order to remove random noise in the depth image while retaining edge details, Apply bilateral filtering. The formula for bilateral filtering is: ; in is the filtered depth value, Indicates the window area, is a spatial distance weight function that depends on the distance between pixels; is a weight function of the pixel depth difference, which depends on the difference in depth values, is the normalization factor. Bilateral filtering can effectively remove noise in flat areas while keeping edges clear. Morphological operations such as corrosion and dilation are performed on the filtered depth data. The structure is adjusted. The erosion operation removes small isolated pixels, and the dilation operation fills the empty areas to generate smoother denoised depth image data. The depth value of the denoised depth image data is completed and calibrated. The purpose of the completion is to repair the invalid areas in the depth sensor caused by occlusion or reflection, which is achieved through interpolation. The specific formula is: ; in is the completed depth value, is the spatial weight window, is the interpolation weight, calculated based on the distance between pixels. The calibration process converts the depth data to the world coordinate system according to the camera's intrinsic and extrinsic matrix to generate preprocessed depth image data. The preprocessed depth image data is segmented for human body regions. The human body region is separated from the background by a depth threshold method or a semantic segmentation network to generate a human body mask. A skeleton extraction algorithm, such as a method based on the medial axis transformation, is applied to the human body mask to generate skeleton feature data. Based on the human anatomical model, the positions of the key points of the joints are identified in the skeleton feature data. ,in It is The three-dimensional coordinates of the joints, , , Respectively represent their positions in the spatial coordinate system. For the extracted human motion data, calculate the three-dimensional displacement of the skeleton key points between adjacent frames. Frame and The displacement between frames is expressed as: ; These displacement data reflect the dynamic changes of user actions. Temporal differentiation operation is performed on the inter-frame motion displacement data to capture the trend of motion changes, and combined with the change in joint angle , use polynomial combination to generate action energy change value: ; in It is The motion energy of the frame, and is the weight factor of the joint point, is the total number of joints. Perform adaptive threshold segmentation and determine the optimal segmentation threshold by analyzing the dynamic distribution of energy values , the specific calculation is: ; in is the mean energy, is the standard deviation. Using the optimal threshold Perform peak detection on the energy value and The starting point of the segment is marked as the action starting point. Perform sliding window analysis, window length Greater than the preset duration And the energy value is less than The end point of the segment is marked as the end point of the action. The original action sequence of the human action data is segmented by the action start point and end point. The segmented action is similar to the predefined curved display screen interactive action template. For example, the template is calculated using the dynamic time warping algorithm. and segmented actions Similarity: ; in is the difference between the two sequences. When the similarity exceeds the set threshold, the segment is marked as a valid interaction action segment and output.

[0024] In a specific embodiment, the process of executing step 102 may specifically include the following steps: Perform spectrum analysis on effective interactive action segments, calculate the main frequency characteristics of the action signal, obtain the main frequency range of the signal through autocorrelation function operation, and select the wavelet basis function according to the main frequency range of the signal; The scale parameters of the wavelet basis function are optimized, and the multi-scale analysis of the motion characteristics is performed through the information entropy minimization criterion to obtain the optimal decomposition scale sequence; The effective interactive action fragments are convolved with the wavelet basis function to obtain the original time-frequency coefficient matrix, and the original time-frequency coefficient matrix is ​​subjected to singular value decomposition to construct a denoising matrix; Divide the noise reduction matrix into sub-bands according to a preset frequency interval to obtain a plurality of frequency band sub-matrices, and perform energy normalization processing on the plurality of frequency band sub-matrices respectively to obtain normalized time-frequency coefficients; The energy distribution characteristics of each frequency band are calculated according to the normalized time-frequency coefficients to obtain the time-frequency energy distribution matrix, and the time-frequency energy distribution matrix is ​​fused at multiple scales to obtain a multi-scale time-frequency feature map. Local and global features are extracted from the multi-scale time-frequency feature map, and the action recognition results are output.

[0025] Specifically, for effective interactive action fragments Perform Fourier transform to obtain the main frequency characteristics of the signal in the frequency domain. The Fourier transform formula is: ; in It's a signal The spectrum representation of is the frequency, is an imaginary unit. By analyzing The amplitude spectrum of the signal is used to find the dominant frequency of the signal , which is the frequency component with the largest amplitude in the spectrum. The autocorrelation function is used to further determine the main frequency range of the signal. The autocorrelation function is defined as: ; in It's a signal In delay time By finding The periodic peak value of the estimated signal main frequency range [ ]. According to the main frequency range of the signal, a wavelet basis function matching it is selected, such as the Morlet wavelet, which is defined as: ; in is the wavelet basis function, is the center frequency. Choose the appropriate It can ensure that the wavelet basis function has a good representation of the signal characteristics. After selecting the wavelet basis function, its scale parameters are optimized to ensure that the main characteristics of the signal can be captured in the multi-scale analysis. The optimization is performed through the information entropy minimization criterion, and the information entropy is defined as: ; in The signal is By adjusting the scale parameter of wavelet transform, we can find The smallest parameter combination can obtain the optimal decomposition scale sequence. Perform convolution operation with the selected wavelet basis function to calculate the time-frequency coefficient matrix of wavelet transform : ; in is the scale parameter, is the translation parameter, It is the complex conjugate form of the wavelet basis function. The time-frequency coefficient matrix generated by the convolution operation is The matrix that describes the joint distribution of the signal in time and frequency is called the original time-frequency coefficient matrix. In order to eliminate the influence of noise on the analysis, the original time-frequency coefficient matrix is ​​subjected to singular value decomposition (SVD) denoising. The formula of SVD is: ; in and is an orthogonal matrix, is a diagonal matrix containing the matrix By truncating the smaller singular values, the denoised matrix is ​​constructed. , effectively improving the signal-to-noise ratio of the time-frequency features. Divide the frequency range into sub-bands according to the preset frequency range, for example, divide the frequency range into low frequency, medium frequency and high frequency bands, and generate multiple frequency band sub-matrices For each submatrix Perform energy normalization and calculate the normalized time-frequency coefficient: ; in It is The normalized coefficient matrix of the subband. According to the normalized time-frequency coefficient Calculate the energy distribution characteristics of each frequency band and generate the time-frequency energy distribution matrix : ; Through Multi-scale fusion is performed to weightedly combine the energy features of different frequency bands to generate a multi-scale time-frequency feature map. After obtaining the multi-scale time-frequency feature map, the deep learning network is used to extract local and global features and output the action recognition results. Local feature extraction targets significant changes within a short time window, such as energy peak position and amplitude mutation, while global feature extraction analyzes the characteristics of action signals from a global perspective by statistically analyzing the distribution pattern of the entire map, such as frequency concentration and energy trajectory trend.

[0026] In a specific embodiment, the execution step extracts local features and global features from the multi-scale time-frequency feature map, and the process of outputting the action recognition result may specifically include the following steps: The multi-scale time-frequency feature map is input into the depth-separable convolution layer, and the initial feature map is obtained by using point-wise convolution and depth-wise convolution decomposition operations. Perform multi-scale spatial pyramid pooling on the initial feature map and extract multi-level local features through pooling kernels of different scales; Perform channel attention calculation on multi-level local features to obtain channel weight coefficients, and recalibrate feature channels according to the channel weight coefficients to obtain enhanced local features; The enhanced local feature sequence is input into the bidirectional linear self-attention module, and the attention weight is constructed through the linear projection operation of the query matrix and the key value matrix; The feature sequence is weighted and summed according to the attention weight to obtain the global context feature, which is then fused with the enhanced local feature through residual connection to obtain the hybrid feature. The feature channels of the mixed features are reorganized, and the importance coefficient of each feature is calculated through an adaptive weight network to obtain the weighted fusion feature. The weighted fusion feature is input into the fully connected classifier, and the probability distribution of each action category is calculated through the softmax function to obtain the action recognition result.

[0027] Specifically, the multi-scale time-frequency feature map Input depthwise separable convolution layer. Depthwise separable convolution is divided into two parts: pointwise convolution and depthwise convolution. Its purpose is to reduce computational complexity while retaining feature extraction capability. In pointwise convolution, the features of each channel are combined with a The convolution kernel is operated and the calculation formula is: ; in It is a channel The output features of Is the input channel The feature map of yes The convolution weights, is the number of channels of the input feature. The depthwise convolution operates independently on each channel. The convolution kernel captures local spatial features, and the formula is: ; in is the range of the convolution kernel, is the weight of the deep convolution kernel. Through this separation operation, the initial feature map is generated , which not only reduces the number of parameters but also preserves the time-frequency characteristics of the graph. Multi-scale spatial pyramid pooling is performed to capture multi-level local features. In pyramid pooling, pooling kernels of different sizes are selected. , perform block pooling on the feature map. Pooling result It is expressed by the formula: ; in is the size of the pooling region, It is the local feature representation after pooling. Through multi-scale pooling, features are extracted at different spatial scales to generate multi-level local feature representation. After multi-level local feature extraction, channel attention calculation is performed on it to enhance the representation ability of the features. By calculating the importance weight of each channel , recalibrate the features of different channels. The channel attention calculation formula is: ; in It is a channel The global feature aggregation value of and is the weight matrix, is the Sigmoid activation function. By calculating the channel weight coefficient , reweight the feature channels to generate enhanced local features : ; The enhanced local feature sequence is fed into a bidirectional linear self-attention module, which uses the query matrix , key matrix Sum Matrix Linear projection operation to generate attention weights The specific calculation is: ; in , , , , , is the projection matrix, is the dimension scaling factor of the key. Pair Matrix Weighted summation to generate global context features: ; Global context features and enhance local features Generate mixed features through residual connection fusion : ; Mixing features Input the adaptive weight network to reorganize the feature channels. The network calculates the importance coefficient of each feature channel : ; in is the weight matrix. Application , get the weighted fusion feature : ; The weighted fusion features Input the fully connected classifier, which calculates the probability distribution of each action category through the Softmax function: ; in Yes Category The probability of Yes Category The prediction score of is the total number of categories. Get the action recognition result.

[0028] In a specific embodiment, the process of executing step 103 may specifically include the following steps: Perform human body region segmentation based on depth image data, extract human body contours, and obtain human body region masks; Perform skeleton refinement on the human body region mask to obtain a skeleton contour map, and locate key points of the skeleton contour map according to the human anatomical characteristics to obtain two-dimensional skeleton key points; Perform projection transformation according to the two-dimensional skeleton key points and the depth image data to obtain initial coordinate data, and perform kinematic constraint optimization on the initial coordinate data to obtain three-dimensional coordinate data; The user's inertial sensor data is subjected to zero drift compensation and temperature calibration to obtain calibrated acceleration and angular velocity data, and the calibrated acceleration and angular velocity data are integrated to obtain relative displacement data; Perform attitude calculation on the relative displacement data, obtain attitude angle data through quaternion operation, and perform coordinate system conversion on the relative displacement data according to the attitude angle data to obtain global displacement data; The three-dimensional coordinate data and global displacement data are input into the first-stage finite impulse response filter, and the initial fusion data is obtained through weighted average operation; The initial fusion data is input into the second-stage finite impulse response filter, and the real-time spatial coordinates of the user in the curved display screen coordinate system are output through complementary filtering operation.

[0029] Specifically, human body region segmentation is performed based on depth image data, and depth information of human body region is extracted by threshold segmentation method. The depth image data is represented as ,in represents pixel coordinates, Indicates the depth value of the pixel. By setting the threshold range , extract the pixels that meet the conditions to generate a binary human body area mask : ; The generated mask is skeletonized. The medial axis transformation method is used to Perform iterative erosion while preserving the topological structure of the region to generate a skeleton contour map The goal of skeleton refinement is to reduce the multi-pixel width of the human body region to a single-pixel skeleton structure while retaining the main information of joints and limbs. Based on the human anatomical features, key points are located. By detecting specific geometric features of the skeleton, such as endpoints and bifurcations, the positions of the main joints of the human body are determined. Assume that the detected joint points are ,in Indicates the number of joint points, the two-dimensional coordinates of these joint points constitute the user's skeleton key points. and depth image data , and maps it to three-dimensional space through projection transformation. The intrinsic parameter matrix of the depth camera is , the specific form is: ; in is the focal length, are the coordinates of the optical center. The calculation formula is: ; Through the above formula, the initial three-dimensional coordinates of each joint point are obtained In order to ensure the accuracy of the three-dimensional coordinates, kinematic constraints are optimized. By setting the length constraints between the human joints (in For joints and The nonlinear optimization algorithm is used to adjust the joint point coordinates to generate optimized three-dimensional coordinate data that conforms to the human body structure. At the same time, the acceleration collected by the user's inertial sensor is and angular velocity The data is zero-drift compensated and temperature calibrated. The dynamic bias estimation model , the calibrated acceleration and angular velocity are: ; in and are the bias values ​​of acceleration and angular velocity respectively. Integrate the calibrated data to get the relative displacement data: ; in Represents relative displacement. Use the attitude solution method to calculate the attitude angle data based on the quaternion. Let the quaternion be , the posture update formula is: ; in Represents quaternion multiplication. Updated by numerical integration , calculate the user's posture angle, and convert the relative displacement data to the global coordinate system according to the posture angle to generate the global displacement data . The optimized three-dimensional coordinate data and global displacement data Input the first stage finite impulse response (FIR) filter for preliminary fusion. The output of FIR filtering is: ; in is the filter weight, is the delayed input data. The initial fusion data is generated through weighted average operation. The initial fusion data is input into the second-stage FIR filter, and the complementary filtering algorithm is used to fuse the low-frequency three-dimensional coordinate data and the high-frequency inertial sensor data. The formula is: ; in is the weight factor, and Represent low-frequency and high-frequency components respectively. After two-stage fusion filtering, the real-time spatial coordinates of the user in the curved display screen coordinate system are output.

[0030] In a specific embodiment, the process of executing step 104 may specifically include the following steps: The action recognition results are time-series aligned with the user's real-time spatial coordinates in the curved display screen coordinate system to obtain basic system status data; The data collection delay, data transmission delay and display refresh delay of the curved display screen are quantified and calculated to obtain the system delay matrix. The system state basic data and the system delay matrix are combined and operated with parameters to obtain the state space expression. Perform Taylor series expansion and linear approximation calculation on the state space expression to obtain the human-computer interaction state equation; Perform state expansion processing on the human-computer interaction state equation and input it into the disturbance observation module to obtain the expanded state variable; Substitute the expanded state variables into the Gaussian probability distribution model for probability calculation to obtain the state probability distribution parameters, and perform recursive prediction operations based on the state probability distribution parameters and the historical state sequence to obtain multiple sets of predicted state values; Perform error correction on multiple groups of predicted state values ​​and output system state prediction data.

[0031] Specifically, the action recognition results and the user's real-time spatial coordinates Perform timing alignment. Use linear interpolation to map the two sets of data to a unified time axis. Assumptions The timestamp is and , The timestamp is and , then the aligned data is expressed as: ; ; in and It is the aligned action recognition result and spatial coordinate data. These aligned data together constitute the basic data of system status Data collection delay for curved displays , Data transmission delay and display refresh delay Quantify these delays by measuring them with time synchronization tools or by testing the round trip delay of packets and building a system delay matrix : ; System status basic data With the delay matrix Combine, generate state space expressions through parameter combination operations : ; in is the state transfer matrix, describing the dynamic characteristics of the system; is the input matrix, is the input signal (such as user interaction). Since the system state expression It has nonlinear characteristics. To simplify subsequent calculations, Taylor series expansion is used to linearly approximate it. , at point Nearby expansion is: ; in yes The partial derivative matrix of . On this basis, the human-computer interaction state equation is obtained: ; in is the linearized state transfer matrix, is the perturbation matrix, is the random disturbance of the system. In order to improve the adaptability of the model to complex environments, the human-computer interaction state equation is expanded and the expanded state variable is added. ,in The expanded state input perturbation observation module is used to dynamically estimate The change characteristics of the system can improve the robustness of the system to external disturbances. Substitute into the Gaussian probability distribution model for probability calculation. Assume It obeys multidimensional Gaussian distribution, and its probability density function is: ; in is the mean vector, is the covariance matrix, is the dimension of the variable. By maximizing the probability density function, we get the state probability distribution parameter and . Combining state probability distribution parameters and historical state sequence , a recursive prediction algorithm is used to generate multiple sets of predicted state values. The formula for recursive prediction is: ; ; in is the predicted state value, is the uncertainty of the prediction, is the process noise covariance matrix. Error correction is performed on the multiple sets of predicted state values ​​by comparing the deviation between the actual state and the predicted state. , adjust the model parameters to reduce the prediction error. Output the corrected system state prediction data , providing support for real-time interactive control of curved displays.

[0032] In a specific embodiment, the process of executing step 105 may specifically include the following steps: The system state prediction data is input into the disturbance decoupling module, and the internal state sequence and external disturbance sequence are obtained through nonlinear separation operation; Perform rolling time domain optimization calculation on the internal state sequence and the external disturbance sequence to obtain the predictive control cost function, and solve the predictive control cost function according to the quadratic programming method to obtain the control input optimization sequence; The control input optimization sequence is constrained and displayed boundary correction is performed in combination with real-time feedback data to obtain the initial control instructions; The initial control command is input into the adaptive gain matrix to distribute the control force, and a multi-dimensional control quantity is obtained. The fuzzy dynamic compensation calculation is performed on the multi-dimensional control quantity and feedback data to obtain a compensation control quantity. The compensation control amount is mapped and converted to obtain a smooth control sequence, and the smooth control sequence is converted into a display parameter of the arc display screen, and an interactive control instruction for driving the arc display screen to display content is output.

[0033] Specifically, the system status prediction data Input disturbance decoupling module, which decomposes the prediction data into internal state sequences through nonlinear separation operation and the external perturbation sequence . Internal state sequence represents the core dynamic characteristics of the system, while the external perturbation sequence It reflects the impact of the external environment on the system. The nonlinear separation operation is achieved by minimizing the following objective function: ; in is the nonlinear mapping function of the system, is the second norm, which is used to measure the decomposition error. Through iterative optimization, the system prediction data is separated into and .right and Perform rolling horizon optimization calculations to predict control performance in the future. The core of rolling horizon optimization is to construct a predictive control cost function , which is of the form: ; in is the desired system state, is the control input, and are the weight matrices of state deviation and control cost, respectively. represents the weighted quadratic norm. Solve and get the control input optimization sequence ,in: ; After obtaining the optimal sequence of control inputs, constraints are applied to ensure that the generated control instructions meet the physical boundaries of the actual system. For example, for the display range of a curved display, the control input The following constraints need to be met: ; in and are the upper and lower bounds of the control input. By adding constraints to the optimization problem, the initial control instructions that meet the physical constraints are generated. The initial control instructions are corrected by combining the real-time feedback data. Real-time feedback data Contains the current display state of the system, by comparing and , dynamically adjust the initial control instructions to make them smoother in actual operation. The corrected control instructions It is expressed as: ; in is the feedback gain matrix. The corrected control command is input into the adaptive gain matrix , and distribute the control force in multiple dimensions. The adaptive gain matrix dynamically adjusts the weights of the control components according to the task requirements to generate multi-dimensional control quantities. : ; in The elements of are updated adaptively according to the system status and task priority. After generating the multi-dimensional control quantity, in order to improve the robustness of the system, and feedback data Perform fuzzy dynamic compensation calculations. The goal of compensation is to deal with uncertainties caused by nonlinear effects or environmental disturbances. The fuzzy compensator converts the input error into Convert to compensation control amount : ; in is the fuzzy membership function, is the output value of the compensation rule. A mapping transformation is performed to eliminate mutations and ensure smooth output. The mapping transformation is performed by a smoothing function Implementations, for example: ; in is a commonly used Sigmoid function, which is used to smooth the control quantity. Display parameters converted to curved display , such as the position information, size, color or special effects of the displayed content. These display parameters are passed to the display device through the driver interface to generate real-time feedback required for user interaction.

[0034] The above describes the interactive control method of the immersive curved display space in the embodiment of the present invention. The following describes the interactive control system of the immersive curved display space in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, an immersive curved display space interactive control system includes: The acquisition module 201 is used to acquire depth image data of the user's continuous actions through a depth camera arranged in the curved display screen, and identify effective interactive action segments; The feature extraction module 202 is used to perform continuous wavelet transform on the effective interactive action segments, generate multi-scale time-frequency feature maps, extract local features and global features, and output action recognition results; The filtering and fusion module 203 is used to extract the three-dimensional coordinates of the key points of the human skeleton in the depth image data, and perform two-stage impulse response filtering and fusion with the user's inertial sensor data to output the real-time spatial coordinates of the user in the curved display screen coordinate system; A state analysis module 204 is used to perform human-computer interaction state analysis based on the action recognition results and the real-time spatial coordinates to generate system state prediction data; The generation module 205 is used to perform anti-disturbance prediction and dynamic adjustment on the system state prediction data, and generate interactive control instructions for driving the arc display screen to display content.

[0035] Through the cooperation of the above-mentioned components, the present invention collects the user's continuous action data through a depth camera, combines the action energy change value calculation and adaptive threshold segmentation, realizes the accurate recognition of effective interactive action fragments, and effectively reduces the interference of invalid actions on system performance. The continuous wavelet transform and multi-scale time-frequency analysis method are adopted, combined with the deep separable convolutional network and the bidirectional linear attention mechanism, to improve the recognition ability of actions of different durations and complexities, and enhance the robustness of the system. The deep image data and inertial sensor data are fused by the two-stage impulse response filtering algorithm, and the user's high-precision real-time position tracking in the coordinate system of the curved display screen is realized, and the accuracy of spatial positioning is improved. The human-computer interaction state equation based on the extended state observer is designed. Through the system delay matrix and state prediction model, the system response delay problem is effectively solved, and the real-time performance of the interaction process is enhanced. The anti-disturbance model predictive control strategy is adopted, combined with the adaptive gain matrix and fuzzy dynamic compensation, to achieve the smooth generation and accurate execution of interactive control instructions, and improve the stability and reliability of the system. The overall solution fully considers the particularity of the curved display space and significantly improves the overall performance and user experience of the immersive human-computer interaction system through the coordinated optimization of multi-source data fusion and multi-level control strategies.

[0036] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0037] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0038] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An immersive arc display space interactive control method, characterized in that: The method comprises: The depth camera set in the curved display screen collects the depth image data of the user's continuous actions and identifies the effective interactive action segments; Performing continuous wavelet transform on the effective interactive action fragments to generate a multi-scale time-frequency feature map, extracting local features and global features, and outputting action recognition results; Extracting three-dimensional coordinates of key points of the human skeleton in the depth image data, and fusing them with the user's inertial sensor data by two-stage impulse response filtering to output the user's real-time spatial coordinates in the coordinate system of the curved display screen; Performing human-computer interaction state analysis based on the action recognition result and the real-time spatial coordinates to generate system state prediction data; Anti-disturbance prediction and dynamic adjustment are performed on the system state prediction data to generate interactive control instructions for driving the curved display screen to display content.

2. The interactive control method of immersive curved display space according to claim 1, characterized in that: The method of collecting depth image data of the user's continuous actions by a depth camera disposed in the curved display screen and identifying effective interactive action segments includes: The depth camera disposed in the curved display screen collects depth image data of the user's continuous actions, and performs bilateral filtering and morphological operations on the depth image data to obtain noise-reduced depth image data; Performing depth value completion and calibration on the denoised depth image data to obtain pre-processed depth image data, and performing human body region segmentation and skeleton feature extraction on the pre-processed depth image data to obtain human body motion data; Performing three-dimensional spatial displacement calculation on the skeletal joints of adjacent frames in the human motion data to obtain inter-frame motion displacement data, performing temporal differentiation operation on the inter-frame motion displacement data, and performing polynomial combination with the joint angle variation to obtain a motion energy variation value; Performing adaptive threshold segmentation on the action energy change value to obtain an optimal segmentation threshold, performing peak detection on the action energy change value according to the optimal segmentation threshold, and marking the starting point of a data segment that is continuously greater than the optimal segmentation threshold as an action starting point; Perform sliding window analysis on the action energy change value, and mark the end point of the data segment that is continuously smaller than the optimal segmentation threshold and whose window length is greater than a preset time length as the action end point; According to the action start point and the action end point, the original action sequence in the human action data is segmented, and the similarity with the predefined curved display screen interactive action template is calculated, and the effective interactive action fragment is output.

3. The interactive control method of immersive curved display space according to claim 2, characterized in that: The effective interactive action fragments are subjected to continuous wavelet transform to generate a multi-scale time-frequency feature map, and local and global feature extraction is performed to output action recognition results, including: Performing spectrum analysis on the effective interactive action fragments, calculating the main frequency characteristics of the action signal, obtaining the main frequency range of the signal through autocorrelation function calculation, and selecting a wavelet basis function according to the main frequency range of the signal; Optimizing the scale parameters of the wavelet basis function, performing multi-scale analysis on the action features through the information entropy minimization criterion, and obtaining the optimal decomposition scale sequence; Performing a convolution operation on the effective interactive action fragment and the wavelet basis function to obtain an original time-frequency coefficient matrix, and performing singular value decomposition on the original time-frequency coefficient matrix to construct a noise reduction matrix; Dividing the noise reduction matrix into sub-bands according to a preset frequency interval to obtain a plurality of frequency band sub-matrices, and performing energy normalization processing on the plurality of frequency band sub-matrices respectively to obtain normalized time-frequency coefficients; Calculating the energy distribution characteristics of each frequency band according to the normalized time-frequency coefficients to obtain a time-frequency energy distribution matrix, and performing multi-scale fusion on the time-frequency energy distribution matrix to obtain the multi-scale time-frequency feature spectrum; Local features and global features are extracted from the multi-scale time-frequency feature map, and an action recognition result is output.

4. The interactive control method of immersive curved display space according to claim 3, characterized in that: The extracting local features and global features of the multi-scale time-frequency feature map and outputting the action recognition result includes: Input the multi-scale time-frequency feature map into a depth-separable convolutional layer, and use point-wise convolution and depth-wise convolution decomposition operations to obtain an initial feature map; Performing multi-scale spatial pyramid pooling on the initial feature map, and extracting multi-level local features through pooling kernels of different scales; Performing channel attention calculation on the multi-level local features to obtain channel weight coefficients, and recalibrating the feature channels according to the channel weight coefficients to obtain enhanced local features; Input the enhanced local feature sequence into a bidirectional linear self-attention module, and construct an attention weight by performing a linear projection operation of a query matrix and a key value matrix; Performing weighted summation on the feature sequence according to the attention weight to obtain a global context feature, and fusing it with the enhanced local feature through a residual connection to obtain a mixed feature; The mixed features are reorganized into feature channels, the importance coefficient of each feature is calculated through an adaptive weight network to obtain a weighted fusion feature, and the weighted fusion feature is input into a fully connected classifier, and the probability distribution of each action category is calculated through a softmax function to obtain an action recognition result.

5. The interactive control method of immersive curved display space according to claim 4, characterized in that: The three-dimensional coordinates of the key points of the human skeleton in the depth image data are extracted, and two-stage impulse response filtering and fusion are performed with the inertial sensor data of the user to output the real-time spatial coordinates of the user in the coordinate system of the curved display screen, including: Performing human body region segmentation based on the depth image data, extracting human body contours, and obtaining a human body region mask; Performing skeleton refinement processing on the human body region mask to obtain a skeleton outline map, and locating key points of the skeleton outline map according to human anatomical features to obtain two-dimensional skeleton key points; Performing projection transformation on the two-dimensional skeleton key points and the depth image data to obtain initial coordinate data, and performing kinematic constraint optimization on the initial coordinate data to obtain three-dimensional coordinate data; Performing zero drift compensation and temperature calibration on the user's inertial sensor data to obtain calibrated acceleration and angular velocity data, and performing integration operation on the calibrated acceleration and angular velocity data to obtain relative displacement data; Performing attitude calculation on the relative displacement data, obtaining attitude angle data through quaternion calculation, and performing coordinate system conversion on the relative displacement data according to the attitude angle data to obtain global displacement data; Inputting the three-dimensional coordinate data and the global displacement data into a first-stage finite impulse response filter, and obtaining initial fusion data through weighted average operation; The initial fusion data is input into the second-stage finite impulse response filter, and the real-time spatial coordinates of the user in the coordinate system of the arc display screen are output through complementary filtering operation.

6. The interactive control method of immersive curved display space according to claim 5, characterized in that: The human-computer interaction state analysis is performed based on the action recognition result and the real-time spatial coordinates to generate system state prediction data, including: Performing time-series alignment on the action recognition result and the real-time spatial coordinates of the user in the coordinate system of the curved display screen to obtain basic system status data; Quantitatively calculate the data collection delay, data transmission delay and display refresh delay of the curved display screen to obtain a system delay matrix, and perform parameter combination operation on the system state basic data and the system delay matrix to obtain a state space expression; Performing Taylor series expansion and linear approximation calculation on the state space expression to obtain a human-computer interaction state equation; Performing state expansion processing on the human-computer interaction state equation and inputting it into a disturbance observation module to obtain an expanded state variable; Substituting the expanded state variable into a Gaussian probability distribution model for probability calculation to obtain state probability distribution parameters, and performing recursive prediction operations based on the state probability distribution parameters and historical state sequences to obtain multiple groups of predicted state values; Error correction is performed on the multiple groups of predicted state values, and system state prediction data is output.

7. The interactive control method of immersive curved display space according to claim 6, characterized in that: The anti-disturbance prediction and dynamic adjustment of the system state prediction data to generate interactive control instructions for driving the curved display screen to display content includes: Inputting the system state prediction data into a disturbance decoupling module, and obtaining an internal state sequence and an external disturbance sequence through nonlinear separation operation; Performing a rolling time domain optimization calculation on the internal state sequence and the external disturbance sequence to obtain a predictive control cost function, and solving the predictive control cost function according to a quadratic programming method to obtain a control input optimization sequence; Constraining the control input optimization sequence, and performing display boundary correction in combination with real-time feedback data to obtain initial control instructions; Inputting the initial control instruction into an adaptive gain matrix to distribute the control force to obtain a multi-dimensional control quantity, and performing fuzzy dynamic compensation calculation on the multi-dimensional control quantity and feedback data to obtain a compensation control quantity; The compensation control amount is mapped and converted to obtain a smooth control sequence, and the smooth control sequence is converted into a display parameter of an arc display screen, and an interactive control instruction for driving the arc display screen to display content is output.

8. An immersive arc display space interactive control system, characterized in that: A system for executing the immersive curved display space interactive control method according to any one of claims 1 to 7, the system comprising: A collection module, used to collect depth image data of the user's continuous actions through a depth camera set in the curved display screen, and identify effective interactive action segments; A feature extraction module is used to perform continuous wavelet transform on the effective interactive action fragments, generate a multi-scale time-frequency feature map, extract local features and global features, and output action recognition results; A filtering and fusion module, used to extract three-dimensional coordinates of key points of the human skeleton in the depth image data, and perform two-stage impulse response filtering and fusion with the user's inertial sensor data to output the user's real-time spatial coordinates in the coordinate system of the curved display screen; A state analysis module, used to perform human-computer interaction state analysis based on the action recognition result and the real-time spatial coordinates, and generate system state prediction data; A generation module is used to perform anti-disturbance prediction and dynamic adjustment on the system state prediction data, and generate interactive control instructions for driving the curved display screen to display content.

Images