Virtual space physical boundary tactile warning method and system based on ultrasonic array

Abstract

The application relates to a virtual space physical boundary tactile early warning method, system, device and medium based on an ultrasonic array. The method comprises the following steps: acquiring real-time motion state data and virtual space attribute parameters of a warning target and constructing a target real-time motion state vector, predicting a motion trajectory sequence, performing space early warning monitoring based on the virtual space attribute parameters to obtain dynamic early warning monitoring information, obtaining beam forming control parameters through Doppler compensation and sound field parameter mapping, constructing a dynamic sound field sequence, and obtaining a sound field tactile early warning signal through a human skin tactile perception model mapping and coding. The method can realize advanced, smooth and coordinated tactile early warning with the motion state, and solves the problems of time and space dislocation, feedback lag and perception conflict of a traditional reactive method.

Description

Technical Field

[0001] This invention belongs to the field of electronic technology, and in particular relates to a virtual space physical boundary tactile early warning method and system based on an ultrasonic array. Background Technology

[0002] With the advancement of virtual reality and human-computer interaction technologies, non-contact aerial haptic feedback technology based on ultrasonic arrays has been applied. This technology focuses ultrasonic energy onto specific points in the air to generate perceptible tactile stimulation on the skin. Its core feature is that it can achieve precise spatial haptic mapping without the need for wearing devices, providing a novel interactive method for virtual boundary warning.

[0003] In traditional technologies, tactile warnings for virtual boundaries are mainly achieved by using external sensors to obtain the current position of the user's hands and other parts of the body. This position is then compared in real time with a predefined virtual boundary model. If the user is determined to have entered a warning area, an ultrasonic array is immediately controlled to generate a static, fixed-intensity focused tactile point at the current hand position as a warning signal.

[0004] However, this traditional reactive approach has significant drawbacks when faced with high-speed, dynamic user interactions. First, system processing delays cause severe spatiotemporal misalignment of haptic feedback during high-speed movements, resulting in delayed and discontinuous warning signals that undermine the realism of boundaries and the timeliness of alerts. Second, in complex scenarios where virtual boundaries overlap with the real environment, this method fails to predict and coordinate the relationship between user movement and multiple spatial constraints, leading to chaotic haptic feedback and causing user depth perception conflicts and operational deviations. Summary of the Invention

[0005] Therefore, it is necessary to provide a virtual space physical boundary tactile early warning method and system based on ultrasonic array that can realize dynamic coordinated tactile early warning based on motion trajectory prediction to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a virtual space physical boundary tactile early warning method based on an ultrasonic array, including:

[0007] S1. Obtain the real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data;

[0008] S2. Based on the target's real-time motion state vector, predict the motion trajectory to obtain the predicted trajectory sequence;

[0009] S3. Based on the predicted trajectory sequence and virtual space attribute parameters, conduct spatial early warning monitoring to obtain dynamic early warning monitoring information;

[0010] S4. Based on dynamic early warning monitoring information, predicted trajectory sequence and virtual space attribute parameters, beamforming control parameters are calculated through Doppler compensation and sound field parameter mapping.

[0011] S5. Based on the beamforming control parameters, construct a dynamic sound field sequence that evolves continuously in the time and space dimensions.

[0012] S6. Input the dynamic sound field sequence into the human skin tactile perception model, perform matching mapping, and obtain the target tactile perception driving parameter set;

[0013] S7. Based on the target tactile perception driving parameter set, the warning signal is encoded using tactile coding rules to obtain the sound field tactile warning signal; wherein, the sound field tactile warning signal is used to achieve tactile warning by acting on the target skin through an ultrasonic array.

[0014] Secondly, this application also provides a virtual space physical boundary tactile early warning system based on an ultrasonic array, used to implement the method described in the first aspect, including:

[0015] The acquisition module is used to acquire real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data;

[0016] The trajectory prediction module is used to predict the motion trajectory based on the target's real-time motion state vector, and obtain the predicted trajectory sequence.

[0017] The monitoring information generation module is used to perform spatial early warning monitoring based on predicted trajectory sequences and virtual space attribute parameters, and obtain dynamic early warning monitoring information.

[0018] The control parameter generation module is used to obtain beamforming control parameters based on dynamic early warning monitoring information, predicted trajectory sequences, and virtual space attribute parameters, through Doppler compensation and sound field parameter mapping.

[0019] The dynamic sound field construction module is used to construct a dynamic sound field sequence that evolves continuously in the time and space dimensions based on beamforming control parameters.

[0020] The mapping module is used to match and map the dynamic sound field sequence with the human skin tactile perception model to obtain the target tactile perception driving parameter set;

[0021] The output module is used to encode and convert the warning signal according to the target tactile perception driving parameter set to obtain the sound field tactile warning signal, which is used to act on the skin of the warning target through the ultrasonic array.

[0022] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0023] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods in the first aspect of this application.

[0024] The aforementioned virtual space physical boundary tactile warning method and system based on ultrasonic arrays constructs a real-time motion state vector of the target and predicts its trajectory by using real-time motion state data of the warning target and virtual space attribute parameters. Combined with Doppler frequency shift compensation and sound field parameter mapping technology, it dynamically generates beamforming control parameters that match the user's motion trend and spatial distance. This allows for the construction of a dynamic sound field sequence that evolves continuously in time and space. Through mapping using a human skin tactile perception model and conversion using tactile coding rules, it ultimately outputs a high-precision sound field tactile warning signal that can be applied to the target's skin. This enables an advanced, smooth, and motion-coordinated tactile warning when the user approaches or is about to touch the virtual boundary, significantly reducing the problems of spatiotemporal misalignment, feedback lag, and perception conflict caused by system delay and lack of motion prediction in traditional reactive tactile feedback. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of an implementation environment provided for one embodiment of the present invention;

[0027] Figure 2 This is a flowchart of a virtual space physical boundary tactile early warning method based on an ultrasonic array according to one embodiment of the present invention;

[0028] Figure 3 This is a schematic diagram of a virtual space physical boundary tactile early warning system based on an ultrasonic array, according to one embodiment of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] The virtual space physical boundary tactile early warning method based on ultrasonic array provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the tactile warning terminal 100 can communicate with the sensor 101 via a network. The tactile warning terminal 100 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The sensor 101 can be, but is not limited to, infrared sensors, structured light sensors, high-speed visible light cameras, inertial measurement unit sensors, and gyroscope sensors.

[0031] In one exemplary embodiment, such as Figure 2 As shown, a virtual space physical boundary tactile early warning method based on an ultrasonic array is provided, which is then applied to... Figure 1 Taking the tactile warning terminal 100 as an example, the method includes:

[0032] S1. Obtain the real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data.

[0033] For example, the tactile warning terminal 100 can acquire real-time motion state data of the warning target collected by the sensor 101; the real-time motion state data of the warning target can be a multi-dimensional set characterizing the instantaneous motion features of the warning target, and the real-time motion state data of the warning target can include three-dimensional position coordinates, three-dimensional velocity vector, three-dimensional acceleration vector, motion direction angle and motion angular velocity.

[0034] Optionally, the sensor 101 may include a high-speed visible light camera. The tactile warning terminal 100 can obtain the real-time position of the warning target through the high-speed visible light camera. The tactile warning terminal 100 can calculate the three-dimensional position coordinates of the warning target based on the obtained real-time position of the warning target through a stereo matching algorithm.

[0035] Optionally, the sensor 101 may include a binocular camera. The tactile warning terminal 100 may establish a Cartesian coordinate system with the center of the ultrasonic array as the origin, determine the intrinsic and extrinsic parameters of the binocular camera through the Zhang Zhengyou calibration method, transform the pixel coordinates of the images acquired by the camera to the Cartesian coordinate system, extract feature points and perform stereo matching on the target images acquired by the left and right cameras to obtain the disparity information of the corresponding feature points, and calculate the three-dimensional position coordinates of the target in the coordinate system by combining the camera baseline length and focal length parameters through the principle of triangulation.

[0036] Optionally, the three-dimensional velocity vector can be obtained through position difference calculation between adjacent frames, and noise can be suppressed by Kalman filtering. The three-dimensional acceleration vector can be acquired by the three-axis accelerometer of the inertial measurement unit in sensor 101. The motion direction angle can include the horizontal direction angle and the vertical direction angle, which can be calculated by the tactile warning terminal 100 through the angle between the velocity vector projections, respectively; the motion angular velocity can be acquired by the three-axis gyroscope in sensor 101, and the motion angular velocity can be used to characterize the rate of change of the target's motion direction.

[0037] Optionally, virtual space attribute parameters can be used to describe the virtual early warning space and the inherent characteristics of the hardware. These parameters may include the virtual boundary mathematical model, the hierarchical division of the early warning area, and the inherent parameters of the ultrasonic array. The virtual boundary can be modeled using a non-uniform rational B-spline curve, with its shape defined by control point coordinates and weighting factors, adapting to various boundary scenarios. The curve equation for the virtual boundary is as follows:

[0038] In the formula, For non-uniform rational B-spline curves, it is used to define the boundary shape; For parameter variables, ; These are weighting factors used to influence the shape of the curve; These are the coordinates of the control points, used to control the direction of the curve. for The B-spline basis function is used; based on the above real-time motion state data, a multi-dimensional target real-time motion state vector is constructed, which is:

[0039]

[0040] In the formula, Let be the three-dimensional position coordinates of the target in the Cartesian coordinate system; These are the velocity vector components of the target along the three coordinate axes; These are the acceleration vector components of the target along the three coordinate axes; The horizontal direction angle of the target. The vertical direction angle of the target. The instantaneous angular velocity of the target.

[0041] S2. Based on the target's real-time motion state vector, predict the motion trajectory to obtain the predicted trajectory sequence.

[0042] For example, the tactile warning terminal 100 can construct a target motion model based on an extended Kalman filter. Assuming that the target motion follows a variable acceleration motion law, the following is the expression for the state transition matrix:

[0043]

[0044] In the formula, This is the state transition matrix; for The prior state vector at time t represents the predicted motion state without incorporating observation data; for The posterior state vector at time t; The control input matrix is ​​used to introduce the correction effect of external control variables on the prediction results. The matrix dimension is jointly determined by the state vector dimension and the control input vector dimension. for The control input vector at time t represents the intervention effect of the external environment on the motion of the early warning target; the prior error covariance matrix is ​​calculated to quantify the prediction uncertainty, and its expression is:

[0045]

[0046] In the formula, It is a symmetric square matrix, and the diagonal elements are the variances of the prediction errors of each state component; for The posterior error covariance matrix at time t. for The transpose of the matrix, The process noise covariance matrix; the update phase is based on time... The observed data is weighted using the Kalman gain matrix to correct the prior vector and obtain the posterior state vector. and error matrix This completes one iteration; repeated iterations are used to extrapolate the posterior vectors at multiple future time points, extract the positional components, sort them by time, and form a predicted trajectory sequence.

[0047] S3. Based on the predicted trajectory sequence and virtual space attribute parameters, spatial early warning monitoring is carried out to obtain dynamic early warning monitoring information.

[0048] For example, the tactile warning terminal 100 can construct a mathematical model of the virtual boundary based on virtual space attribute parameters, and use a spatial surface equation to represent the contour of the virtual boundary, the expression of which is: 0, where: For three-dimensional spatial surface functions, Let be the spatial coordinate variable; this equation can be used to determine the relative relationship between any spatial point and the virtual boundary. When, the location point is located on the virtual boundary; when 0 or At this time, the location points are located on both sides of the virtual boundary, and the specific positive and negative correspondence is determined according to the coordinate system setting. During the spatial early warning monitoring process, the tactile early warning terminal 100 can extract each predicted location point in the predicted trajectory sequence one by one, substitute its coordinates into the virtual boundary mathematical model and the early warning area definition standard, and determine whether the location point is located within the early warning area or is about to enter the early warning area; the monitoring logic can use a hierarchical judgment mechanism to identify the distance between the predicted location point and the virtual boundary, and the expression for the vertical distance from the observation point to the virtual boundary can be:

[0049]

[0050] In the formula, For observation point The vertical distance to the virtual boundary; , and Virtual boundary surface functions right The partial derivative is used to characterize the direction of the normal vector of the boundary surface at that point; The system compares the data with preset warning area thresholds to classify different warning levels, with higher warning levels for closer proximity. It also combines information on overlapping areas between the virtual space and the real environment to specially mark predicted location points within the overlapping areas, analyzes multiple spatial constraints in the area, and generates dynamic warning monitoring information. This dynamic warning monitoring information includes core components such as warning level, warning trigger time, spatial coordinate range of the warning area, overlapping area markings, and warning duration.

[0051] S4. Based on dynamic early warning monitoring information, predicted trajectory sequence and virtual space attribute parameters, beamforming control parameters are calculated through Doppler compensation and sound field parameter mapping.

[0052] For example, the tactile warning terminal 100 can perform Doppler compensation using the original transmission frequency of the ultrasonic array, and the expression for the compensated ultrasonic frequency can be:

[0053]

[0054] In the formula, The compensated ultrasonic frequency. This is the original transmission frequency of the ultrasonic array. This represents the speed at which ultrasound travels through the air. To indicate the speed of the target's movement. To provide early warning of the angle between the target's direction of motion and the direction of ultrasonic wave propagation. The speed of motion of the ultrasonic array transmitting unit. Let be the angle between the direction of motion of the ultrasonic array transmitting unit and the direction of ultrasonic wave propagation; the compensated frequency is calculated using the above formula; the sound field parameter mapping adopts a nonlinear mapping model to establish the correspondence between dynamic early warning monitoring information, predicted trajectory sequence and ultrasonic array sound field parameters. The sound field parameters may include transmission power, phase, focal point coordinates and beamwidth, etc. The expression of the mapping model is:

[0055]

[0056] In the formula, Sound field parameter vector, For dynamic early warning monitoring information vectors, To predict the trajectory sequence vector, This is a vector of virtual space attribute parameters. As a nonlinear mapping function, the tactile warning terminal 100 can be constructed using a neural network model or piecewise function fitting; the tactile warning terminal 100 can further transform the sound field parameter vector obtained by the above mapping into executable control commands for each transmitting unit based on the hardware structure of the ultrasonic array, thereby obtaining beamforming control parameters.

[0057] S5. Based on the beamforming control parameters, construct a dynamic sound field sequence that evolves continuously in the time and space dimensions.

[0058] For example, the tactile warning terminal 100 can construct a spatiotemporally continuous dynamic sound field sequence based on the real-time changes in beamforming control parameters. The construction of the dynamic sound field sequence is time-axis-based, combined with the spatial change trend of the predicted trajectory sequence, so that the ultrasonic focusing point moves synchronously with the predicted motion trajectory of the warning target. The sound field intensity and beam characteristics are dynamically adjusted according to the changes in the warning level. The expression for the change of the spatial coordinates of the sound field focusing point over time is:

[0059]

[0060] In the formula, for The coordinate vector of the focal point of the sound field at time t. This is the offset of the predicted time. for The predicted position coordinate vector of the target under constant warning; The focus point offset vector is set according to the virtual boundary position and warning level to ensure that the focus point is always located at a suitable position on the skin surface of the warning target; the dynamic adjustment of the sound field intensity adopts a gradient change mechanism, the expression of which is:

[0061]

[0062] In the formula, for Sound field intensity at any given moment Based on the basic sound field intensity, This is the warning level coefficient. The distance coefficient is used; the temporal continuity of the dynamic sound field sequence is ensured by the parametric interpolation algorithm. The beamforming control parameters between two adjacent moments are processed by linear interpolation or nonlinear interpolation. Spatial continuity is achieved by smoothing the movement trajectory of the focal point to ensure that the movement of the focal point in three-dimensional space is without jumps. At the same time, the beamwidth is adjusted according to the curvature of the predicted trajectory. The greater the curvature of the trajectory, the greater the beamwidth is appropriately increased to cover the possible movement range of the warning target. The dynamic sound field sequence is constructed in the above way.

[0063] S6. Input the dynamic sound field sequence into the human skin tactile perception model, perform matching mapping, and obtain the target tactile perception driving parameter set.

[0064] For example, the tactile warning terminal 100 can input a dynamic sound field sequence into a human skin tactile perception model. The human skin tactile perception model has a three-layer progressive structure: the perception input layer receives the dynamic sound field intensity vector, frequency, and environmental parameters, and eliminates dimensional differences after standardization; the feature extraction layer combines a convolutional neural network and a fully connected layer to extract the spatiotemporal features of the sound field and fuse them into a perception intermediate vector, embedding a perception delay correction module to dynamically adjust the weights and adapt to the influence of motion speed on perception; the mapping output layer maps the perception intermediate vector into a perception signal vector containing perception intensity, frequency, and stability through a nonlinear activation function. The following is the core expression of the human skin tactile perception model:

[0065]

[0066] In the formula, For sensing signal vectors, For a trained and calibrated nonlinear mapping function, This is a dynamic sound field intensity sequence vector. The frequency parameters after compensation, To perceive the delay time.

[0067] The training of the human skin tactile perception model can be divided into four stages: The data acquisition stage involves collecting tactile feedback data under multi-parameter stimulation from different populations using an ultrasound experimental platform, recording perception scores, latency, and other labels to construct training and validation datasets; the preprocessing stage involves removing outliers, normalizing data, and enhancing the dataset through interpolation and random perturbation to improve generalization ability; the training stage uses sound field parameters as input and perception labels as output, employing a backpropagation algorithm to optimize parameters, using mean squared error as the loss function, combined with regularization to suppress overfitting, and dynamically adjusting the learning rate to ensure convergence; and the validation stage tests the perception error rate, latency accuracy, and cross-population adaptability using independent datasets. If the performance does not meet the standards, additional data or structural adjustments are made until the performance criteria are met.

[0068] Optionally, the tactile warning terminal 100 can input dynamic sound field parameters into the trained model in a time sequence and output corresponding sensing signals. The tactile warning terminal 100 can adjust the model delay correction weights based on the motion state of the warning target to offset the sensing bias caused by motion. Based on the matched sensing signals, the tactile warning terminal 100 can further convert and generate a target tactile sensing driving parameter set, which may include transmit power fine-tuning, frequency correction values, focus point position calibration, and sensing stability parameters.

[0069] S7. Based on the target tactile perception driving parameter set, the warning signal is encoded using tactile coding rules to obtain the sound field tactile warning signal.

[0070] Optionally, the acoustic tactile warning signal can be used to achieve tactile warning by applying an ultrasonic array to the target skin.

[0071] For example, the tactile warning terminal 100 quantifies the contribution of each factor based on the target tactile perception driving parameter set, and calculates the dynamic encoding comprehensive weight. The expression for the dynamic encoding comprehensive weight can be:

[0072]

[0073] In the formula, For dynamic encoding of comprehensive weights, As a weight for the early warning level, Weighted by motion speed. For perceived delay weights, and satisfying ; This is the warning level coefficient. To determine the minimum distance from the early warning target to the virtual boundary, For real-time motion speed, To preset the maximum adaptation speed, To perceive the delay time; based on Encode using the following encoding functions:

[0074]

[0075] In the formula, This is the sound field tactile warning signal vector. For the target tactile perception driving parameter set, As a dynamic early warning information vector, To predict the trajectory sequence vector, It is a time-division-frequency-division hybrid coding function. The dynamic encoding comprehensive weight is used to accurately correct the encoding result; the encoded digital signal is converted into an analog signal that can be recognized by the ultrasonic array through digital-analog conversion, which is used to adjust the emission state of each transmitting unit, so that the ultrasonic energy is focused on the target skin, forming a continuous tactile stimulation that matches the warning level and motion state, and thus obtaining a sound field tactile warning signal.

[0076] This application provides a virtual space physical boundary tactile warning method based on an ultrasonic array. By using the real-time motion state data of the warning target and the virtual space attribute parameters, a real-time motion state vector of the target is constructed and the motion trajectory is predicted. Combined with Doppler frequency shift compensation and sound field parameter mapping technology, beamforming control parameters that match the user's motion trend and spatial distance are dynamically generated. This allows the construction of a dynamic sound field sequence that evolves continuously in time and space. After mapping through a human skin tactile perception model and tactile coding rule conversion, a high-precision sound field tactile warning signal that can be applied to the target skin is finally output. This enables an advanced, smooth, and motion-coordinated tactile warning prompt when the user approaches or is about to touch the virtual boundary, significantly reducing the problems of spatiotemporal misalignment, feedback lag, and perception conflict caused by system delay and lack of motion prediction in traditional reactive tactile feedback.

[0077] Based on the above embodiments, motion trajectory prediction is performed based on the target's real-time motion state vector to obtain a predicted trajectory sequence, including:

[0078] S11. Input the target's real-time motion state vector into a Kalman filter for noise reduction to obtain denoised state data.

[0079] For example, the tactile warning terminal can construct an observation vector with the same dimension as the state vector and calculate the Kalman gain matrix. The expression for the Kalman gain matrix can be:

[0080]

[0081] In the formula, for The Kalman gain matrix at time 10:00. Let be the prior error covariance matrix. For the observation matrix, for The transpose of the matrix, To observe the noise covariance matrix, the Kalman denoising matrix is ​​calculated by inputting the Kalman gain matrix into the following expression:

[0082]

[0083] In the formula, for Kalman denoising matrix at time step for The prior state estimate vector at time t. for The Kalman gain matrix at time 10:00. for The observation vector at time t; input the Kalman gain matrix into the following expression to obtain the posterior error covariance matrix:

[0084]

[0085] In the formula, for The posterior error covariance matrix at time t. It is the identity matrix. Here is the Kalman gain matrix. For the observation matrix, Let be the prior error covariance matrix.

[0086] S12. Input the denoised state data into the state transition model, perform the prediction step operation, and obtain the predicted state and the prediction covariance matrix.

[0087] For example, the haptic warning terminal can input noise reduction status data into the following state transition equation to calculate the predicted state:

[0088]

[0089] In the formula, for The prior state vector at each time step, i.e., the predicted state; This is a general state evolution function that can be flexibly defined according to the motion characteristics of the early warning target. for Kalman denoising matrix at time step The external control vector is used; the formula for calculating the prediction covariance matrix can be:

[0090]

[0091] In the formula, To predict the covariance matrix; Let be the state evolution function. right The Jacobian matrix is ​​used to transmit the error of the denoised state data. In linear scenarios, the Jacobian matrix is ​​equivalent to the traditional state transition matrix, while in nonlinear scenarios it is calculated through the partial derivative matrix. for The posterior error covariance matrix at time t. Let be the process noise covariance matrix.

[0092] S13. Input the predicted state and the predicted covariance matrix into the iterative state transition model, and predict the trajectory based on the predicted state and the predicted covariance matrix to obtain the initial predicted trajectory sequence.

[0093] For example, the tactile warning terminal can input the predicted state and the predicted covariance matrix into an iterative state transition model. The iterative state transition model can include a prediction unit, an error verification unit, and an iterative control unit. The prediction unit, error verification unit, and iterative control unit can be connected sequentially. The tactile warning terminal can input the predicted state into the prediction unit and perform iterative calculations. The expression for the predicted state input into the prediction unit can be:

[0094]

[0095] In the formula, for Prior prediction vector at time step, Let be the time-varying state evolution function. For the first iteration Prior prediction vector at time step, for The control input vector at time t, For the sampling time step, For the iterative decay coefficient, The iteration number is used; the tactile warning terminal can input the results of the iterations into the error verification unit to predict the covariance matrix and monitor uncertainty. The formula for calculating the covariance matrix is:

[0096]

[0097] In the formula, For the first The next iteration yields The prior error covariance matrix of the predicted state at time step. Evolution function In the current state Jacobian matrix at the location, For the first The corresponding one used in the next iteration The prior error covariance matrix at time t. Let be the process noise covariance matrix; if If the prior error covariance matrix trace of the predicted state at any given time exceeds a threshold, gain calibration correction is triggered. After iterating to a preset number of times, the three-dimensional position components of the predicted state at each time time are extracted, sorted in time sequence, and smoothed by moving average to obtain the initial predicted trajectory sequence.

[0098] S14. Obtain historical state data of the early warning target, and extract features from the historical state data to obtain historical state features.

[0099] For example, the tactile warning terminal can acquire historical state data of the warning target. Based on a time-domain feature analysis method, the tactile warning terminal can perform smoothing preprocessing on the historical data and extract features to obtain historical state features. The expression for the historical state features can be:

[0100]

[0101] In the formula, for Class feature quantization value, The number of data points within the sliding window. For the first in the window Each state component data; For feature type mapping function, As the time-series decay weight; by adjusting By extracting multiple features, the tactile warning terminal can extract position features based on a statistical mapping function, velocity features based on a rate of change mapping function, and acceleration features based on an impact mapping function. After extracting historical state features, the tactile warning terminal can calculate the correlation coefficients between these features, remove redundant features, and standardize them to obtain a fixed-dimensional historical state feature vector.

[0102] For example, the tactile warning terminal can use a moving average algorithm to smooth historical data and eliminate accidental interference.

[0103] For example, the time window of historical data needs to cover a sufficiently long period to fully capture the motion pattern, and the data content should be consistent with the real-time state vector, covering the position, velocity and acceleration components at each historical moment.

[0104] S15. Input the historical state features into a shallow neural network to obtain the probability distribution data of motion intention.

[0105] For example, a shallow neural network can adopt a three-layer structure of "input layer-hidden layer-output layer" to meet the needs of rapid recognition of motion intentions. The number of nodes in the input layer is consistent with the dimension of the historical state feature vector. The hidden layer uses the ReLU activation function, and the number of nodes is optimized through cross-validation. The output layer uses the Softmax activation function, and the number of nodes corresponds to the preset motion intention category. The tactile warning terminal can input the historical state features into the input layer of the shallow neural network to normalize the feature vector and obtain normalized features. The tactile warning terminal can then input the normalized features into the hidden layer to perform nonlinear fusion of the normalized features and obtain fused features. Finally, the tactile warning terminal can input the fused features into the output layer to generate motion intention probability values ​​for various motion intentions and construct motion intention probability distribution data based on these probability values.

[0106] S16. Perform a weighted mapping on the probability distribution data of motion intentions to obtain the direction bias vector.

[0107] For example, the haptic warning terminal can set a directional reference vector corresponding to the motion intention probability distribution data based on the motion intention probability distribution data. The haptic warning terminal can set a pointing-to-boundary unit vector corresponding to the near-boundary intention probability based on the near-boundary intention probability, and a deviating-from-boundary unit vector corresponding to the far-boundary intention probability based on the far-from-boundary intention probability. The haptic warning terminal can also set zero vectors corresponding to the parallel-boundary intention probability and the irregular motion intention probability based on the parallel-boundary intention probability or the irregular motion intention probability. The haptic warning terminal can use the various probabilities in the motion intention probability distribution data as weights to perform a weighted summation of the directional reference vectors corresponding to the motion intention probability distribution data. The formula for calculating the directional bias vector can be:

[0108]

[0109] In the formula, For the number of movement intention categories, for The probability of class intent, for The directional reference vector corresponding to the class intent. This is the direction offset vector.

[0110] S17. Calculate the probability distribution of the motion intention probability distribution data to obtain the motion confidence level.

[0111] For example, the haptic warning terminal can extract the maximum probability value from the motion intention probability distribution data and calculate the probability distribution entropy value of the motion intention probability distribution data. The smaller the probability distribution entropy value, the more concentrated the probability distribution. The haptic warning terminal can calculate the motion confidence level using the following normalized expression:

[0112]

[0113] In the formula, The maximum probability value, The entropy value of the probability distribution. For motion confidence.

[0114] S18. Based on the orientation bias vector and motion confidence, the initial predicted trajectory sequence is corrected to obtain the predicted trajectory sequence.

[0115] For example, the haptic warning terminal can input the orientation bias vector and motion confidence into a correction formula for correction. The expression of the correction formula is as follows:

[0116]

[0117] In the formula, for The trajectory coordinates after real-time correction These are the initial trajectory coordinates. For motion confidence, This is the direction offset vector. This is the distance coefficient, representing the distance from the warning target to the virtual boundary. calculate, The coordinates of the initial predicted trajectory sequence are adjusted one by one to verify whether the corrected trajectory conforms to the constraints between the virtual space and the real environment. If the corrected coordinates exceed a reasonable range, boundary constraint calibration is triggered, and the weights of the direction offset vector are adjusted. After the correction is completed, the predicted trajectory sequence is obtained.

[0118] This application provides a virtual space physical boundary tactile early warning method based on ultrasonic arrays. Through a closed-loop process of "noise reduction-prediction-intent recognition-trajectory correction", it can achieve high-precision motion trajectory prediction. It relies on Kalman filtering to effectively remove noise from state data and ensure the reliability of basic data. It also uses a general state evolution function and a three-layer iterative model to be compatible with linear and nonlinear motion scenarios, suppress iterative accumulation error, and expand the scope of technology adaptability. At the same time, it integrates historical state feature extraction and motion intent recognition. With the help of feature quantization formulas and probabilistic analysis models, it accurately outputs motion intent bias and confidence. Through a weighted correction mechanism, it achieves deep integration of kinematic inference and intent prediction, effectively avoiding the limitations of single trajectory prediction. It significantly improves the accuracy and stability of trajectory prediction in dynamic scenarios, and provides high-quality data support for subsequent spatial early warning monitoring, dynamic sound field construction, and tactile early warning signal generation, ensuring the timeliness and accuracy of the overall tactile early warning system.

[0119] Based on the above embodiments, historical state features are input into a shallow neural network to obtain motion intent probability distribution data, including:

[0120] S21. Normalize the historical state features to obtain standardized historical state features.

[0121] For example, the tactile warning terminal can normalize historical state features. The tactile warning terminal can use the Z-score normalization method to eliminate the dimensional differences between different feature dimensions. The expression for normalizing historical state features can be:

[0122]

[0123] In the formula, These are the normalized standard components. These are the original historical state feature components. This represents the mean of the corresponding feature dimension. To obtain the standard deviation of the corresponding feature dimension, we calculate it one by one by traversing all components of the historical state feature vector to obtain the standardized historical state feature with the same dimension as the original feature vector.

[0124] S22. Enhance the standardized historical state features to obtain the enhanced feature vector.

[0125] For example, the tactile warning terminal can perform pairwise cross operations on standardized historical state features to generate feature interaction terms to capture the correlation between features of different dimensions; the tactile warning terminal can also extend the standardized historical state features through nonlinear activation function mapping to obtain an enhanced feature vector, the expression of which can be:

[0126]

[0127] In the formula, To enhance the feature vector, The pairwise cross feature matrix of the standardized features. The cross-feature weight matrix, For bias terms, It is a non-linear activation function; after enhancement, the feature vector is dimensionally normalized to ensure that it matches the number of nodes in the input layer of the shallow neural network.

[0128] S23. Input the enhanced feature vector into the shallow neural network to generate motion intention probability distribution data.

[0129] For example, the haptic warning terminal can input the enhanced feature vector into a shallow neural network. The shallow neural network can adopt a three-layer structure consisting of an input layer, a hidden layer, and an output layer connected in sequence. The number of nodes in the input layer is consistent with the dimension of the enhanced feature vector. The hidden layer performs non-linear fusion of the enhanced features through a weight matrix and a bias term, and outputs an intermediate feature vector. The output layer uses the Softmax activation function to map the intermediate features into probability values ​​of various motion intentions, generating motion intention probability distribution data.

[0130] Alternatively, the expression for the intermediate features of the hidden layer of a shallow neural network can be:

[0131]

[0132] In the formula, For intermediate feature vectors, To enhance the feature vector, Here is the weight matrix of the hidden layer. The bias vector of the hidden layer. The ReLU activation function is non-linear; the input layer calculates the motion intent probability distribution data using the following expression:

[0133]

[0134] In the formula, This is data on the probability distribution of movement intentions. The weight matrix of the output layer. For intermediate feature vectors, This is the bias vector for the output layer.

[0135] This application provides a virtual space physical boundary tactile early warning method based on ultrasonic arrays. Through progressive processing of normalization, feature enhancement, and network inference, it effectively eliminates feature dimension differences and redundant interference. It enhances the representation power of motion features by feature cross-fusion and nonlinear mapping, and accurately outputs motion intention probability distribution data based on shallow neural networks. This not only improves the accuracy and stability of motion intention recognition, but also provides high-quality data support for subsequent trajectory correction, ensuring the reliability and adaptability of the overall trajectory prediction process.

[0136] Based on the above embodiments, the shallow neural network may include a shallow feature mapping module, a regularization optimization module, and a probability output module. The enhanced feature vector is input into the shallow neural network to generate motion intent probability distribution data, which may include:

[0137] Specifically, the tactile warning terminal can input the enhanced feature vector into the shallow feature mapping module of the shallow neural network, and perform nonlinear feature extraction and dimensionality upscaling mapping on the enhanced feature vector to obtain a high-dimensional feature mapping vector.

[0138] For example, the haptic warning terminal can input the enhanced feature vector into the shallow feature mapping module of a shallow neural network, and use a weight matrix to upscale the enhanced feature vector to a preset high-dimensional space, ensuring that the enhanced feature vector can adapt to the needs of distinguishing motion intention categories. The haptic warning terminal can extract deep-level related features through a non-linear activation function to obtain a high-dimensional feature mapping vector. The calculation expression for the high-dimensional feature mapping vector can be:

[0139]

[0140] In the formula, It is a high-dimensional feature mapping vector; The dimensions of the weight matrix are set based on the dimensions of the enhanced feature vectors and the preset high-dimensional space dimensions. For mapping bias terms, It is a nonlinear ReLU activation function;

[0141] Specifically, the tactile warning terminal can input the high-dimensional feature mapping vector into the regularization optimization module, and suppress overfitting by randomly discarding the regularization operation to obtain the optimized high-dimensional feature vector.

[0142] For example, the tactile warning terminal can generate a binary mask vector with the same dimension as the high-dimensional feature mapping vector. The elements in the binary mask vector can be randomly selected as 0 or 1 according to the discard probability. Selecting 0 can indicate that the output of the corresponding neuron is discarded, and selecting 1 can indicate that it is retained. The tactile warning terminal can multiply the high-dimensional feature mapping vector and the mask vector element by element to obtain the optimized high-dimensional feature vector.

[0143] Specifically, the tactile warning terminal can map the optimized high-dimensional feature vector to a dimensional space that matches the number of motion intention categories to obtain the mapped vector value.

[0144] For example, a tactile warning terminal can project an optimized high-dimensional feature vector onto a dimensional space consistent with the number of motion intention categories through a linear mapping operation to obtain a mapped vector value. The formula for calculating the mapped vector value can be:

[0145]

[0146] In the formula, The value is a mapping vector, and its dimension is equal to the number of motion intent categories. The projection weight matrix corresponds to the optimized high-dimensional feature vector dimension and the number of motion intent categories; This is the projection bias term, calibrated through model pre-training.

[0147] Specifically, the tactile warning terminal can input the mapping vector value into the probability output module, convert the mapping vector value into the motion intention probability value of each motion intention category, and summarize the motion intention probability values ​​of each motion intention category to obtain motion intention probability distribution data.

[0148] For example, the haptic warning terminal can input the mapping vector value into the probability output module to obtain the probability value of the motion intention. The formula for calculating the probability value of the motion intention can be:

[0149]

[0150] In the formula, For the first The probability value of the type of motion intention. For the mapping vector of the first The mapping vector values ​​of each component, The total number of movement intention categories. For the mapping vector of the first The mapping vector values ​​of each component; the probability values ​​of all categories are summarized in order of intent category to form the motion intent probability distribution data.

[0151] The virtual space physical boundary tactile early warning method based on ultrasonic arrays provided in this application relies on the synergistic effect of three modules of a shallow neural network to eliminate spatiotemporal misalignment, feedback lag, and perceptual conflicts, achieving advanced, smooth, and motion-coordinated high-precision tactile early warning. Through nonlinear dimensionality-upgrading mapping, enhanced feature discriminative power, and random dropout regularization, the method can improve the model's generalization ability and suppress overfitting. Through dimensionality adaptation and probability transformation, it can accurately translate features into standardized probability distributions, thus continuing the feature enhancement effects described earlier. Furthermore, through layered processing, it can improve the accuracy and stability of motion intent recognition. The probability distribution data output by the shallow neural network can directly provide high-quality input for subsequent orientation bias vector construction and motion confidence calculation, ensuring the reliability of the trajectory correction process.

[0152] In one embodiment of the present invention, spatial early warning monitoring is performed based on predicted trajectory sequences and virtual space attribute parameters to obtain dynamic early warning monitoring information, including:

[0153] S31. Based on the virtual space attribute parameters, construct a virtual boundary model with attached normal directions.

[0154] Specifically, the tactile warning terminal can use virtual spatial attribute parameters to cover spatial range, boundary shape, material properties, and safety zone definition standards.

[0155] For example, the tactile warning terminal can construct a virtual boundary model based on the virtual space attribute parameters using a polygonal surface modeling method. During the modeling process, the normal direction of the boundary surface is defined simultaneously, with the direction from the inside of the boundary to the outside being the positive normal direction. The normal vector magnitude is unified to 1 through vector normalization processing, thus clarifying the relationship between the spatial position and the inside and outside of the boundary.

[0156] S32. Based on the predicted trajectory sequence and the virtual boundary model, the shortest normal distance from each predicted location point to the virtual boundary surface is calculated using a geometric projection algorithm.

[0157] For example, the tactile warning terminal can perform geometric projection operations on each predicted location point using a constructed virtual boundary model. The tactile warning terminal can determine the boundary surface region corresponding to the predicted location point, and can project the predicted location point onto the boundary surface region along the boundary normal direction to obtain the coordinates of the projected landing point. The tactile warning terminal can calculate the straight-line distance between the predicted location point and the projected landing point using the distance formula between two points in space to obtain the shortest normal distance.

[0158] S33. Based on the predicted trajectory sequence and the virtual boundary model, the normal approach velocity is calculated using the vector dot product projection algorithm.

[0159] For example, the tactile warning terminal can calculate the adjacent predicted position displacement vectors of two adjacent predicted position points based on the predicted trajectory sequence. The adjacent predicted position displacement vectors can represent the motion trend per unit time. The tactile warning terminal can extract the normal vector of the boundary region of the virtual boundary model. The tactile warning terminal can project the displacement vector onto the normal direction through vector dot product operation. The dot product result can be the normal velocity component. Positive values ​​indicate moving away from the boundary, and negative values ​​indicate moving towards the boundary. The tactile warning terminal can take the absolute value of the dot product result to obtain the normal approach velocity.

[0160] S34. Input the normal approach velocity into the dynamic threshold function to calculate the dynamic warning distance.

[0161] Alternatively, the expression for the dynamic threshold function can be:

[0162]

[0163] in, For dynamic early warning distance, Let τ be the static warning distance, and τ be the estimated total system delay. The normal approach velocity, This is the correlation coefficient for human braking.

[0164] For example, the dynamic threshold function can dynamically adjust the warning threshold based on system latency, human braking characteristics, and normal approach velocity, making it more suitable for real-time motion scenarios compared to a fixed static threshold.

[0165] S35. Based on the shortest normal distance and dynamic warning distance, a warning trigger judgment is made to obtain the warning trigger judgment result.

[0166] For example, the tactile warning terminal can compare the shortest normal distance to each time-series predicted location point with the corresponding dynamic warning distance. When the absolute value of the shortest normal distance is less than or equal to the dynamic warning distance, the tactile warning terminal can determine that the warning triggering condition is met and mark the current time-series predicted location point as being in a warning state. When the absolute value of the shortest normal distance is greater than the dynamic warning distance, the tactile warning terminal can determine that there is no warning risk and mark the current time-series predicted location point as being in a normal state. The tactile warning terminal can simultaneously record the judgment result of each time-series predicted location point to construct a time-series warning trigger sequence.

[0167] S36. Extract the timing information of the early warning trigger judgment result to obtain the early warning start time.

[0168] For example, the tactile warning terminal can perform a traversal analysis of the time-series warning trigger sequence to extract the time point at which the warning state first appears. The tactile warning terminal can determine the warning start time by combining the timestamps corresponding to the predicted trajectory sequence. If there are multiple consecutive warning state time points, the tactile warning terminal can use the time of the first trigger point as the warning start time; if there is no warning state throughout, the tactile warning terminal can mark the time-series warning trigger sequence as having no warning trigger.

[0169] S37. Integrate the early warning start time, early warning trigger location, dynamic early warning distance, shortest normal distance and normal approach speed to construct dynamic early warning monitoring information.

[0170] For example, the tactile warning terminal can integrate the warning start time, warning trigger position, dynamic warning distance, shortest normal distance, and normal approach velocity calculated and extracted from each stage in a temporal sequence: the tactile warning terminal can set the warning trigger position based on the three-dimensional coordinates of the predicted position point that triggers the warning for the first time, and the tactile warning terminal can set the dynamic warning distance, shortest normal distance, and normal approach velocity based on the quantified parameters of the predicted position point. The tactile warning terminal can set the warning start time as the temporal reference, and integrate the warning start time, warning trigger position, dynamic warning distance, shortest normal distance, and normal approach velocity to construct structured dynamic warning monitoring information. The structured dynamic warning monitoring information can include dynamic warning monitoring information with multiple dimensions, including temporal dimension, spatial dimension, and risk quantification.

[0171] The virtual space physical boundary tactile early warning method based on ultrasonic array provided in this application, through a closed-loop process of boundary modeling, multi-dimensional quantization calculation, dynamic early warning judgment and information integration, can accurately obtain the normal distance and approach speed based on a virtual boundary model with attached normal direction. Combined with system delay and human braking characteristics, it can realize dynamic early warning distance adaptive adjustment, which can significantly improve the early warning adaptability and accuracy compared with a fixed threshold. By finally integrating multi-dimensional core parameters to form structured dynamic early warning monitoring information, it can provide comprehensive temporal, spatial and risk quantification support for subsequent dynamic sound field construction and tactile early warning signal generation, ensuring the timeliness, reliability and scene adaptability of the overall early warning system.

[0172] Based on the above embodiments, and using dynamic early warning monitoring information, predicted trajectory sequences, and virtual space attribute parameters, beamforming control parameters are obtained through Doppler compensation and sound field parameter mapping, including:

[0173] S41. Based on the warning start time in the dynamic warning monitoring information, extract the target warning location and target warning speed from the predicted trajectory sequence.

[0174] For example, the tactile warning terminal can extract the time-series timestamp of the predicted trajectory sequence corresponding to the warning start time in the dynamic warning monitoring information. The tactile warning terminal can match the corresponding time-series node in the predicted trajectory sequence based on the time-series timestamp. The tactile warning terminal can extract the three-dimensional coordinates of the time-series node as the target warning position. The target warning position can represent the spatial position when the warning is first triggered. The tactile warning terminal can extract the adjacent time-series displacement vector of the time-series node and its previous adjacent time-series node, and calculate the instantaneous velocity vector of the time-series node by combining it with the sampling time step, as the target warning velocity.

[0175] S42. Calculate the projection component of the target warning velocity in the direction of the line connecting the center of the ultrasonic array to the target warning position to obtain the radial relative velocity component.

[0176] Optionally, the virtual space attribute parameters may include the center coordinates of the ultrasonic array and array layout information.

[0177] For example, the tactile warning terminal can set the center of the ultrasonic array as the vector starting point and the target warning position as the vector ending point to construct a radial direction vector. This radial direction vector is then normalized to obtain a unit radial vector. The tactile warning terminal can perform a dot product operation between the target warning velocity vector and the unit radial vector to obtain the projection component of the target warning velocity in the radial direction. Based on this projection component, the tactile warning terminal can set the radial relative velocity component. A positive dot product result indicates that the target is moving away from the array, while a negative result indicates that the target is moving closer to the array.

[0178] S43. The Doppler-compensated transmission frequency is calculated based on the radial relative velocity component.

[0179] Alternatively, the expression for the Doppler-compensated transmission frequency can be:

[0180]

[0181] in, To compensate for the transmission frequency, To preset the optimal frequency for tactile perception. For the speed of sound, This represents the radial relative velocity component.

[0182] For example, the tactile warning terminal can compensate for the Doppler frequency shift caused by radial motion by using a correction factor. When the target is close to the array, the denominator of the Doppler compensation transmission frequency can be reduced, and the Doppler compensation transmission frequency can be increased to compensate for the shift in the sensing frequency at the receiver. When the target is far away, the denominator of the Doppler compensation transmission frequency can be increased, and the transmission frequency of the Doppler compensation transmission frequency can be decreased.

[0183] S44. Based on the target warning location and virtual space attribute parameters, the beam pointing parameters are calculated using a coordinate transformation algorithm.

[0184] For example, the haptic warning terminal can obtain the azimuth and elevation angles through a coordinate transformation algorithm. The expression for calculating the azimuth and elevation angles using the coordinate transformation algorithm can be:

[0185]

[0186]

[0187] In the formula, The azimuth angle represents the deflection angle of the target relative to the array's x-axis reference on the horizontal plane, and its value ranges from [value missing]. , The elevation angle represents the height of the target relative to the array plane on the vertical plane, and its value ranges from [value missing]. The azimuth and elevation angles are used as beam pointing parameters.

[0188] S45. Based on the shortest normal distance and normal approach velocity, and combined with the tactile perception intensity mapping function, the sound pressure intensity parameters are calculated.

[0189] For example, the tactile warning terminal can calculate the sound pressure intensity parameter through a mapping function. The formula for calculating the sound pressure intensity parameter can be:

[0190]

[0191] In the formula, For sound pressure intensity parameters, As the reference sound pressure level, For speed weighting coefficients, This is the distance weighting coefficient. The normal approach velocity, To calculate the shortest normal distance, the shortest normal distance and normal approach velocity are input into the dynamic early warning monitoring information. The risk quantification index is then converted into sound pressure intensity parameters through a mapping function.

[0192] S46. Integrate Doppler compensation transmission frequency, beam pointing parameters, and sound pressure intensity parameters to generate beamforming control parameters.

[0193] For example, the tactile warning terminal can integrate Doppler-compensated transmission frequency in terms of frequency control, direction control, and intensity control. The Doppler-compensated transmission frequency is used to control the transmission frequency of the ultrasonic array, the beam pointing parameter can be used to control the phase difference of each unit in the array to achieve beam focusing, and the sound pressure level parameter can be used to control the transmission power of the array. The tactile warning terminal can integrate the Doppler-compensated transmission frequency, beam pointing parameter, and sound pressure level parameter, and add parameter verification identifiers to generate standardized beamforming control parameters. The standardized beamforming control parameters can be used to drive the ultrasonic array to output tactile feedback signals adapted to the warning scenario.

[0194] The virtual space physical boundary tactile early warning method based on ultrasonic array provided in this application accurately extracts the spatiotemporal parameters of the early warning target, quantifies the radial relative velocity to compensate for Doppler frequency shift, combines coordinate transformation to calibrate beam pointing, and maps graded sound pressure intensity according to risk level. Finally, it integrates to form a three-dimensional beamforming control parameter of "frequency-direction-intensity", effectively eliminating frequency deviation caused by motion, realizing precise focusing of ultrasonic beam and adaptive adjustment of tactile feedback intensity. It not only ensures the stability and pointing accuracy of tactile perception, but also provides differentiated feedback to match different early warning risk levels. It provides core control support for the ultrasonic array to output high-quality tactile early warning signals, significantly improving the adaptability, accuracy and user experience of the overall system.

[0195] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0196] Based on the same inventive concept, this application also provides an ultrasonic array-based virtual space physical boundary tactile warning system for implementing the aforementioned ultrasonic array-based virtual space physical boundary tactile warning method. The solution provided by this system is similar to the implementation described in the above method. Therefore, the specific limitations of one or more ultrasonic array-based virtual space physical boundary tactile warning system embodiments provided below can be found in the above-described limitations of the ultrasonic array-based virtual space physical boundary tactile warning method, and will not be repeated here.

[0197] In one exemplary embodiment, such as Figure 3 As shown, a virtual space physical boundary tactile early warning system based on an ultrasonic array is provided to implement the methods in the above-described embodiments, including:

[0198] The acquisition module 501 is used to acquire real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data.

[0199] The trajectory prediction module 502 is used to predict the motion trajectory based on the real-time motion state vector of the target and obtain the predicted trajectory sequence.

[0200] The monitoring information generation module 503 is used to perform spatial early warning monitoring based on the predicted trajectory sequence and virtual space attribute parameters to obtain dynamic early warning monitoring information.

[0201] The control parameter generation module 504 is used to obtain beamforming control parameters based on dynamic early warning monitoring information, predicted trajectory sequence and virtual space attribute parameters, through Doppler compensation and sound field parameter mapping.

[0202] The dynamic sound field construction module 505 is used to construct a dynamic sound field sequence that evolves continuously in the time and space dimensions based on beamforming control parameters.

[0203] The mapping module 506 is used to match and map the dynamic sound field sequence and the human skin tactile perception model to obtain the target tactile perception driving parameter set.

[0204] The output module 507 is used to encode and convert the warning signal according to the target tactile perception driving parameter set to obtain the sound field tactile warning signal, wherein the sound field tactile warning signal is used to act on the skin of the warning target through the ultrasonic array.

[0205] In one embodiment, the orbit prediction module 502 can also be used for:

[0206] S11. Input the target's real-time motion state vector into a Kalman filter for noise reduction to obtain denoised state data.

[0207] S12. Input the denoised state data into the state transition model, perform the prediction step operation, and obtain the predicted state and the prediction covariance matrix.

[0208] S13. Input the predicted state and the predicted covariance matrix into the iterative state transition model, and predict the trajectory based on the predicted state and the predicted covariance matrix to obtain the initial predicted trajectory sequence.

[0209] S14. Obtain historical state data of the early warning target, and extract features from the historical state data to obtain historical state features.

[0210] S15. Input the historical state features into a shallow neural network to obtain the probability distribution data of motion intention.

[0211] S16. Perform a weighted mapping on the probability distribution data of motion intentions to obtain the direction bias vector.

[0212] S17. Calculate the probability distribution of the motion intention probability distribution data to obtain the motion confidence level:

[0213] S18. Based on the orientation bias vector and motion confidence, the initial predicted trajectory sequence is corrected to obtain the predicted trajectory sequence.

[0214] Based on the above embodiments, the orbit prediction module 502 can also be used for:

[0215] S21. Normalize the historical state features to obtain standardized historical state features.

[0216] S22. Enhance the standardized historical state features to obtain the enhanced feature vector.

[0217] S23. Input the enhanced feature vector into the shallow neural network to generate motion intention probability distribution data.

[0218] In one embodiment of the present invention, the monitoring information generation module 503 can also be used for:

[0219] S31. Based on the virtual space attribute parameters, construct a virtual boundary model with attached normal directions.

[0220] S32. Based on the predicted trajectory sequence and the virtual boundary model, the shortest normal distance from each predicted location point to the virtual boundary surface is calculated using a geometric projection algorithm.

[0221] S33. Based on the predicted trajectory sequence and the virtual boundary model, the normal approach velocity is calculated using the vector dot product projection algorithm.

[0222] S34. Input the normal approach velocity into the dynamic threshold function to calculate the dynamic warning distance.

[0223] S35. Based on the shortest normal distance and dynamic warning distance, a warning trigger judgment is made to obtain the warning trigger judgment result.

[0224] S36. Extract the timing information of the early warning trigger judgment result to obtain the early warning start time.

[0225] S37. Integrate the early warning start time, early warning trigger location, dynamic early warning distance, shortest normal distance and normal approach speed to construct dynamic early warning monitoring information.

[0226] Based on the above embodiments, the monitoring information generation module 503 can also be used for:

[0227] S41. Based on the warning start time in the dynamic warning monitoring information, extract the target warning location and target warning speed from the predicted trajectory sequence.

[0228] S42. Calculate the projection component of the target warning velocity in the direction of the line connecting the center of the ultrasonic array to the target warning position to obtain the radial relative velocity component.

[0229] S43. The Doppler-compensated transmission frequency is calculated based on the radial relative velocity component.

[0230] S44. Based on the target warning location and virtual space attribute parameters, the beam pointing parameters are calculated using a coordinate transformation algorithm.

[0231] S45. Based on the shortest normal distance and normal approach velocity, and combined with the tactile perception intensity mapping function, the sound pressure intensity parameters are calculated.

[0232] S46. Integrate Doppler compensation transmission frequency, beam pointing parameters, and sound pressure intensity parameters to generate beamforming control parameters.

[0233] In one embodiment, this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0234] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0235] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0236] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for ultrasonic array based virtual space physical boundary tactile warning, characterized in that, The method includes: S1. Obtain the real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data; S2. Based on the real-time motion state vector of the target, predict the motion trajectory to obtain a predicted trajectory sequence; S3. Based on the predicted trajectory sequence and the virtual space attribute parameters, perform spatial early warning monitoring to obtain dynamic early warning monitoring information; S4. Based on the dynamic early warning monitoring information, the predicted trajectory sequence, and the virtual space attribute parameters, beamforming control parameters are calculated through Doppler compensation and sound field parameter mapping. S5. Based on the beamforming control parameters, construct a dynamic sound field sequence that continuously evolves in the time and space dimensions; S6. Input the dynamic sound field sequence into the human skin tactile perception model, perform matching mapping, and obtain the target tactile perception driving parameter set; S7. Based on the target tactile perception driving parameter set, the warning signal is encoded using tactile coding rules to obtain a sound field tactile warning signal; wherein, the sound field tactile warning signal is used to achieve tactile warning by acting on the target skin through an ultrasonic array.

2. The method of claim 1, wherein, The step of predicting the motion trajectory based on the real-time motion state vector of the target to obtain a predicted trajectory sequence includes: S11. Input the real-time motion state vector of the target into a Kalman filter for noise reduction to obtain noise-reduced state data. S12. Input the noise reduction state data into the state transition model, perform prediction step calculation, and obtain the prediction state and prediction covariance matrix; S13. Input the predicted state and the predicted covariance matrix into the iterative state transition model, and predict the trajectory based on the predicted state and the predicted covariance matrix to obtain the initial predicted trajectory sequence; S14. Obtain historical state data of the warning target, and extract features from the historical state data to obtain historical state features; S15. Input the historical state features into a shallow neural network to obtain motion intention probability distribution data; S16. Perform a weighted mapping on the motion intention probability distribution data to obtain a direction bias vector; S17. Calculate the probability distribution of the motion intention probability distribution data to obtain the motion confidence level: S18. Based on the orientation bias vector and the motion confidence, the initial predicted trajectory sequence is corrected to obtain the predicted trajectory sequence.

3. The method of claim 2, wherein, The step of inputting the historical state features into a shallow neural network to obtain motion intent probability distribution data includes: S21. Normalize the historical state features to obtain standardized historical state features; S22. Perform feature enhancement on the standardized historical state features to obtain an enhanced feature vector; S23. Input the enhanced feature vector into the shallow neural network to generate the motion intention probability distribution data.

4. The method according to claim 3, characterized in that, The shallow neural network includes a shallow feature mapping module, a regularization optimization module, and a probability output module. The step of inputting the enhanced feature vector into the shallow neural network to generate the motion intent probability distribution data includes: The enhanced feature vector is input into the shallow feature mapping module of the shallow neural network, and nonlinear feature extraction and dimension upscaling are performed on the enhanced feature vector to obtain a high-dimensional feature mapping vector. The high-dimensional feature mapping vector is input into the regularization optimization module, and overfitting is suppressed by random discarding regularization operation to obtain the optimized high-dimensional feature vector; The optimized high-dimensional feature vector is mapped to a dimensional space that matches the number of motion intention categories to obtain the mapped vector value; The mapping vector value is input into the probability output module, which converts the mapping vector value into a motion intent probability value for each motion intent category, and then summarizes the motion intent probability values ​​for each motion intent category to obtain the motion intent probability distribution data.

5. The method according to claim 1, characterized in that, The spatial early warning monitoring based on the predicted trajectory sequence and the virtual space attribute parameters, to obtain dynamic early warning monitoring information, includes: S31. Based on the virtual space attribute parameters, construct a virtual boundary model with attached normal directions; S32. Based on the predicted trajectory sequence and the virtual boundary model, the shortest normal distance from each predicted location point to the virtual boundary surface is calculated using a geometric projection algorithm. S33. Based on the predicted trajectory sequence and the virtual boundary model, the normal approach velocity is calculated using the vector dot product projection algorithm. S34. Input the normal approach velocity into a dynamic threshold function to calculate the dynamic warning distance; wherein, the expression of the dynamic threshold function is: in, For dynamic early warning distance, Let τ be the static warning distance, and τ be the estimated total system delay. The normal approach velocity, The correlation coefficient for human braking; S35. Based on the shortest normal distance and the dynamic warning distance, a warning trigger judgment is made to obtain the warning trigger judgment result; S36. Extract the timing information of the warning trigger judgment result to obtain the warning start time; S37. Integrate the warning start time, the warning trigger location, the dynamic warning distance, the shortest normal distance, and the normal approach speed to construct the dynamic warning monitoring information.

6. The method according to claim 5, characterized in that, The beamforming control parameters are obtained based on the dynamic early warning monitoring information, the predicted trajectory sequence, and virtual space attribute parameters, through Doppler compensation and sound field parameter mapping, including: S41. Based on the warning start time in the dynamic warning monitoring information, extract the target warning location and target warning speed from the predicted trajectory sequence; S42. Calculate the projection component of the target warning velocity in the direction of the line connecting the center of the ultrasonic array to the target warning position to obtain the radial relative velocity component; S43. The Doppler-compensated transmission frequency is calculated based on the radial relative velocity component. The expression for the Doppler-compensated transmission frequency is: in, For the Doppler compensation transmission frequency, To preset the optimal frequency for tactile perception. For the speed of sound, The radial relative velocity component; S44. Based on the target warning location and the virtual space attribute parameters, the beam pointing parameters are calculated using a coordinate transformation algorithm. S45. Based on the shortest normal distance and the normal approach velocity, and combined with the tactile perception intensity mapping function, the sound pressure intensity parameters are calculated. S46. Integrate the Doppler compensation transmission frequency, the beam pointing parameter, and the sound pressure intensity parameter to generate the beamforming control parameters.

7. A virtual space physical boundary tactile early warning system based on an ultrasonic array, used to implement the method according to any one of claims 1 to 6, characterized in that, The system includes: The acquisition module is used to acquire real-time motion state data and virtual space attribute parameters of the warning target, and construct the real-time motion state vector of the warning target based on the real-time motion state data. The trajectory prediction module is used to predict the motion trajectory based on the real-time motion state vector of the target, and obtain a predicted trajectory sequence. The monitoring information generation module is used to perform spatial early warning monitoring based on the predicted trajectory sequence and the virtual space attribute parameters to obtain dynamic early warning monitoring information; The control parameter generation module is used to obtain beamforming control parameters based on the dynamic early warning monitoring information, the predicted trajectory sequence, and virtual space attribute parameters, through Doppler compensation and sound field parameter mapping. The dynamic sound field construction module is used to construct a dynamic sound field sequence that evolves continuously in the time and space dimensions according to the beamforming control parameters. The mapping module is used to match and map the dynamic sound field sequence and the human skin tactile perception model to obtain the target tactile perception driving parameter set; The output module is used to encode and convert the warning signal according to the target tactile perception driving parameter set to obtain a sound field tactile warning signal, wherein the sound field tactile warning signal is used to act on the skin of the warning target through an ultrasonic array.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images