Holographic projection 3D imaging interaction system based on mathematical algorithm driving

Abstract

The invention discloses a holographic projection 3D imaging interaction system based on mathematical algorithm driving, and belongs to the technical field of 3D imaging. The holographic projection 3D imaging interaction system driven based on the mathematical algorithm comprises a holographic image generation module, an intelligent interaction control module and a real-time data processing module. The problems that in the prior art, a large amount of three-dimensional data is needed in the actual use process, network delay is prone to occurring when a large amount of data is transmitted, and then the holographic projection 3D imaging effect is affected are solved, data are transmitted through multi-node compression, the data transmission speed and efficiency can be remarkably improved, the network burden is relieved, and the holographic projection 3D imaging effect is improved. According to the holographic projection 3D imaging interaction method and system, network congestion and delay are avoided, smoother and more stable interaction experience can be provided, the multi-node compression technology can adapt to different network environments, a good transmission effect can be provided whether a high-speed optical fiber network or a low-speed mobile network, and the applicability and flexibility of holographic projection 3D imaging interaction are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of 3D imaging technology, and particularly to a holographic projection 3D imaging interaction system driven by a mathematical algorithm. Background Art

[0002] Holographic projection technology is a technology that uses the principles of interference and diffraction to record and reproduce the true three-dimensional image of an object, and it is a 3D technology that does not require wearing glasses. This technology can not only produce a stereoscopic virtual image in the air, but also interact with users, giving users a good visual experience. This technology has been widely applied in museums, science and technology museums, 3D animations, product displays, real estate displays, stage shows, etc.

[0003] Chinese Patent with the publication number CN110096144B discloses an interactive holographic projection method and system based on three-dimensional reconstruction. By collecting the three-dimensional image information of an object, performing three-dimensional reconstruction after information processing, and displaying the three-dimensional reconstruction on a holographic projection pyramid to show the three-dimensional image, and controlling the change of the model through a sensor, it greatly improves the user's sensory experience; by preprocessing the three-dimensional data to obtain preprocessed data, the computer can filter out the dynamic blurred pixels of the motion collected by the sensor and generate an accurate and smooth holographic projection 3D image in real time, reducing visual fatigue; by applying a projector, it can improve the flexibility of the projection device and make it suitable for holographic projection pyramids of different specifications; by applying a curtain, a clear and uniform holographic projection 3D image can be obtained.

[0004] During the actual use of the above patent, due to the need for a large amount of three-dimensional data during 3D imaging, network latency is likely to occur when transmitting a large amount of data, which in turn affects the effect of holographic projection 3D imaging. Summary of the Invention

[0005] The purpose of the present invention is to provide a holographic projection 3D imaging interaction system driven by a mathematical algorithm, which can significantly improve the speed and efficiency of data transmission, thereby reducing the network burden, avoiding network congestion and latency, and being able to provide a smoother and more stable interaction experience. The multi-node compression technology can adapt to different network environments. Whether it is a high-speed fiber optic network or a low-speed mobile network, it can provide a good transmission effect, enhancing the applicability and flexibility of holographic projection 3D imaging interaction, and solving the problems raised in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solution: A holographic projection 3D imaging interaction system driven by a mathematical algorithm, comprising:

[0007] A holographic image generation module, which is used to obtain the three-dimensional data of the target object for finite element analysis, perform a three-dimensional model of the target object, and output a holographic projection 3D image of the target object;

[0008] An intelligent interaction control module, which is used to utilize a convolutional neural network and a spatio-temporal sequence analysis algorithm to parse the actions of a target object in real time and map them into three-dimensional space instructions;

[0009] A real-time data processing module, which is used to segment and preliminarily compress and encode holographic projection 3D image data, transmit the data after compression, restore a high-precision holographic image after receiving the compressed data, and dynamically adjust the compressive sensing parameters and the distributed task allocation strategy in combination with gesture recognition and voice interaction input.

[0010] Preferably, the holographic image generation module includes:

[0011] A three-dimensional information acquisition unit, which is used to acquire three-dimensional data of a target object through laser scanning and preprocess the three-dimensional data to obtain preprocessed data;

[0012] A three-dimensional modeling unit, which is used to perform finite element analysis on the preprocessed data to obtain three-dimensional modeling data of the target object, and use the three-dimensional modeling data to create a three-dimensional model of the target object;

[0013] An image output unit, which is used to capture the somatosensory information and environmental data of the target object, fuse the captured somatosensory information and environmental data with the three-dimensional model, and output a holographic projection 3D image of the target object.

[0014] Preferably, the projection image output unit includes:

[0015] Capture the somatosensory information of the target object, and use computer vision algorithms to analyze the illumination and obstacle distribution of the projection environment in real time;

[0016] Fuse the captured somatosensory information, the illumination of the projection environment, and the obstacle distribution with the three-dimensional model;

[0017] Convert digital content into light wave phase information, and use laser interference to form a holographic projection 3D image of the target object;

[0018] Based on the parallax mapping algorithm, generate a differentiated grating distribution for different viewing angles of the holographic projection 3D image of the target object to achieve a naked-eye stereoscopic vision effect.

[0019] Preferably, the intelligent interaction control module includes:

[0020] A spatial feature extraction unit, which is used to extract the morphological features and motion features of the holographic projection 3D image of the target object, as well as the local motion data of consecutive frames;

[0021] A temporal dynamic modeling unit, which is used to analyze the long-term dependencies of the action sequence of the target object, adopts a spatio-temporal graph convolutional network to perform topological modeling on the skeletal nodes of the target object, and quantifies the spatio-temporal evolution laws of joint angles and motion trajectories;

[0022] A three-dimensional space mapping unit, which is used to classify the feature vectors of the spatio-temporal evolution laws of joint angles and motion trajectories, predict the motion trajectory of the target object, and generate a standardized control instruction for the holographic projection 3D image of the target object.

[0023] Preferably, the analysis of the long-term dependencies of the action sequence of the target object specifically includes:

[0024] Using the morphological features and motion features of the holographic projection 3D image of the target object and the local motion data of consecutive frames, capturing the spatio-temporal features of the spatial coordinates, motion trajectory and morphological changes of the target object, synchronously recording the ambient light and projection angle auxiliary parameters, and constructing a multi-dimensional original data stream;

[0025] Denoise and filter the original data, and extract the key frame action features and temporal continuity features;

[0026] Using the principle of light wave interference to record the phase and amplitude information of each point of the target object, and forming a spatio-temporal correlation matrix;

[0027] Establish a self-attention mechanism for the action sequence, capture the correlation across time steps, and analyze the light wave phase change and motion continuity between adjacent frames of the holographic projection 3D image of the target object according to the correlation across time steps.

[0028] Preferably, the three-dimensional space mapping unit specifically includes:

[0029] Construct a standard action library for the target object, classify the feature vectors of the spatio-temporal evolution laws of joint angles and motion trajectories, and identify the standard action set of the target object in the preset action library according to the classification results;

[0030] Predict the motion trajectory of the target object in combination with the standard action set of the target object, use the motion trajectory of the target object to output the pose estimation of the next N frames, and perform action semantic parsing on the pose estimation of the next N frames;

[0031] Convert the parsed action semantics into a coordinate system transformation matrix, calculate the joint rotation angle and the end effector coordinates of the target object, and generate a standardized control instruction.

[0032] Preferably, the real-time data processing module includes:

[0033] A data extraction unit, which is used to obtain the key parameters of the multi-dimensional light field data of the holographic projection 3D image, and extract the significant feature information in the sparse representation of the key parameters of the multi-dimensional light field data;

[0034] A task allocation unit, which is used to divide the sparsified data into multiple subtasks, process them in parallel through distributed computing nodes, and use each node to combine a deep learning model to perform preliminary compression encoding on the subtasks;

[0035] A network optimization unit, which is used to reduce the data volume, reduce the network bandwidth occupancy, and preferentially transmit key data packets;

[0036] An image reconstruction unit, which is used to restore a high-precision holographic image after each node receives the compressed data, and ensure the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol;

[0037] A dynamic adjustment unit, which is used to dynamically adjust the compressive sensing parameters and the distributed task allocation strategy by combining gesture recognition and voice interaction input.

[0038] Preferably, the real-time data processing module specifically includes:

[0039] Obtain the key parameters of the multi-dimensional light field data of the holographic projection 3D image, and extract the significant feature information in the sparse representation of the key parameters of the multi-dimensional light field data;

[0040] Divide the sparsified data into multiple subtasks, use each node to combine a deep learning model to perform preliminary compression encoding on the subtasks, and combine the deep learning model to improve the data compression rate and feature extraction efficiency;

[0041] Reduce the data volume during the transmission process, and preferentially transmit key data packets during the transmission process. After each node receives the compressed data, restore the high-precision holographic projection 3D image of the target user;

[0042] Through a distributed clock synchronization protocol, ensure the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension, combine gesture recognition and voice interaction input, dynamically adjust the compressive sensing parameters and the distributed task allocation strategy, and optimize the subsequent data transmission efficiency.

[0043] Preferably, the dynamically adjusting the compressive sensing parameters by combining gesture recognition and voice interaction input specifically includes:

[0044] Extract the gesture data corresponding to the obtained result of gesture recognition;

[0045] Retrieve the weight corresponding to the gesture data from the database according to the gesture data;

[0046] Retrieve the speech recognition result from the voice interaction input data;

[0047] Retrieve the voice interaction instruction obtained by recognition from the speech recognition result;

[0048] Retrieve the weight corresponding to the voice interaction instruction from the database according to the voice interaction instruction;

[0049] Extract the generation time of the gesture data and the generation time of the voice interaction input data;

[0050] Obtain the time difference between the generation time of the gesture data and the generation time of the voice interaction input data according to the generation time of the gesture data and the generation time of the voice interaction input data;

[0051] Compare the time difference between the generation time of the gesture data and the generation time of the voice interaction input data with a preset time difference reference value;

[0052] Dynamically adjust the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the voice interaction input data and the preset time difference reference value.

[0053] Preferably, dynamically adjusting the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the voice interaction input data and the preset time difference reference value specifically includes:

[0054] When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the voice interaction input data does not exceed the preset time difference reference value, retrieve the weight corresponding to the gesture data and the weight corresponding to the voice interaction instruction;

[0055] Perform normalization processing on the weight corresponding to the gesture data and the weight corresponding to the voice interaction instruction;

[0056] Obtain the compression ratio corresponding to the dynamic compressive sensing parameter by using the weight corresponding to the gesture data and the weight corresponding to the voice interaction instruction after normalization processing;

[0057] When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the voice interaction input data exceeds the preset time difference reference value, retrieve the weight corresponding to the data with the earlier generation time and the weight corresponding to the data with the later generation time among the generation time of the gesture data and the generation time of the voice interaction input data;

[0058] Take the weight corresponding to the data with the earlier generation time as the target weight;

[0059] Take the weight corresponding to the data with the later generation time as the reference weight;

[0060] Perform normalization processing on the target weight and the reference weight;

[0061] The time difference between the generation time of the gesture data and the generation time of the voice interaction input data is obtained by combining the comparison result of the target weight and the reference weight after normalization processing, and the compression ratio corresponding to the dynamic compressive sensing parameter is obtained;

[0062] The dynamic compressive sensing parameter is adjusted according to the adjusted compression ratio.

[0063] Compared with the prior art, the beneficial effects of the present invention are:

[0064] The present invention can reduce the amount of data by compressing and transmitting data through multiple nodes, thereby reducing the bandwidth requirement and delay during the transmission process, and can significantly improve the speed and efficiency of data transmission. The multi-node compression technology can effectively reduce the amount of data transmitted, thereby reducing the network burden and avoiding network congestion and delay. By reducing the transmission time and delay, a smoother and more stable interaction experience can be provided. Compressing and transmitting data can also reduce the risk of data exposure during the transmission process and enhance data security. By reducing the amount of data transmitted, the risk of being intercepted or tampered with can be reduced. The multi-node compression technology can adapt to different network environments, and can provide good transmission effects whether it is a high-speed fiber optic network or a low-speed mobile network, enhancing the applicability and flexibility of the holographic projection 3D imaging interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the module of the holographic projection 3D imaging interaction system driven by a mathematical algorithm according to the present invention;

[0066] Figure 2 It is a schematic flow chart of the holographic projection 3D imaging interaction system driven by a mathematical algorithm according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0067] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0068] In order to solve the problem that in the prior art, when using 3D imaging, a large amount of three-dimensional data is required, and network delay is likely to occur when transmitting a large amount of data, which in turn affects the effect of holographic projection 3D imaging, please refer to Figure 1 - Figure 2 This embodiment provides the following technical solutions:

[0069] A holographic projection 3D imaging interaction system driven by a mathematical algorithm, comprising:

[0070] A holographic image generation module, which is used to obtain the three-dimensional data of the target object for finite element analysis, perform a three-dimensional model of the target object, and output the holographic projection 3D image of the target object;

[0071] An intelligent interaction control module, which is used to utilize a convolutional neural network and a spatio-temporal sequence analysis algorithm to real-time analyze the actions of the target object and map them into three-dimensional space instructions;

[0072] A real-time data processing module, which is used to segment and preliminarily compress and encode the holographic projection 3D image data, transmit the data after compression, restore a high-precision holographic image after receiving the compressed data, and dynamically adjust the compressive sensing parameters and distributed task allocation strategies in combination with gesture recognition and voice interaction input.

[0073] The holographic image generation module includes:

[0074] A three-dimensional information acquisition unit, which is used to obtain the three-dimensional data of the target object through laser scanning and preprocess the three-dimensional data to obtain preprocessed data;

[0075] A three-dimensional modeling unit, which is used to perform finite element analysis on the preprocessed data to obtain the three-dimensional modeling data of the target object, and use the three-dimensional modeling data to perform a three-dimensional model of the target object;

[0076] An image output unit, which is used to capture the somatosensory information and environmental data of the target object, fuse the captured somatosensory information and environmental data with the three-dimensional model, and output the holographic projection 3D image of the target object.

[0077] The projection image output unit includes:

[0078] Capture the somatosensory information of the target object, and use computer vision algorithms to real-time analyze the illumination and obstacle distribution of the projection environment;

[0079] Fuse the captured somatosensory information, the illumination of the projection environment, and the obstacle distribution with the three-dimensional model;

[0080] Convert digital content into optical wave phase information, and use laser interference to form the holographic projection 3D image of the target object;

[0081] Based on the parallax mapping algorithm, generate a differentiated grating distribution for different viewing angles of the holographic projection 3D image of the target object to achieve the naked-eye stereoscopic vision effect.

[0082] The intelligent interaction control module includes:

[0083] A spatial feature extraction unit, which is used to extract the morphological features and motion features of the holographic projection 3D image of the target object, as well as the local motion data of consecutive frames, introduce an attention mechanism to dynamically weight the key regions, and suppress the interference of background noise on feature extraction;

[0084] A temporal dynamic modeling unit, which is used to analyze the long-term dependence relationship of the action sequence of the target object, adopts a spatio-temporal graph convolutional network to perform topological modeling on the skeletal nodes of the target object, and quantifies the spatio-temporal evolution law of the joint angle and the motion trajectory;

[0085] A three-dimensional space mapping unit, which is used to classify the feature vectors of the spatio-temporal evolution law of the joint angle and the motion trajectory, predict the motion trajectory of the target object, and generate a standardized control instruction for the holographic projection 3D image of the target object.

[0086] Analyzing the long-term dependence relationship of the action sequence of the target object specifically includes:

[0087] Using the morphological features and motion features of the holographic projection 3D image of the target object and the local motion data of consecutive frames, capturing the spatio-temporal features of the spatial coordinates, motion trajectory and morphological changes of the target object, synchronously recording the ambient light and the auxiliary parameters such as the projection angle, and constructing a multi-dimensional original data stream;

[0088] Performing denoising and filtering processing on the original data, extracting the key frame action features and the temporal continuity features, where the key frame action features include displacement vectors and attitude angles, and the temporal continuity features include acceleration and frequency change.

[0089] Using the principle of light wave interference to record the phase and amplitude information of each point of the target object, forming a spatio-temporal correlation matrix;

[0090] Establishing a self-attention mechanism for the action sequence, capturing the correlation across time steps, and analyzing the light wave phase change and motion continuity between adjacent frames of the holographic projection 3D image of the target object according to the correlation across time steps.

[0091] The three-dimensional space mapping unit specifically includes:

[0092] Constructing a standard action library of the target object, classifying the feature vectors of the spatio-temporal evolution law of the joint angle and the motion trajectory, and identifying the standard action set (such as grasping, waving, etc.) of the target object in the preset action library according to the classification result;

[0093] Combining the standard action set of the target object to predict the motion trajectory of the target object, using the motion trajectory of the target object to output the pose estimation of the next N frames, and performing action semantic parsing on the pose estimation of the next N frames;

[0094] Converting the parsed action semantics into a coordinate system transformation matrix, calculating the joint rotation angle and the end effector coordinates of the target object, and generating a standardized control instruction.

[0095] A real-time data processing module, including:

[0096] A data extraction unit for obtaining the key parameters of the multi-dimensional light field data of the holographic projection 3D image. The key parameters include phase, polarization, spectrum, etc., and extracting the significant feature information under the sparse representation in the key parameters of the multi-dimensional light field data to avoid redundant data transmission;

[0097] A task allocation unit for dividing the sparsified data into multiple subtasks, processing them in parallel through distributed computing nodes to accelerate the optimization of the light field phase and image reconstruction, and using each node to combine a deep learning model to perform preliminary compression encoding on the subtasks, and combining the deep learning model to improve the data compression rate and feature extraction efficiency;

[0098] A network optimization unit for reducing the data volume, reducing the network bandwidth occupancy, preferentially transmitting key data packets, and ensuring that the transmission delay is controlled within milliseconds (such as 1.2 ms) to meet the real-time interaction requirements;

[0099] An image reconstruction unit for restoring a high-precision holographic image after each node receives the compressed data, and ensuring the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol to avoid projection distortion;

[0100] A dynamic adjustment unit for dynamically adjusting the compressive sensing parameters and the distributed task allocation strategy by combining gesture recognition and voice interaction input to optimize the subsequent data transmission efficiency.

[0101] A real-time data processing module specifically includes:

[0102] Obtaining the key parameters of the multi-dimensional light field data of the holographic projection 3D image, and extracting the significant feature information under the sparse representation in the key parameters of the multi-dimensional light field data;

[0103] Dividing the sparsified data into multiple subtasks, using each node to combine a deep learning model to perform preliminary compression encoding on the subtasks, and combining the deep learning model to improve the data compression rate and feature extraction efficiency;

[0104] Reducing the data volume during the transmission process, preferentially transmitting key data packets during the transmission process, and restoring a high-precision holographic projection 3D image of the target user after each node receives the compressed data;

[0105] Ensuring the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol, combining gesture recognition and voice interaction input, and dynamically adjusting the compressive sensing parameters and the distributed task allocation strategy to optimize the subsequent data transmission efficiency.

[0106] Working principle: When using the holographic projection 3D imaging interaction system driven by a mathematical algorithm of the present invention, according to Figure 1 and Figure 2 , it includes the following steps:

[0107] S1: Obtain the three-dimensional data of the target object through laser scanning, preprocess the three-dimensional data to obtain preprocessed data, perform finite element analysis on the preprocessed data to obtain the three-dimensional modeling data of the target object, and use the three-dimensional modeling data to create a three-dimensional model of the target object;

[0108] S2: Capture the somatosensory information and environmental data of the target object, fuse the captured somatosensory information and environmental data with the three-dimensional model, and output the holographic projection 3D image of the target object. Extract the morphological features and motion features of the holographic projection 3D image of the target object, as well as the local motion data of consecutive frames;

[0109] S3: Analyze the long-term dependence relationship of the action sequence of the target object using the morphological features and motion features of the holographic projection 3D image of the target object, as well as the local motion data of consecutive frames. Use a spatio-temporal graph convolutional network to perform topological modeling on the skeletal nodes of the target object, and quantify the spatio-temporal evolution law of joint angles and motion trajectories;

[0110] S4: Classify the feature vectors of the spatio-temporal evolution law of joint angles and motion trajectories, predict the motion trajectory of the target object, and generate a standardized control instruction for the holographic projection 3D image of the target object;

[0111] S5: Obtain the key parameters of the multi-dimensional light field data of the holographic projection 3D image, extract the significant feature information in the sparse representation of the key parameters of the multi-dimensional light field data, divide the sparsified data into multiple subtasks, and process them in parallel through distributed computing nodes. Use each node to perform preliminary compression encoding on the subtasks in combination with a deep learning model;

[0112] S6: Reduce the data volume during data transmission, give priority to transmitting key data packets, and at the same time combine gesture recognition and voice interaction input to dynamically adjust the compressive sensing parameters and distributed task allocation strategy;

[0113] S7: After each node receives the compressed data, restore the high-precision holographic image, and ensure the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol.

[0114] Specifically, the combination of gesture recognition and voice interaction input to dynamically adjust the compressive sensing parameters specifically includes:

[0115] Extract the gesture data corresponding to the result obtained from gesture recognition;

[0116] Retrieve the weight corresponding to the gesture data from the database according to the gesture data;

[0117] Retrieve the speech recognition result from the voice interaction input data;

[0118] Retrieve the voice interaction instruction obtained from the voice recognition result;

[0119] Retrieve the weight corresponding to the voice interaction instruction from the database according to the voice interaction instruction;

[0120] Extract the generation time of the gesture data and the generation time of the voice interaction input data;

[0121] Obtain the time difference between the generation time of the gesture data and the generation time of the voice interaction input data according to the generation time of the gesture data and the generation time of the voice interaction input data;

[0122] Compare the time difference between the generation time of the gesture data and the generation time of the voice interaction input data with a preset time difference reference value;

[0123] Dynamically adjust the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the voice interaction input data and the preset time difference reference value.

[0124] The technical effects of the above technical solution are as follows: First, gesture data corresponding to the obtained results of gesture recognition are extracted respectively, and speech recognition results are obtained from the speech interaction input data, and further the speech interaction instructions therein are retrieved. This is a preliminary collection of data generated by the two interaction methods, providing a basis for subsequent processing. According to the extracted gesture data, the corresponding weights are retrieved from the database, and at the same time, according to the speech interaction instructions, the weights corresponding to the instructions are retrieved from the database. These weights represent the importance or influence of gestures and speech instructions in the system. The generation times of the gesture data and the speech interaction input data are extracted, the time difference between the two is calculated, and it is compared with a preset time difference reference value. The time difference reflects the correlation between the two interaction methods in the time dimension. According to the comparison result of the time difference and the preset reference value, the compressive sensing parameters are dynamically adjusted. That is, the system determines how to adjust the compressive sensing parameters according to the temporal relationship between gesture and speech interaction and their respective weights to adapt to different interaction scenarios. Combining the two interaction methods of gesture and speech can meet the diverse input needs of users, and users can freely choose the interaction method according to their own habits and scenarios. Dynamically adjusting the compressive sensing parameters can make the system respond more accurately to user operations, reduce interaction latency, improve interaction fluency and naturalness, and enhance the user experience. By considering the temporal relationship between gesture and speech interaction and their respective weights to adjust the parameters, the system can better adapt to the interaction rhythms and operation habits of different users. Whether the user has frequent gesture operations or many speech instructions, the system can flexibly adjust to ensure stable and efficient operation in various interaction situations. Reasonably adjusting the compressive sensing parameters can optimize the system's processing method for the collected data. On the premise of ensuring data accuracy, it can more efficiently compress and process a large amount of gesture and speech interaction data, reduce the data storage and transmission burden, and improve the system's data processing efficiency and resource utilization rate.

[0125] Specifically, dynamically adjusting the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the speech interaction input data and a preset time difference reference value specifically includes:

[0126] When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the speech interaction input data does not exceed the preset time difference reference value, the weight corresponding to the gesture data and the weight corresponding to the speech interaction instruction are retrieved;

[0127] Perform normalization processing on the weight corresponding to the gesture data and the weight corresponding to the speech interaction instruction;

[0128] Use the weight corresponding to the gesture data and the weight corresponding to the speech interaction instruction after normalization processing to obtain the compression ratio corresponding to the dynamic compressive sensing parameters;

[0129] Among them, the adjusted compression ratio is obtained through the following formula:

[0130]

[0131] Among them, C represents the adjusted compression ratio; C0 represents the preset initial compression ratio; x 01 and x 02 respectively represent the weights corresponding to the gesture data after normalization processing and the weights corresponding to the voice interaction instructions; specifically, The periodicity of the sine function and the value range [-1, 1] are used to quantify the influence of the weight difference on the compression ratio. This formula can adaptively adjust the compression ratio according to the differences between the gesture and voice interaction weights. When the weights of the two interaction methods are close, the compression ratio is relatively stable and moderately improved; when the weight difference is large, the compression ratio can be adjusted accordingly according to the direction and degree of the difference. This adaptive adjustment can make the data compression process better adapt to different interaction situations, improving the flexibility and pertinence of data processing. By reasonably adjusting the compression ratio, while ensuring the quality of data processing, the amount of data can be effectively controlled. For example, when the difference between the two interaction weights requires more detailed data processing (such as a large weight difference representing a complex interaction situation), the compression ratio is appropriately reduced to reduce data loss and ensure data quality; when the interaction situation is relatively simple (small weight difference), the compression ratio is moderately increased to reduce the occupation of data storage space and transmission bandwidth, achieving a balance between the amount of data and quality. An appropriate compression ratio can optimize the performance of the system in data storage, transmission, and processing. It can avoid data distortion caused by excessive compression, which affects subsequent analysis and applications, and also prevent resource waste caused by insufficient compression. Thereby improving the overall operating efficiency of the system, reducing the hardware resource requirements, and enhancing the stability and reliability of the system.

[0132] When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the voice interaction input data exceeds the preset time difference reference value, then retrieve the weight corresponding to the data with the earlier generation time among the generation time of the gesture data and the generation time of the voice interaction input data and the weight corresponding to the data with the later generation time;

[0133] Take the weight corresponding to the data with the earlier generation time as the target weight;

[0134] Take the weight corresponding to the data with the later generation time as the reference weight;

[0135] Perform normalization processing on the target weight and the reference weight;

[0136] Use the normalized target weight and reference weight, combined with the comparison result indicating the time difference between the generation time of the gesture data and the generation time of the voice interaction input data, to obtain the compression ratio corresponding to the dynamic compressive sensing parameter;

[0137] Among them, the adjusted compression ratio is obtained through the following formula:

[0138]

[0139] Among them, C represents the adjusted compression ratio; C0 represents the preset initial compression ratio; x m and x c respectively represent the target weight and the reference weight after normalization processing; t y represents the preset time difference reference value; t represents the time difference between the generation moment of the gesture data and the generation moment of the voice interaction input data; The characteristic of the exponential function exp is that its value is always greater than 0, and when the independent variable approaches negative infinity, the function value approaches 0; when the independent variable approaches positive infinity, the function value approaches positive infinity. Here, the independent variable comprehensively considers the ratio of the target weight to the reference weight and the ratio of the time difference to the preset time difference reference value. Taking the absolute value is to ensure that unified processing can be carried out regardless of the positive or negative of the difference. reflects the relative magnitude relationship between the target weight and the reference weight, and reflects the comparison of the importance of the earlier generated interaction (corresponding to the target weight) and the later generated interaction (corresponding to the reference weight). reflects the relationship between the actual time difference and the preset time difference reference value, and reflects the temporal tightness of the two interactions. The combination of the two comprehensively measures the impact of the weight difference and time relationship of the interaction on the compression ratio. This formula can adjust the compression ratio by integrating the target weight, reference weight, and time difference information, and can accurately adapt to different gesture and voice interaction scenarios. For example, when the target weight is much larger than the reference weight and the time difference is small, it means that the earlier interaction plays a dominant role and the two interactions are closely connected. At this time, the compression ratio will be adjusted accordingly according to the formula to make the data processing fit this interaction situation and improve the pertinence and accuracy of data processing. Dynamically adjusting the compression ratio according to the weight and time difference can find a balance between resource utilization and data quality. If the weight difference between the two interactions is large and the time difference exceeds the expectation, it means that the interaction situation is complex. By adjusting the compression ratio, the compression degree can be appropriately reduced to ensure data quality; if the interaction situation is relatively simple (small weight difference, time difference meets the expectation), the compression ratio is increased to reduce the resources required for data storage and transmission and improve resource utilization efficiency. A reasonable compression ratio setting enables the system to more flexibly respond to different gesture and voice interaction inputs, quickly adjust the data processing method. In different interaction scenarios, data can be compressed in a timely and accurate manner, reducing data processing latency, improving the system's response performance to user interactions, and enhancing the user experience.

[0140] Adjust the dynamic compressive sensing parameters according to the adjusted compression ratio.

[0141] The technical effects of the above technical solution are as follows: First, compare the time difference between the generation times of the gesture data and the voice interaction input data with a preset reference value. If it does not exceed the reference value, it indicates that the two are relatively synchronous in time, and the gesture and voice command weights are retrieved simultaneously; if it exceeds the reference value, it indicates that the time interval between the two is large, and the target weight of the data generated first and the reference weight of the data generated later are retrieved. This step determines the weight data participating in the subsequent calculation based on the time relationship between the two interaction methods. Normalize the selected weights and map them to the same value range (such as [0, 1]). The purpose is to eliminate the influence of the original numerical differences of the weights, make the weights from different sources and magnitudes comparable, and lay a foundation for accurately calculating the compression ratio. When the time difference does not exceed the reference value, calculate the compression ratio using the two normalized weights. Since the two are synchronous, comprehensively consider the influence degrees of the two on the system to determine the compression ratio, which reflects the effect of the two interaction methods acting on the system at similar times. When the time difference exceeds the reference value, calculate the compression ratio by combining the normalized target weight, reference weight, and time difference. At this time, more emphasis is placed on the weight of the data generated first, and the influence of the time difference is considered simultaneously, reflecting the dominant role of the interaction that occurs first and the role of the time interval when they are out of sync. According to the calculated compression ratio, adjust the dynamic compressive sensing parameters so that the system optimizes the data processing method according to the gesture and voice interaction conditions.

[0142] Setting the compression ratio according to the time relationship and weights of gesture and voice interactions can make the compressive sensing parameters fit the actual interaction situation. In different interaction scenarios, the compression degree of data processing can be accurately adjusted, avoiding excessive or insufficient compression, ensuring the balance between data integrity and processing efficiency, and improving the accuracy and quality of data processing. Dynamically adjusting the compressive sensing parameters allows the system to quickly respond to gesture and voice interactions. When they are synchronous, comprehensive processing is performed; when they are out of sync, more emphasis is placed on the interaction that occurs first, making the system's feedback to user operations more timely and reasonable, enhancing the user interaction experience, and improving the system's interaction performance. A reasonable compression ratio setting can reduce the occupation of data storage and transmission resources on the premise of meeting the data processing requirements. Avoid unnecessary data redundancy processing, improve the utilization rate of system resources, reduce the consumption of hardware resources, and enhance the overall operation efficiency and stability of the system.

[0143] In summary, the holographic projection 3D imaging interaction system driven by mathematical algorithms of the present invention can achieve molecular-level nanotechnology components by performing holographic projection 3D imaging on a target object and interacting with the holographic projection 3D image, providing clear, three-dimensional and realistic display effects. This high definition and three-dimensional sense are incomparable to ordinary projection technologies. Holographic projection makes the spatial imaging colors more vivid, with very high contrast and clarity, thus providing a better visual experience. Holographic projection can produce three-dimensional phantom imaging, without the need to wear any wearable devices, and the experience effect can be achieved only with the naked eye, giving people a feeling of combining virtual and reality, as if being on the scene, and it is not restricted by space and venue. The display mode is rich and the utilization efficiency is high. Holographic projection combined with interactive technology enables the audience to operate through gestures or touches and interact with virtual objects in the projection, increasing the interest and sense of participation in the visit. Compared with traditional physical exhibits, holographic projection can display more content in a limited space, saving exhibition space and bringing greater flexibility to exhibition design. The visual effect of holographic projection is easy to attract attention and spread on social media and the Internet, which can increase the influence and dissemination effect of the exhibition.

[0144] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0145] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention.

Claims

1. A holographic projection 3D imaging interaction system driven by a mathematical algorithm, characterized in that, Including: A holographic image generation module, which is used to obtain the three-dimensional data of the target object for finite element analysis, perform a three-dimensional model of the target object, and output a holographic projection 3D image of the target object; An intelligent interaction control module, which is used to utilize a convolutional neural network and a spatio-temporal sequence analysis algorithm to parse the actions of the target object in real time and map them into three-dimensional space instructions; A real-time data processing module, which is used to segment and preliminarily compress and encode the holographic projection 3D image data, transmit the data after compression, restore a high-precision holographic image after receiving the compressed data, and dynamically adjust the compressive sensing parameters and distributed task allocation strategies in combination with gesture recognition and voice interaction input.

2. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 1, wherein: The holographic image generation module includes: A three-dimensional information acquisition unit, which is used to obtain the three-dimensional data of the target object by laser scanning and preprocess the three-dimensional data to obtain preprocessed data; A three-dimensional modeling unit, which is used to perform finite element analysis on the preprocessed data to obtain three-dimensional modeling data of the target object, and use the three-dimensional modeling data to perform a three-dimensional model of the target object; An image output unit, which is used to capture the somatosensory information and environmental data of the target object, fuse the captured somatosensory information and environmental data with the three-dimensional model, and output a holographic projection 3D image of the target object.

3. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 2, characterized in that: The projection image output unit includes: Capture the somatosensory information of the target object, and use computer vision algorithms to analyze the illumination and obstacle distribution of the projection environment in real time; Fuse the captured somatosensory information, the illumination of the projection environment, and the obstacle distribution with the three-dimensional model; Convert digital content into optical wave phase information, and use laser interference to form a holographic projection 3D image of the target object; Based on the parallax mapping algorithm, generate a differentiated grating distribution for different viewing angles of the holographic projection 3D image of the target object to achieve a naked-eye stereoscopic vision effect.

4. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 1, wherein: The intelligent interaction control module includes: A spatial feature extraction unit, which is used to extract the morphological features and motion features of the holographic projection 3D image of the target object, as well as the local motion data of consecutive frames; A temporal dynamic modeling unit, which is used to analyze the long-term dependence relationship of the action sequence of the target object, perform topological modeling on the bone nodes of the target object using a spatio-temporal graph convolutional network, and quantify the spatio-temporal evolution law of joint angles and motion trajectories; A three-dimensional space mapping unit, which is used to classify the feature vectors of the spatio-temporal evolution law of joint angles and motion trajectories, predict the motion trajectory of the target object, and generate a standardized control instruction for the holographic projection 3D image of the target object.

5. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 4, wherein: The analysis of the long-term dependence relationship of the action sequence of the target object specifically includes: Utilize the morphological features and motion features of the holographic projection 3D image of the target object and the local motion data of consecutive frames to capture the spatio-temporal features of the spatial coordinates, motion trajectory, and morphological changes of the target object, synchronously record the environmental illumination and projection angle auxiliary parameters, and construct a multi-dimensional original data stream; Perform denoising and filtering processing on the original data to extract key frame action features and temporal continuity features; Use the principle of light wave interference to record the phase and amplitude information of each point of the target object to form a spatio-temporal correlation matrix; Establish a self-attention mechanism for the action sequence to capture the correlation across time steps, and analyze the light wave phase change and motion continuity between adjacent frames of the holographic projection 3D image of the target object according to the correlation across time steps.

6. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 4, wherein: The three-dimensional space mapping unit specifically includes: Construct a standard action library for the target object, classify the eigenvectors of the spatio-temporal evolution law of joint angles and motion trajectories, and identify the standard action set of the target object in the preset action library according to the classification results; Predict the motion trajectory of the target object in combination with the standard action set of the target object, output the pose estimation of the next N frames by using the motion trajectory of the target object, and perform action semantic parsing on the pose estimation of the next N frames; Convert the parsed action semantics into a coordinate system transformation matrix, calculate the joint rotation angle and the end effector coordinates of the target object, and generate a standardized control instruction.

7. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 1, wherein: The real-time data processing module includes: A data extraction unit for obtaining the key parameters of the multi-dimensional light field data of the holographic projection 3D image and extracting the significant feature information in the sparse representation of the key parameters of the multi-dimensional light field data; A task allocation unit for dividing the sparsified data into multiple subtasks, processing them in parallel through distributed computing nodes, and using each node to perform preliminary compression encoding on the subtasks in combination with a deep learning model, and improving the data compression rate and feature extraction efficiency in combination with the deep learning model; A network optimization unit for reducing the data volume, reducing the network bandwidth occupancy, and preferentially transmitting key data packets; An image reconstruction unit for restoring a high-precision holographic image after each node receives the compressed data, and ensuring the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol; A dynamic adjustment unit for dynamically adjusting the compressive sensing parameters and the distributed task allocation strategy in combination with gesture recognition and voice interaction input to optimize the subsequent data transmission efficiency.

8. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 7, characterized in that: The real-time data processing module specifically includes: Obtain the key parameters of the multi-dimensional light field data of the holographic projection 3D image, and extract the significant feature information in the sparse representation of the key parameters of the multi-dimensional light field data; Divide the sparsified data into multiple subtasks, and use each node to perform preliminary compression encoding on the subtasks in combination with a deep learning model; Reduce the data volume during transmission, and preferentially transmit key data packets during transmission. After each node receives the compressed data, restore the high-precision holographic projection 3D image of the target user; Ensure the consistency of the light field information generated by multiple nodes in the spatio-temporal dimension through a distributed clock synchronization protocol, and dynamically adjust the compressive sensing parameters and the distributed task allocation strategy in combination with gesture recognition and voice interaction input.

9. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 8, characterized in that: The dynamic adjustment of the compressive sensing parameters in combination with gesture recognition and voice interaction input specifically includes: Extract the gesture data corresponding to the result obtained by gesture recognition; Retrieve the weight corresponding to the gesture data from the database according to the gesture data; Retrieve the speech recognition result from the voice interaction input data; Retrieve the voice interaction instruction obtained by recognition from the speech recognition result; Retrieve the weight corresponding to the voice interaction instruction from the database according to the voice interaction instruction; Extract the generation time of the gesture data and the generation time of the voice interaction input data; Obtain the time difference between the generation time of the gesture data and the generation time of the voice interaction input data according to the generation time of the gesture data and the generation time of the voice interaction input data; Compare the time difference between the generation time of the gesture data and the generation time of the voice interaction input data with a preset time difference reference value; Dynamically adjust the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the voice interaction input data and the preset time difference reference value.

10. The holographic projection 3D imaging interaction system driven by a mathematical algorithm according to claim 9, characterized in that: Dynamically adjust the compressive sensing parameters according to the comparison result between the time difference between the generation time of the gesture data and the generation time of the voice interaction input data and the preset time difference reference value, which specifically includes: When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the voice interaction input data does not exceed the preset time difference reference value, then retrieve the weight corresponding to the gesture data and the weight corresponding to the voice interaction instruction; Perform normalization processing on the weight corresponding to the gesture data and the weight corresponding to the voice interaction instruction; Obtain the compression ratio corresponding to the dynamic compressive sensing parameter by using the weight corresponding to the gesture data after normalization processing and the weight corresponding to the voice interaction instruction; When the comparison result indicates that the time difference between the generation time of the gesture data and the generation time of the voice interaction input data exceeds the preset time difference reference value, then retrieve the weight corresponding to the data with the earlier generation time and the weight corresponding to the data with the later generation time among the generation time of the gesture data and the generation time of the voice interaction input data; Take the weight corresponding to the data with the earlier generation time as the target weight; Take the weight corresponding to the data with the later generation time as the reference weight; Perform normalization processing on the target weight and the reference weight; Obtain the compression ratio corresponding to the dynamic compressive sensing parameter by using the target weight and the reference weight after normalization processing and combining the comparison result indicating the time difference between the generation time of the gesture data and the generation time of the voice interaction input data; Adjust the dynamic compressive sensing parameter according to the adjusted compression ratio.

Images