Information mapping method of intelligent perception information in virtual space

Abstract

The embodiment of the present application provides an information mapping method for intelligent perception information in a virtual space, which collects perception data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and uses self-supervised learning and geometric constraints to establish a spatial mapping relationship. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a method for mapping intelligent perception information in a virtual space. Background Art

[0002] With the rapid development of virtual space technology, how to accurately map the intelligent perception information of the physical world into the virtual space has become a key challenge. Traditional information mapping methods mainly use simple coordinate transformation and data synchronization mechanisms, which are difficult to handle the real-time mapping requirements of large-scale multi-source perception data. Although there are some mapping solutions based on deep learning, these solutions still have problems such as poor data consistency and low computational efficiency when dealing with complex scenes.

[0003] Existing systems generally have problems such as unstable mapping accuracy and insufficient real-time performance when processing high-dimensional, heterogeneous sensory data. Especially when processing multi-sensor array data, it is difficult for the system to effectively coordinate and optimize data flows, resulting in low resource utilization. At the same time, existing methods lack a deep understanding of data features and efficient organization, and cannot implement intelligent data processing and transmission strategies, which seriously affects the data synchronization quality of virtual space.

[0004] Therefore, how to build an efficient feature extraction and mapping mechanism, realize the intelligent organization and optimized transmission of perception data, and improve the real-time processing capability of the system are key issues that need to be solved urgently. This is not only related to the data quality of the virtual space, but also an important basis for realizing the deep integration of the physical world and the virtual space. Summary of the invention

[0005] In response to the problems in the prior art, the present application provides an information mapping method for intelligent perception information in a virtual space, which can effectively solve the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data.

[0006] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a method for mapping intelligent perception information in a virtual space, comprising:

[0008] Deploy camera arrays, temperature sensor arrays, and sound sensor arrays in the scene space to collect intelligent perception data, write the intelligent perception data into a multidimensional data buffer matrix in timestamp order, construct a multi-scale feature pyramid to extract data feature vectors, use an attention mechanism to weight the importance of the data feature vectors, train a self-supervised spatial feature extractor based on temporal consistency constraints, mark corresponding point sets in the scene space and virtual space, and solve the spatial transformation matrix in combination with geometric invariance constraints;

[0009] Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find set, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache;

[0010] Construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0011] Furthermore, the camera array, the temperature sensor array, and the sound sensor array are deployed in the scene space to collect intelligent perception data, the intelligent perception data is written into a multidimensional data buffer matrix in timestamp order, and a multi-scale feature pyramid is constructed to extract data feature vectors, including:

[0012] The scene space is divided into grid units, and three groups of sensor arrays are set at the boundary position of each grid unit, wherein the three groups of sensor arrays include a six-degree-of-freedom camera array, a temperature sensor array, and a sound sensor array, and the sensor array sampling coordinates are calculated based on the spatial position of the grid unit to collect the intelligent perception data in the grid unit;

[0013] A time window threshold is set for the intelligent perception data, the intelligent perception data within the time window threshold is stored in a four-dimensional data buffer matrix, a three-layer pyramid structure is constructed based on the four-dimensional data buffer matrix, a feature map of each layer of the pyramid is calculated by a Gaussian filter, non-maximum suppression is performed on the feature map to extract data feature points, and the data feature points are mapped to a feature space to generate a data feature vector.

[0014] Furthermore, the attention mechanism is used to weight the importance of the data feature vector, a self-supervised spatial feature extractor is trained based on temporal consistency constraints, corresponding point sets are marked in the scene space and the virtual space, and the spatial transformation matrix is ​​solved in combination with geometric invariance constraints, including:

[0015] Divide the data feature vector into feature blocks of fixed size, calculate the self-attention weight matrix of the feature block, weight the feature block using the self-attention weight matrix, calculate the temporal correlation score based on the weighted feature block, construct a temporal consistency loss function, and use the gradient descent method to train the network parameters of the spatial feature extractor;

[0016] A plurality of corner points are selected as marking points in the scene space, and the three-dimensional coordinates of the marking points are calculated. The three-dimensional coordinates are mapped to the virtual space to generate a corresponding point set, and a space transformation equation group is established using the corresponding point set. Rotational and translational invariance constraints are introduced into the space transformation equation group, and the parameter values ​​of the space transformation matrix are solved by the least squares method.

[0017] Furthermore, the step of dividing the intelligent sensing data into local data blocks, constructing a hierarchical index tree based on a union-find set, performing complexity evaluation on the local data blocks, and generating an adaptive computation flow graph includes:

[0018] An octree structure is constructed in the smart sensing data, a data space is divided into local data blocks based on the octree structure, boundary coordinates of the local data blocks are calculated, a union-find data structure is constructed using the boundary coordinates, the local data blocks are added to a union-find node, and a hierarchical index tree is established based on the connectivity relationship of the union-find node;

[0019] Calculate the data density distribution of the local data block, set a computational complexity threshold based on the data density distribution, convert the computational complexity threshold into a computational weight coefficient, construct a directed acyclic graph according to the computational weight coefficient, and use the directed acyclic graph as the topological structure of the adaptive computational flow graph.

[0020] Further, the mapping transformation is performed at the nodes of the adaptive computation flow graph, the high access probability data is loaded in advance by using a predictive cache mechanism, a differential update strategy is generated based on a local consistency constraint, and the result of the mapping transformation is written into a virtual space cache by using hierarchical compression coding, including:

[0021] Selecting a computing node from the adaptive computing flow graph, applying the spatial transformation matrix to the data of the computing node, constructing a Markov prediction model to calculate the access probability of a data block, establishing a cache queue according to the access probability, loading the data in the cache queue into a video memory, constructing a local consistency constraint function based on the spatial distance between data blocks, and calculating the data update amount using the local consistency constraint function;

[0022] Perform wavelet transform on the data update amount to obtain frequency coefficients, sort the frequency coefficients into layers according to energy size, set quantization parameters for each layer of frequency coefficients, encode and compress the frequency coefficients using the quantization parameters, establish a virtual space buffer pool, and write the compressed data into the corresponding area of ​​the virtual space buffer pool.

[0023] Further, the constructing of the pipeline data channel, transmitting the incremental mapping data in parallel in the pipeline data channel, and organizing the incremental mapping data into a refresh sequence according to the update priority, includes:

[0024] Establish a data transmission link between computing nodes, allocate a buffer for the data transmission link, divide the buffer into two read and write areas, set a synchronization signal between the read and write areas, build a pipeline channel based on the synchronization signal, perform task segmentation on the pipeline channel, and transmit incremental mapping data in parallel in each task segment;

[0025] The data change rate of the incremental mapping data is calculated, an update priority threshold is set according to the data change rate, the update priority threshold is divided into multiple levels, a timestamp index is established for data of each level, and the incremental mapping data is organized into a refresh sequence based on the timestamp index.

[0026] Furthermore, the method of ensuring state synchronization by using a data consistency verification mechanism, establishing an adaptive computing resource pool, dynamically scheduling computing nodes based on load prediction, and synchronizing the perception information state to the computing nodes includes:

[0027] Construct a state verification table for each data block, record the data version number in the state verification table, calculate the checksum value of the data block, add the checksum value to the state verification table, compare the state verification tables of different computing nodes, construct a consistency score based on the difference between the state verification tables, and trigger a data synchronization operation according to the consistency score;

[0028] The computing nodes are grouped into resource groups, the processing capacity index of each group of computing nodes is calculated, the processing capacity index is input into the load prediction model, the resource utilization is calculated based on the load prediction model, the computing tasks are dynamically allocated according to the resource utilization, a status broadcast channel between computing nodes is established, and the perception information status is broadcast to the target computing node.

[0029] In a second aspect, the present application provides an information mapping device for intelligent perception information in a virtual space, comprising:

[0030] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for mapping intelligent perception information in a virtual space are implemented.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for mapping intelligent perception information in a virtual space.

[0032] In a fifth aspect, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the method for mapping intelligent perception information in a virtual space.

[0033] It can be seen from the above technical solutions that the present application provides an information mapping method for intelligent perception information in virtual space, which collects perception data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and uses self-supervised learning and geometric constraints to establish a spatial mapping relationship. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 This is one of the flow charts of the information mapping method of intelligent perception information in a virtual space in an embodiment of the present application;

[0036] Figure 2 The second flowchart of the information mapping method of intelligent perception information in a virtual space in the embodiment of the present application;

[0037] Figure 3 The third flowchart of the information mapping method of intelligent perception information in a virtual space in the embodiment of the present application;

[0038] Figure 4 This is a fourth flow chart of the information mapping method of intelligent perception information in a virtual space in an embodiment of the present application;

[0039] Figure 5 FIG5 is a flowchart of the information mapping method of intelligent perception information in a virtual space in an embodiment of the present application;

[0040] Figure 6 This is a sixth flow chart of the information mapping method of intelligent perception information in a virtual space in an embodiment of the present application;

[0041] Figure 7FIG7 is a flow chart of the information mapping method of intelligent perception information in a virtual space in an embodiment of the present application;

[0042] Figure 8 It is a structural diagram of an information mapping device for intelligent perception information in a virtual space in an embodiment of the present application;

[0043] Fig. 9 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application.

[0044] Reference numerals:

[0045] Electronic device 9600, central processing unit 9100, memory 9140, communication module 9110, input unit 9120, audio processor 9130, display 9160, power supply 9170, buffer memory 9141, application / function storage unit 9142, data storage unit 9143, driver program storage unit 9144, antenna 9111, speaker 9131, microphone 9132. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] The acquisition, storage, use, and processing of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0048] Taking into account the problems existing in the prior art, the present application provides an information mapping method for intelligent perception information in virtual space, which collects perception data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and uses self-supervised learning and geometric constraints to establish a spatial mapping relationship. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data.

[0049] In order to effectively solve the performance bottleneck problem of traditional mapping solutions when processing large-scale multi-source perception data, this application provides an embodiment of an information mapping method for intelligent perception information in a virtual space, see Figure 1The information mapping method of the intelligent perception information in the virtual space specifically includes the following contents:

[0050] Step S101: deploying a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, writing the intelligent perception data into a multidimensional data buffer matrix in timestamp order, constructing a multi-scale feature pyramid to extract data feature vectors, using an attention mechanism to weight the importance of the data feature vectors, training a self-supervised spatial feature extractor based on temporal consistency constraints, marking corresponding point sets in the scene space and the virtual space, and solving the spatial transformation matrix in combination with geometric invariance constraints;

[0051] Optionally, this embodiment first deploys a sensor array network in the scene space. In application scenarios such as industrial production workshops, the sensor array adopts a regular hexagonal distribution to ensure uniform coverage of the sampling area. Each sampling node includes a set of six-degree-of-freedom camera arrays for capturing visual information of the scene, a set of temperature sensor arrays for obtaining thermal distribution data, and a set of sound sensor arrays for collecting acoustic features.

[0052] This embodiment uses a spatiotemporal alignment strategy to process multi-source sensor data. Each sensor data frame carries high-precision timestamp information, and the multimodal data at the same time is organized into a four-dimensional buffer matrix through a time window mechanism. The four dimensions of the matrix correspond to spatial coordinates, timestamps, and data channels, respectively, to achieve unified expression and storage of data.

[0053] This embodiment extracts multi-scale features by constructing a feature pyramid. Based on the original resolution data, a three-layer pyramid structure is constructed by step-by-step downsampling. Each layer uses Gaussian kernels of different scales for convolution to extract feature information in different frequency ranges. Feature points are selected by non-maximum suppression, and these feature points are mapped to a high-dimensional feature space to form feature vectors.

[0054] This embodiment introduces an attention mechanism to weight the feature vectors. The attention weight matrix is ​​obtained by calculating the correlation between the feature vectors, which reflects the importance of different features. In industrial scenarios, this weighting mechanism can highlight the status characteristics of key production equipment and improve the accuracy of anomaly detection.

[0055] This embodiment designs a self-supervised learning framework based on temporal consistency. By analyzing the changing rules of features in adjacent time windows, a temporal consistency loss function is constructed. This function guides the feature extractor to learn stable feature expressions in the scene and improves the robustness of feature extraction. The training process does not require manual labeling, which reduces deployment costs.

[0056] This embodiment establishes a correspondence between the actual scene and the virtual space. By detecting the characteristic corner points in the scene as marking points, the three-dimensional coordinates of these points are calculated. At the same time, the corresponding positions are marked in the virtual space to establish a mapping relationship between the two spaces. This correspondence mechanism provides a basis for subsequent space transformation.

[0057] This embodiment introduces geometric invariance constraints when solving the spatial transformation matrix. A transformation equation group is established based on the marked point pairs, and rotation and translation invariance constraints are added to the optimization target. The transformation matrix parameters are solved through iterative optimization to ensure that the transformation result maintains the geometric characteristics of the scene.

[0058] This embodiment realizes the accurate mapping of industrial scenes to virtual space. In practical applications, this solution can accurately capture the operating status of the production line and map multi-source sensory data to the virtual space for analysis and monitoring. Through multimodal data fusion and feature extraction, the system's ability to identify abnormal conditions is improved. The self-supervised learning mechanism based on temporal consistency reduces the labor cost of system maintenance and updating. The geometric invariance constraint ensures the stability of spatial mapping and provides a reliable data foundation for industrial digital twins.

[0059] Step S102: Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache;

[0060] Optionally, this embodiment uses an octree structure to spatially partition the intelligent sensing data. In industrial scenarios, the data density and importance of different areas vary. For example, the amount of data in areas with dense production equipment is large. Through an adaptive spatial partitioning strategy, the size of the data block is ensured to be compatible with the complexity of the area, effectively balancing the computing load.

[0061] This embodiment uses a union-find data structure to construct a hierarchical index. A union-find is established based on the spatial adjacency of data blocks, and query efficiency is improved through path compression optimization. In the index tree, each node stores the boundary information and adjacency relationship of the data block, supporting fast spatial range query and neighborhood search.

[0062] This embodiment designs a complexity evaluation mechanism based on data density. The complexity of the data is evaluated by calculating the information entropy and gradient distribution within the data block. For high-complexity areas such as key positions of the production line, more computing resources are allocated to ensure processing accuracy. The evaluation results are converted into calculation weights to guide subsequent task scheduling.

[0063] This embodiment constructs an adaptive computational flow graph based on a directed acyclic graph. The nodes in the graph represent data processing tasks, and the edges represent data dependencies. The order of task execution is determined by topological sorting, supporting parallel processing of tasks. The computational flow graph can be dynamically adjusted according to data characteristics to adapt to changes in data distribution in industrial scenarios.

[0064] This embodiment implements mapping transformation in the computing node. The spatial transformation matrix solved in the previous step is applied to the data block to complete the coordinate mapping from the actual scene to the virtual space. The transformation process takes into account the geometric invariance constraint and maintains the spatial structure characteristics.

[0065] This embodiment introduces a Markov prediction model to implement predictive caching. By analyzing the data access pattern, the data blocks with high access probability are predicted and loaded into the video memory in advance. This mechanism is particularly suitable for periodic data access scenarios in industrial production, significantly reducing data loading delays.

[0066] This embodiment generates a differential update strategy based on local consistency constraints. By calculating the spatial correlation between adjacent data blocks, a local consistency constraint function is constructed. This function guides the system to update only the areas where significant changes occur, reducing data transmission overhead.

[0067] This embodiment adopts a hierarchical compression coding scheme. Wavelet transform is performed on the data update amount to decompose the signal into different frequency components. Hierarchical quantization parameters are set according to energy distribution, high energy coefficients are finely quantized, and low energy coefficients are roughly quantized. The compressed data is written into the virtual space cache pool to achieve efficient storage.

[0068] This embodiment implements an efficient processing and caching mechanism for industrial scene data. Through spatial partitioning and union-find indexing, the organizational efficiency of large-scale data is improved. Predictive caching and differential update strategies reduce system response delays and meet the real-time monitoring needs of industrial sites. Hierarchical compression coding significantly reduces storage overhead while ensuring data quality. This solution provides an efficient data processing and caching solution for industrial digital twin systems.

[0069] Step S103: construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0070] Optionally, this embodiment designs a multi-level pipeline data channel structure. In the industrial digital twin scenario, different types of incremental mapping data have different timeliness requirements, for example, equipment failure information needs to be transmitted first. The pipeline channel adopts a hierarchical design, high-priority data is transmitted through a fast channel, and regular data is transmitted through a standard channel, realizing differentiated processing of data transmission.

[0071] This embodiment implements a priority-based data organization mechanism. By evaluating the timeliness, importance, and relevance of the data, update priorities are assigned to incremental mapping data. In industrial sites, the status data and abnormal information of key equipment are given higher priority to ensure that they are reflected in the virtual space in a timely manner. Data is organized into update queues according to priority to form an orderly refresh sequence.

[0072] This embodiment introduces a distributed consistency verification mechanism. A two-phase commit protocol is used to ensure the consistency of data during transmission and update. The verification process includes two stages: pre-commit and confirm commit, which effectively prevents state deviation caused by inconsistent data. In the collaborative processing of multi-source data in industrial sites, this mechanism ensures the reliability of data.

[0073] This embodiment builds an elastic computing resource pool. The resource pool contains computing nodes of different specifications and supports dynamic allocation of computing tasks. By monitoring the computing load and resource utilization of the nodes, the available computing resources are adjusted in real time to adapt to load fluctuations in industrial scenarios.

[0074] This embodiment uses a deep learning model for load forecasting. By analyzing the time series characteristics of historical load data, a load forecasting model is established. The model takes into account the cyclical characteristics and temporary fluctuations of industrial production, provides accurate load forecasting results, and guides resource scheduling decisions.

[0075] This embodiment implements dynamic scheduling based on load balancing. The scheduler dynamically adjusts the allocation scheme of computing nodes according to the load prediction results. For the predicted high load period, the computing resources are expanded in advance; for the low load period, the resources are appropriately recycled to achieve the optimization of resource utilization.

[0076] This embodiment designs an efficient state synchronization mechanism. An incremental synchronization strategy is adopted to transmit only the changed state information. The state data is compressed and encoded and transmitted to the computing node, and the computing node updates the local state through the state merging algorithm. This mechanism reduces the data transmission overhead and improves the synchronization efficiency.

[0077] This embodiment provides an efficient data processing and synchronization solution for the industrial digital twin system. Through the pipeline data channel and priority mechanism, the timely transmission of key data is guaranteed. Data consistency verification ensures the reliability of state synchronization. The adaptive computing resource pool and dynamic scheduling strategy improve the resource utilization efficiency of the system, enabling the system to flexibly respond to load changes in the industrial site. This solution realizes the efficient collaboration of physical space and virtual space in industrial scenes, providing reliable technical support for industrial digital twins.

[0078] From the above description, it can be seen that the information mapping method of intelligent perception information in virtual space provided by the embodiment of the present application can collect perception data by deploying a multi-sensor array, extract data features based on a multi-scale feature pyramid and attention mechanism, and establish a spatial mapping relationship using self-supervised learning and geometric constraints. A hierarchical index tree based on union-find is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data.

[0079] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 2 , and can also include the following:

[0080] Step S201: Divide the scene space into grid units, set three groups of sensor arrays at the boundary position of each grid unit, the three groups of sensor arrays include a six-degree-of-freedom camera array, a temperature sensor array, and a sound sensor array, calculate the sensor array sampling coordinates based on the spatial position of the grid unit, and collect the intelligent perception data in the grid unit;

[0081] Step S202: setting a time window threshold for the intelligent perception data, storing the intelligent perception data within the time window threshold into a four-dimensional data buffer matrix, constructing a three-layer pyramid structure based on the four-dimensional data buffer matrix, calculating a feature map of each layer of the pyramid through a Gaussian filter, performing non-maximum suppression on the feature map to extract data feature points, and mapping the data feature points to a feature space to generate a data feature vector.

[0082] Optionally, this embodiment uses an adaptive grid partitioning strategy to partition the scene space. In an industrial production environment, the scene space is divided into grid units of varying sizes according to the equipment distribution density and monitoring requirements. For equipment-intensive areas, a smaller grid size is used to improve sampling accuracy; for open areas, a larger grid size is used to optimize resource allocation.

[0083] This embodiment deploys a multimodal sensor array at the boundaries of the grid cells. The six-degree-of-freedom camera array consists of three sets of binocular cameras, which are distributed in an equilateral triangle to achieve all-round visual information collection. The temperature sensor array uses an infrared thermal imaging sensor and is equipped with a temperature compensation mechanism to ensure measurement accuracy. The sound sensor array realizes sound source localization and acoustic feature extraction through a microphone array.

[0084] This embodiment designs a method for calculating sensor coordinates based on spatial position. The sampling coordinates of each sensor are calculated based on the global coordinate system of the scene space and the geometric features of the grid unit. The determination of the sampling coordinates takes into account the field of view, detection range, and overlapping area of ​​the sensor to ensure the continuity and integrity of data collection.

[0085] This embodiment implements the time series organization of intelligent perception data. According to the dynamic characteristics of the industrial scene, an adaptive time window threshold is set. For example, for high-speed equipment areas, a smaller time window is used; for areas with slow changes, a larger time window is used. This differentiation strategy optimizes data collection efficiency.

[0086] This embodiment uses a four-dimensional data buffer matrix to store multimodal data. The four dimensions of the matrix correspond to the spatial coordinates (x, y, z) and the time dimension, respectively, to achieve a unified expression of the data. The matrix structure supports fast data retrieval and update operations, facilitating subsequent feature extraction.

[0087] This embodiment constructs a multi-scale pyramid structure for feature extraction. The three-layer structure of the pyramid corresponds to different spatial resolutions and is formed by downsampling. Each layer uses Gaussian filters of different scales for smoothing to extract multi-scale features. The lower layer retains the detail features, and the higher layer extracts the global features to achieve multi-scale expression of features.

[0088] This embodiment extracts feature points through a non-maximum suppression algorithm. On the feature map, the pixel values ​​in the local area are compared, and the local maximum points are retained as feature points. These feature points represent significant features in the scene, such as device edges, temperature anomalies, acoustic feature points, etc.

[0089] This embodiment realizes the mapping of feature points to feature space. By calculating the local descriptor of the neighborhood of the feature point, the feature point is mapped to the high-dimensional feature space. The feature vector contains multi-dimensional information such as position, shape, texture, etc., providing a basis for subsequent feature matching and state recognition.

[0090] This embodiment provides a high-quality data collection and feature extraction solution for the industrial digital twin system. The coordinated deployment of multimodal sensors improves the comprehensiveness and reliability of data. The adaptive data organization strategy and multi-scale feature extraction method effectively capture the dynamic characteristics of industrial scenes. This solution realizes the accurate perception and feature expression of industrial field data, laying the foundation for the state mapping of digital twins.

[0091] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 3 , and can also include the following:

[0092] Step S301: Divide the data feature vector into feature blocks of fixed size, calculate the self-attention weight matrix of the feature block, weight the feature block using the self-attention weight matrix, calculate the temporal correlation score based on the weighted feature block, construct a temporal consistency loss function, and use the gradient descent method to train the network parameters of the spatial feature extractor;

[0093] Step S302: Select multiple corner points in the scene space as marking points, calculate the three-dimensional coordinates of the marking points, map the three-dimensional coordinates to the virtual space to generate a corresponding point set, use the corresponding point set to establish a space transformation equation group, introduce rotation and translation invariance constraints into the space transformation equation group, and use the least squares method to solve the parameter values ​​of the space transformation matrix.

[0094] Optionally, this embodiment uses a block processing strategy to process the data feature vector. In industrial scenarios, features in different regions have local correlations. By dividing the feature vector into feature blocks of fixed size, local feature patterns can be better captured. The size of the feature block is determined according to the spatial distribution characteristics of the scene features, which ensures both computational efficiency and feature integrity.

[0095] This embodiment introduces a self-attention mechanism to process feature blocks. By calculating the correlation between the elements within the feature block, a self-attention weight matrix is ​​generated. The matrix reflects the dependency between feature elements, such as the association status between different components of industrial equipment. The weight matrix is ​​used to adaptively weight features and highlight key feature information.

[0096] This embodiment implements feature correlation analysis based on time series. By calculating the similarity between feature blocks at different time points, a time series correlation score is obtained. This analysis is particularly suitable for periodic process monitoring in industrial production and can effectively capture the time series change characteristics of the equipment operation status.

[0097] This embodiment constructs a temporal consistency loss function. This function comprehensively considers the spatial consistency and temporal continuity of features, and guides the feature extractor to learn stable feature representations. The design of the loss function prompts the model to pay attention to the temporal evolution of features and improves the stability of feature extraction.

[0098] This embodiment uses an improved gradient descent algorithm to optimize network parameters. An adaptive learning rate adjustment mechanism is introduced into the algorithm to dynamically adjust the parameter update step size according to the convergence of the loss function to improve training efficiency and model performance.

[0099] This embodiment selects corner points with obvious features in the scene space as marking points. These corner points are usually located at the corners of the device, the intersection of the structure, etc., and have stable geometric features. The three-dimensional coordinates of the marking points are obtained through multi-view measurement to ensure the accuracy of the coordinate measurement.

[0100] This embodiment realizes the mapping conversion from physical space to virtual space. The three-dimensional coordinates of the marking point are mapped to the virtual space to generate a set of corresponding points. These corresponding points constitute the reference points of the space transformation and provide constraints for solving the transformation matrix.

[0101] This embodiment establishes a set of spatial transformation equations containing rotation and translation invariance constraints. The constraints ensure that the relative position relationship of objects before and after the transformation remains unchanged, which is particularly important for the spatial layout of industrial equipment. The set of equations comprehensively considers the position correspondence relationship and geometric invariance requirements.

[0102] This embodiment uses the weighted least square method to solve the transformation matrix. By assigning weights to different marking points, the influence of key positions is highlighted. The solution process adopts an iterative optimization strategy and continuously adjusts parameters until convergence conditions are reached.

[0103] This embodiment provides an efficient feature processing and space mapping solution for the industrial digital twin system. The self-attention mechanism enhances the selectivity of feature extraction, and the temporal consistency constraint improves the stability of feature representation. The precise solution of the spatial transformation matrix ensures the accurate mapping of the physical space to the virtual space. This solution achieves high-quality extraction and reliable mapping of industrial scene features, providing strong support for the state synchronization of the digital twin system.

[0104] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 4 , and can also include the following:

[0105] Step S401: constructing an octree structure in the smart sensing data, dividing the data space into local data blocks based on the octree structure, calculating the boundary coordinates of the local data blocks, constructing a union-find data structure using the boundary coordinates, adding the local data blocks to the union-find nodes, and establishing a hierarchical index tree based on the connectivity relationship of the union-find nodes;

[0106] Step S402: Calculate the data density distribution of the local data block, set a computational complexity threshold based on the data density distribution, convert the computational complexity threshold into a computational weight coefficient, construct a directed acyclic graph according to the computational weight coefficient, and use the directed acyclic graph as the topological structure of the adaptive computational flow graph.

[0107] Optionally, this embodiment uses an octree structure to spatially organize the intelligent sensing data. In industrial scenarios, data distribution usually has spatial locality characteristics, and the octree structure can adaptively divide the space according to the data density. For equipment-dense areas, more detailed spatial division is performed; for sparse areas, larger spatial blocks are used to achieve adaptability of spatial division.

[0108] This embodiment implements local data block management based on boundary coordinates. By calculating the boundary coordinates of each data block, the spatial range and adjacent relationship of the data block are determined. This spatial division method is particularly suitable for spatial layout analysis of industrial equipment and can effectively capture the spatial correlation between equipment.

[0109] This embodiment constructs a union-find data structure to manage the relationship between data blocks. The union-find structure represents the connectivity between data blocks through the parent-child node relationship, supporting fast connectivity query and merge operations. In industrial scenarios, this structure effectively expresses the hierarchical relationship and functional association between device components.

[0110] This embodiment establishes a hierarchical index tree to optimize data access. The index tree is constructed based on the spatial location and connectivity of data blocks, supporting multi-level data retrieval. For example, when querying the device status of a specific area, the relevant data blocks can be quickly located, improving data access efficiency.

[0111] This embodiment implements adaptive calculation of data density distribution. The data density is calculated by analyzing the distribution characteristics of data points in local data blocks. The density distribution reflects the information richness of different areas in the industrial scene and provides a basis for computing resource allocation.

[0112] This embodiment designs a complexity evaluation mechanism based on data features. According to the data density distribution, a computational complexity threshold is set and the complexity is converted into a computational weight coefficient. This mechanism takes into account the computational requirements of data processing in different regions and realizes the reasonable allocation of computing resources.

[0113] This embodiment constructs a directed acyclic graph as the organizational structure of the computational flow. The nodes in the graph represent computational tasks, the edges represent data dependencies, and the weights reflect computational complexity. This structure is particularly suitable for multi-stage data processing in industrial scenarios, such as equipment status analysis, fault diagnosis, and other tasks.

[0114] This embodiment realizes the dynamic adjustment of adaptive computing flow. Based on the topological structure of directed acyclic graph, the system can dynamically adjust the execution order and resource allocation of computing tasks according to data characteristics and computing requirements. For example, for high-weight critical tasks, computing resources are allocated preferentially; for low-weight routine tasks, conventional processing strategies are adopted.

[0115] This embodiment provides an efficient data organization and computing scheduling solution for the industrial digital twin system. The octree structure and union-find set realize hierarchical management of data and improve data access efficiency. The computational complexity evaluation based on data features and the computational flow organization of directed acyclic graphs realize the optimal allocation of computing resources. This solution enables the system to adaptively adjust the data processing strategy according to the characteristics of the industrial scenario, thereby improving the operating efficiency of the digital twin system.

[0116] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 5 , and can also include the following:

[0117] Step S501: Select a computing node from the adaptive computing flow graph, apply the spatial transformation matrix to the data of the computing node, build a Markov prediction model to calculate the access probability of a data block, establish a cache queue according to the access probability, load the data in the cache queue into the video memory, build a local consistency constraint function based on the spatial distance between data blocks, and use the local consistency constraint function to calculate the data update amount;

[0118] Step S502: Perform wavelet transform on the data update amount to obtain frequency coefficients, sort the frequency coefficients into layers according to energy size, set quantization parameters for each layer of frequency coefficients, encode and compress the frequency coefficients using the quantization parameters, establish a virtual space buffer pool, and write the compressed data into the corresponding area of ​​the virtual space buffer pool.

[0119] Optionally, this embodiment implements an intelligent selection strategy for computing nodes. Based on the topological structure of the adaptive computing flow graph, nodes with high computing weight and urgency are preferentially selected. In industrial scenarios, these nodes usually correspond to status monitoring points of key equipment or control points of important processes and need to be processed preferentially.

[0120] This embodiment applies the space transformation matrix to data conversion. Through matrix transformation, the data of the physical space is mapped to the virtual space to maintain the consistency of the spatial relationship. This is of great significance for the spatial layout synchronization and state mapping of industrial equipment, ensuring that the data expression in the virtual space is consistent with the actual scene.

[0121] This embodiment constructs a Markov prediction model for data access analysis. The model predicts the access probability of a data block based on historical access patterns. For example, in a continuous production process, the status data of certain key equipment is frequently accessed, and the prediction model can be used to identify these high-frequency access data in advance.

[0122] This embodiment designs an intelligent cache management mechanism. A priority queue is established based on the access probability, and data blocks with high probability of access are loaded into the video memory first. This strategy is particularly suitable for real-time monitoring of industrial sites, which can reduce data access delays and improve response speed.

[0123] This embodiment introduces local consistency constraints to ensure the rationality of data updates. By considering the spatial distance between data blocks, a constraint function is constructed. This ensures the continuity of data updates in adjacent areas and avoids unreasonable data jumps in the virtual space.

[0124] This embodiment uses wavelet transform to process data update amount. By converting the data update amount into frequency domain space, it is possible to effectively separate the change characteristics of different scales. This can accurately capture both subtle state changes and drastic state transitions of industrial equipment.

[0125] This embodiment implements energy-based frequency coefficient stratification. The frequency coefficients are sorted by energy size, with high energy coefficients corresponding to significant change features and low energy coefficients corresponding to detailed changes. This stratification strategy enables the system to select the appropriate accuracy level according to actual needs.

[0126] This embodiment designs an adaptive quantization coding scheme. For frequency coefficients of different levels, corresponding quantization parameters are set. Fine quantization is used for important levels to maintain accuracy, and coarse quantization is used for secondary levels to reduce the amount of data. This differentiated coding strategy effectively reduces storage pressure while ensuring data quality.

[0127] This embodiment constructs a virtual space buffer pool for data management. The buffer pool is partitioned according to the spatial location and update frequency of the data, and the compressed data is written to the corresponding area. This organization method facilitates the rapid retrieval and update of data and supports the efficient operation of the industrial digital twin system.

[0128] This embodiment provides an efficient data processing and cache management solution for the industrial digital twin system. Through Markov prediction and intelligent cache strategy, data access efficiency is improved. Wavelet transform and adaptive coding compression achieve efficient data storage. This solution enables the system to respond to the data update needs of the industrial site in a timely manner, ensuring the real-time and reliability of the digital twin system.

[0129] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 6 , and can also include the following:

[0130] Step S601: establishing a data transmission link between computing nodes, allocating a buffer for the data transmission link, dividing the buffer into two read and write areas, setting a synchronization signal between the read and write areas, building a pipeline channel based on the synchronization signal, performing task segmentation on the pipeline channel, and transmitting incremental mapping data in parallel in each task segment;

[0131] Step S602: Calculate the data change rate of the incremental mapping data, set an update priority threshold according to the data change rate, divide the update priority threshold into multiple levels, establish a timestamp index for each level of data, and organize the incremental mapping data into a refresh sequence based on the timestamp index.

[0132] Optionally, this embodiment establishes an efficient data transmission mechanism. A dedicated data transmission link is established between computing nodes to ensure the stability and reliability of data flow. In industrial scenarios, these links carry the real-time transmission of important data such as equipment status and process parameters, which is crucial to system operation.

[0133] This embodiment implements a dual buffer data management strategy. The buffer is divided into independent read and write areas, and the data read and write operations are coordinated through synchronization signals. For example, when one area is writing data, another area can read data at the same time, avoiding read and write conflicts and improving data throughput.

[0134] This embodiment constructs a pipeline data transmission channel. Based on the synchronization signal mechanism, the data transmission process is divided into multiple continuous processing stages. Each stage is responsible for a specific data processing task, such as data packaging, verification, transmission, etc., and each stage runs in parallel to form an efficient data processing pipeline.

[0135] This embodiment realizes the parallel transmission of incremental mapping data. Through task segmentation, a large amount of incremental data is divided into multiple independent data packets and transmitted in parallel in the pipeline channel. This method is particularly suitable for real-time data updates in industrial sites and can quickly respond to dynamic changes in equipment status.

[0136] This embodiment designs a method for calculating the data change rate. By comparing the data differences at adjacent time points, the data change rate is calculated. The change rate reflects the severity of the state change of the industrial equipment and provides a basis for the formulation of data update strategies.

[0137] This embodiment establishes a multi-level update priority mechanism. Different priority thresholds are set according to the data change rate to divide the data into multiple levels. For example, the rapid change of key parameters of the equipment is given the highest priority and needs to be updated immediately; while the regular parameters that change slowly are arranged at a lower priority.

[0138] This embodiment implements a timestamp-based data index. A timestamp index is established for each level of data to record the time series information of data generation and update. This index mechanism supports historical tracing and time series analysis of data, which is helpful for monitoring and optimization of industrial processes.

[0139] This embodiment organizes an orderly data refresh sequence. Based on the timestamp index, the incremental mapping data is organized into a refresh sequence according to priority and timing. This ensures the timely update of important data while maintaining the overall orderliness of data updates.

[0140] This embodiment provides an efficient data transmission and update solution for the industrial digital twin system. The double buffer and pipeline mechanism improve the data transmission efficiency, and the multi-level priority and timestamp index ensure the timeliness and orderliness of data updates. This solution can effectively process a large amount of real-time data at the industrial site, ensure the synchronization of the virtual space and the physical device status, and support real-time monitoring and intelligent decision-making of industrial processes.

[0141] In one embodiment of the information mapping method of the intelligent perception information in the virtual space of the present application, see Figure 7 , and can also include the following:

[0142] Step S701: construct a state verification table for each data block, record the data version number in the state verification table, calculate the checksum value of the data block, add the checksum value to the state verification table, compare the state verification tables of different computing nodes, construct a consistency score based on the difference between the state verification tables, and trigger a data synchronization operation according to the consistency score;

[0143] Step S702: Group the computing nodes, calculate the processing capacity index of each group of computing nodes, input the processing capacity index into the load prediction model, calculate the resource utilization based on the load prediction model, dynamically allocate computing tasks according to the resource utilization, establish a status broadcast channel between computing nodes, and broadcast the perception information status to the target computing node.

[0144] Optionally, this embodiment first makes an innovative design for data block management. Considering the version management requirements of massive data in the industrial digital twin system, a status verification table is constructed for each data block. This verification table contains basic information such as version number and timestamp. Since the status data of industrial field equipment is constantly updated, the version number is designed in an incremental manner to ensure the timing of data updates and facilitate tracking of data change history.

[0145] This embodiment uses a checksum algorithm for data integrity verification. In specific implementation, the system reads the content of the data block and calculates a checksum value of a fixed length through a hash function. This checksum value corresponds one-to-one with the data content and can effectively detect whether the data has been tampered with or transmitted incorrectly. For example, when a bit error occurs in the status data of an industrial device during transmission, the recalculated checksum will not match the original value, and the system can detect and handle it in time.

[0146] This embodiment innovatively proposes a method for calculating the difference of the status verification table. The system collects the status verification tables on different computing nodes and calculates the difference of the data status by comparing the version number and checksum. The difference calculation takes into account the importance weight of the data block and gives a higher weight to the status data of key devices. This design ensures that the consistency of important data is guaranteed first.

[0147] This embodiment builds a consistency scoring mechanism based on the difference. The scoring calculation comprehensively considers the timeliness, integrity and consistency of the data. When the score is lower than the preset threshold, the data synchronization operation is triggered. This mechanism can avoid unnecessary synchronization overhead while ensuring data consistency.

[0148] This embodiment adopts a hierarchical grouping strategy in computing resource management. First, the nodes are grouped according to their physical location and network topology, and then they are subdivided based on hardware configuration and processing power. This grouping method takes into account the actual situation of the industrial site, can make full use of network bandwidth, and reduce communication delay.

[0149] This embodiment designs a multi-dimensional processing capacity evaluation method. By monitoring indicators such as CPU usage, memory usage, and IO load, combined with historical processing data, a processing capacity indicator system is constructed. These indicators are input into the load prediction model, and the model uses a time series analysis method to predict future resource demand trends.

[0150] This embodiment implements an intelligent task scheduling mechanism. Based on the output of the load prediction model, the system calculates the resource utilization of each node group. When the utilization of a node group exceeds the threshold, the system automatically assigns new tasks to the node group with lighter load to achieve load balancing. This dynamic scheduling is particularly suitable for scenarios with large load fluctuations in industrial production.

[0151] This embodiment establishes an efficient status broadcast mechanism. By building a dedicated broadcast channel, the status information is propagated in a publish-subscribe mode. The design of the broadcast channel takes into account the priority of data, and emergency status information can be transmitted first to ensure that the system can respond quickly to abnormal situations.

[0152] Through the above technical innovations, this embodiment effectively solves the problems of data consistency maintenance and resource scheduling optimization in the industrial digital twin system. This solution can timely discover and handle data inconsistency problems, ensuring the reliability of the system; at the same time, through intelligent resource scheduling, it improves the processing efficiency of the system and provides strong support for the stable operation of the industrial digital twin system.

[0153] In order to effectively solve the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data, the present application provides an embodiment of an intelligent perception information mapping device in a virtual space for implementing all or part of the content of the information mapping method of intelligent perception information in a virtual space, see Figure 8 The information mapping device of the intelligent perception information in the virtual space specifically includes the following contents:

[0154] The transformation matrix construction module 10 is used to deploy a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, write the intelligent perception data into a multidimensional data buffer matrix in timestamp order, construct a multi-scale feature pyramid to extract data feature vectors, use an attention mechanism to weight the importance of the data feature vectors, train a self-supervised spatial feature extractor based on temporal consistency constraints, mark corresponding point sets in the scene space and the virtual space, and solve the spatial transformation matrix in combination with geometric invariance constraints;

[0155] A mapping execution module 20 is used to divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on a union-find set, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the results of the mapping transformation into a virtual space cache;

[0156] The state synchronization module 30 is used to build a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0157] From the above description, it can be seen that the information mapping device of intelligent perception information in virtual space provided by the embodiment of the present application can collect perception data by deploying a multi-sensor array, extract data features based on a multi-scale feature pyramid and attention mechanism, and establish a spatial mapping relationship using self-supervised learning and geometric constraints. A hierarchical index tree based on union-find is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source perception data.

[0158] From the hardware level, in order to effectively solve the performance bottleneck problem of traditional mapping solutions when processing large-scale multi-source perception data, the present application provides an embodiment of an electronic device for implementing all or part of the content of the information mapping method of intelligent perception information in a virtual space, and the electronic device specifically includes the following content:

[0159] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between the information mapping device of intelligent perception information in virtual space and related devices such as core business system, user terminal and related database; the logic controller can be a desktop computer, tablet computer and mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the logic controller can be implemented with reference to the embodiment of the information mapping method of intelligent perception information in virtual space and the embodiment of the information mapping device of intelligent perception information in virtual space in the embodiment, and the contents thereof are incorporated herein, and the repeated parts are not repeated.

[0160] It is understandable that the user terminal may include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, etc. Among them, the smart wearable device may include smart glasses, a smart watch, a smart bracelet, etc.

[0161] In practical applications, part of the information mapping method of intelligent perception information in virtual space can be executed on the electronic device side as described above, or all operations can be completed in the client device. The specific selection can be based on the processing capability of the client device and the limitations of the user's usage scenario. This application does not limit this. If all operations are completed in the client device, the client device may also include a processor.

[0162] The client device may have a communication module (i.e., a communication unit) that can communicate with a remote server to achieve data transmission with the server. The server may include a server on the task scheduling center side, and other implementation scenarios may also include a server on an intermediate platform, such as a server on a third-party server platform that has a communication link with the task scheduling center server. The server may include a single computer device, or a server cluster consisting of multiple servers, or a server structure of a distributed device.

[0163] Fig. 9 FIG. 9 is a schematic block diagram of the system structure of the electronic device 9600 according to an embodiment of the present application. Fig. 9 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig. 9 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0164] In one embodiment, the function of the information mapping method of intelligent perception information in the virtual space can be integrated into the central processing unit 9100. The central processing unit 9100 can be configured to perform the following control:

[0165] Step S101: deploying a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, writing the intelligent perception data into a multidimensional data buffer matrix in timestamp order, constructing a multi-scale feature pyramid to extract data feature vectors, using an attention mechanism to weight the importance of the data feature vectors, training a self-supervised spatial feature extractor based on temporal consistency constraints, marking corresponding point sets in the scene space and the virtual space, and solving the spatial transformation matrix in combination with geometric invariance constraints;

[0166] Step S102: Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache;

[0167] Step S103: construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0168] From the above description, it can be seen that the electronic device provided by the embodiment of the present application collects sensory data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and establishes a spatial mapping relationship using self-supervised learning and geometric constraints. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source sensory data.

[0169] In another embodiment, the information mapping device of intelligent perception information in the virtual space can be configured separately from the central processing unit 9100. For example, the information mapping device of intelligent perception information in the virtual space can be configured as a chip connected to the central processing unit 9100, and the function of the information mapping method of intelligent perception information in the virtual space is realized through the control of the central processing unit.

[0170] like Fig. 9 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig. 9 In addition, the electronic device 9600 may also include Fig. 9 For components not shown, reference may be made to the prior art.

[0171] like Fig. 9 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0172] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0173] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0174] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0175] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0176] The communication module 9110 is a transmitter / receiver that sends and receives signals via the antenna 9111. The communication module 9110 (transmitter / receiver) is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0177] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module 9110 (transmitter / receiver) is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0178] The embodiments of the present application also provide a computer-readable storage medium capable of implementing all the steps of the information mapping method of the intelligent perception information in the virtual space in the above-mentioned embodiment, in which the execution subject is a server or a client. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, all the steps of the information mapping method of the intelligent perception information in the virtual space in the above-mentioned embodiment are implemented. For example, when the processor executes the computer program, the following steps are implemented:

[0179] Step S101: deploying a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, writing the intelligent perception data into a multidimensional data buffer matrix in timestamp order, constructing a multi-scale feature pyramid to extract data feature vectors, using an attention mechanism to weight the importance of the data feature vectors, training a self-supervised spatial feature extractor based on temporal consistency constraints, marking corresponding point sets in the scene space and the virtual space, and solving the spatial transformation matrix in combination with geometric invariance constraints;

[0180] Step S102: Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache;

[0181] Step S103: construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0182] From the above description, it can be seen that the computer-readable storage medium provided in the embodiment of the present application collects sensory data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and establishes a spatial mapping relationship using self-supervised learning and geometric constraints. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source sensory data.

[0183] The embodiments of the present application also provide a computer program product capable of implementing all the steps of the information mapping method of intelligent perception information in a virtual space in the above embodiments, where the execution subject is a server or a client. When the computer program / instruction is executed by a processor, the steps of the information mapping method of intelligent perception information in a virtual space are implemented. For example, the computer program / instruction implements the following steps:

[0184] Step S101: deploying a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, writing the intelligent perception data into a multidimensional data buffer matrix in timestamp order, constructing a multi-scale feature pyramid to extract data feature vectors, using an attention mechanism to weight the importance of the data feature vectors, training a self-supervised spatial feature extractor based on temporal consistency constraints, marking corresponding point sets in the scene space and the virtual space, and solving the spatial transformation matrix in combination with geometric invariance constraints;

[0185] Step S102: Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache;

[0186] Step S103: construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

[0187] From the above description, it can be seen that the computer program product provided by the embodiment of the present application collects sensory data by deploying a multi-sensor array, extracts data features based on a multi-scale feature pyramid and an attention mechanism, and establishes a spatial mapping relationship using self-supervised learning and geometric constraints. A hierarchical index tree based on a union-find set is used to organize local data blocks, an adaptive computational flow graph is constructed to achieve efficient mapping, and predictive caching and differential update strategies are combined to optimize data transmission. At the same time, parallel processing of incremental mapping is achieved through pipeline data channels, and a dynamic scheduling mechanism based on load prediction ensures data synchronization efficiency. This method effectively solves the performance bottleneck problem of traditional mapping schemes when processing large-scale multi-source sensory data.

[0188] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0189] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0192] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for mapping intelligent perception information in a virtual space, characterized in that: The method comprises: Deploy camera arrays, temperature sensor arrays, and sound sensor arrays in the scene space to collect intelligent perception data, write the intelligent perception data into a multidimensional data buffer matrix in timestamp order, construct a multi-scale feature pyramid to extract data feature vectors, use an attention mechanism to weight the importance of the data feature vectors, train a self-supervised spatial feature extractor based on temporal consistency constraints, mark corresponding point sets in the scene space and virtual space, and solve the spatial transformation matrix in combination with geometric invariance constraints; Divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on union-find set, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the result of the mapping transformation into a virtual space cache; Construct a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

2. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The camera array, the temperature sensor array, and the sound sensor array are deployed in the scene space to collect intelligent perception data, the intelligent perception data is written into a multidimensional data buffer matrix in a timestamp order, and a multi-scale feature pyramid is constructed to extract data feature vectors, including: The scene space is divided into grid units, and three groups of sensor arrays are set at the boundary position of each grid unit, wherein the three groups of sensor arrays include a six-degree-of-freedom camera array, a temperature sensor array, and a sound sensor array, and the sensor array sampling coordinates are calculated based on the spatial position of the grid unit to collect the intelligent perception data in the grid unit; A time window threshold is set for the intelligent perception data, the intelligent perception data within the time window threshold is stored in a four-dimensional data buffer matrix, a three-layer pyramid structure is constructed based on the four-dimensional data buffer matrix, a feature map of each layer of the pyramid is calculated by a Gaussian filter, non-maximum suppression is performed on the feature map to extract data feature points, and the data feature points are mapped to a feature space to generate a data feature vector.

3. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The attention mechanism is used to weight the importance of the data feature vector, a self-supervised spatial feature extractor is trained based on temporal consistency constraints, corresponding point sets are marked in the scene space and the virtual space, and the spatial transformation matrix is ​​solved in combination with geometric invariance constraints, including: Divide the data feature vector into feature blocks of fixed size, calculate the self-attention weight matrix of the feature block, weight the feature block using the self-attention weight matrix, calculate the temporal correlation score based on the weighted feature block, construct a temporal consistency loss function, and use the gradient descent method to train the network parameters of the spatial feature extractor; A plurality of corner points are selected as marking points in the scene space, and the three-dimensional coordinates of the marking points are calculated. The three-dimensional coordinates are mapped to the virtual space to generate a corresponding point set, and a space transformation equation group is established using the corresponding point set. Rotational and translational invariance constraints are introduced into the space transformation equation group, and the parameter values ​​of the space transformation matrix are solved by the least squares method.

4. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The step of dividing the intelligent sensing data into local data blocks, constructing a hierarchical index tree based on a union-find set, performing complexity evaluation on the local data blocks, and generating an adaptive computation flow graph includes: An octree structure is constructed in the smart sensing data, a data space is divided into local data blocks based on the octree structure, boundary coordinates of the local data blocks are calculated, a union-find data structure is constructed using the boundary coordinates, the local data blocks are added to a union-find node, and a hierarchical index tree is established based on the connectivity relationship of the union-find node; Calculate the data density distribution of the local data block, set a computational complexity threshold based on the data density distribution, convert the computational complexity threshold into a computational weight coefficient, construct a directed acyclic graph according to the computational weight coefficient, and use the directed acyclic graph as the topological structure of the adaptive computational flow graph.

5. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The step of performing mapping transformation at the nodes of the adaptive computation flow graph, using a predictive cache mechanism to load high access probability data in advance, generating a differential update strategy based on a local consistency constraint, and using hierarchical compression coding to write the result of the mapping transformation into a virtual space cache includes: Selecting a computing node from the adaptive computing flow graph, applying the spatial transformation matrix to the data of the computing node, constructing a Markov prediction model to calculate the access probability of a data block, establishing a cache queue according to the access probability, loading the data in the cache queue into a video memory, constructing a local consistency constraint function based on the spatial distance between data blocks, and calculating the data update amount using the local consistency constraint function; Perform wavelet transform on the data update amount to obtain frequency coefficients, sort the frequency coefficients into layers according to energy size, set quantization parameters for each layer of frequency coefficients, encode and compress the frequency coefficients using the quantization parameters, establish a virtual space buffer pool, and write the compressed data into the corresponding area of ​​the virtual space buffer pool.

6. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The step of constructing a pipeline data channel, transmitting incremental mapping data in parallel in the pipeline data channel, and organizing the incremental mapping data into a refresh sequence according to an update priority includes: Establish a data transmission link between computing nodes, allocate a buffer for the data transmission link, divide the buffer into two read and write areas, set a synchronization signal between the read and write areas, build a pipeline channel based on the synchronization signal, perform task segmentation on the pipeline channel, and transmit incremental mapping data in parallel in each task segment; The data change rate of the incremental mapping data is calculated, an update priority threshold is set according to the data change rate, the update priority threshold is divided into multiple levels, a timestamp index is established for data of each level, and the incremental mapping data is organized into a refresh sequence based on the timestamp index.

7. The method for mapping intelligent perception information in virtual space according to claim 1, characterized in that: The method of ensuring state synchronization by using a data consistency verification mechanism, establishing an adaptive computing resource pool, dynamically scheduling computing nodes based on load prediction, and synchronizing the perception information state to the computing nodes includes: Construct a state verification table for each data block, record the data version number in the state verification table, calculate the checksum value of the data block, add the checksum value to the state verification table, compare the state verification tables of different computing nodes, construct a consistency score based on the difference between the state verification tables, and trigger a data synchronization operation according to the consistency score; The computing nodes are grouped into resource groups, the processing capacity index of each group of computing nodes is calculated, the processing capacity index is input into the load prediction model, the resource utilization is calculated based on the load prediction model, the computing tasks are dynamically allocated according to the resource utilization, a status broadcast channel between computing nodes is established, and the perception information status is broadcast to the target computing node.

8. An information mapping device for intelligent perception information in virtual space, characterized in that: The device comprises: A transformation matrix construction module is used to deploy a camera array, a temperature sensor array, and a sound sensor array in the scene space to collect intelligent perception data, write the intelligent perception data into a multidimensional data buffer matrix in timestamp order, construct a multi-scale feature pyramid to extract data feature vectors, use an attention mechanism to weight the importance of the data feature vectors, train a self-supervised spatial feature extractor based on temporal consistency constraints, mark corresponding point sets in the scene space and the virtual space, and solve the spatial transformation matrix in combination with geometric invariance constraints; A mapping execution module, used to divide the intelligent perception data into local data blocks, construct a hierarchical index tree based on a union-find set, perform complexity evaluation on the local data blocks, generate an adaptive computation flow graph, perform mapping transformation on the nodes of the adaptive computation flow graph, use a predictive cache mechanism to load high access probability data in advance, generate a differential update strategy based on local consistency constraints, and use hierarchical compression coding to write the results of the mapping transformation into a virtual space cache; A state synchronization module is used to build a pipeline data channel, transmit incremental mapping data in parallel in the pipeline data channel, organize the incremental mapping data into a refresh sequence according to the update priority, use the data consistency verification mechanism to ensure state synchronization, establish an adaptive computing resource pool, dynamically schedule computing nodes based on load prediction, and synchronize the perception information state to the computing nodes.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the information mapping method of intelligent perception information in a virtual space as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for mapping intelligent perception information in a virtual space as described in any one of claims 1 to 7 are implemented.

Images