Three-dimensional logistics center modeling method and system based on digital twinning

Abstract

The invention relates to the technical field of data processing, in particular to a three-dimensional logistics center modeling method and system based on digital twinning, and the method comprises the steps: carrying out the voxelization of a point cloud of a logistics center to obtain the initial features of all voxel grids, constructing a multi-layer point cloud pyramid, determining the fusion features of the voxel grids, and carrying out the modeling of the three-dimensional logistics center based on digital twinning. Calculating feature similarity based on the fusion features and the interlayer distance, constructing a mapping relation based on the feature similarity, updating the fusion features of the corresponding voxel grids of the voxel grids and calculating interlayer propagation factors, screening significant feature points based on the interlayer propagation factors, determining an initial matching point pair set according to the significant feature points, and performing matching on the initial matching point pair set according to the initial matching point pair set; an iterative nearest point algorithm is used to initialize and register the three-dimensional point cloud to obtain a three-dimensional point cloud model, and a dynamic target object in the video data of the logistics center and the three-dimensional point cloud model are registered and fused, so that the actual change state of the logistics center can be reflected in real time, the point cloud registration precision is improved, and the real-time performance of the logistics center is improved. And the modeling accuracy of the three-dimensional logistics center is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and more particularly to a three-dimensional logistics center modeling method and system based on digital twins. Background Art

[0002] With the continuous development of digital twin technology in the field of logistics, higher requirements are placed on the accuracy and efficiency of logistics center modeling. Digital twins aim to build virtual digital models of physical entities so that they can be accurately mapped and synchronized with physical entities throughout their life cycle, and accurate three-dimensional point cloud matching is the key prerequisite for realizing a high-fidelity digital twin model of a logistics center. In the digital twin scenario, the real-time status of the logistics center needs to be reflected in the virtual model through accurate point cloud registration to support complex applications such as real-time inventory monitoring, equipment failure prediction, and intelligent scheduling optimization.

[0003] Point cloud registration is a key step in modeling. By accurately positioning the source point cloud and the target point cloud, data collected from different perspectives or time periods can be accurately aligned. For example, in logistics facility layout monitoring, registration makes multi-source point clouds fit together and unifies the coordinate system, providing a framework for building accurate geometric models. Otherwise, the geometric model is prone to distortion and inaccuracy, affecting facility mapping and inventory counting accuracy, leading to decision-making bias. The iterative closest point algorithm is a classic algorithm for rigid point cloud registration. The core idea is to iteratively find the closest point pair between two point clouds and calculate the optimal rigid transformation (including rotation and translation) to align the source point cloud in the coordinate system of the target point cloud.

[0004] For example, the existing Chinese patent document with the announcement number CN113160129B discloses a fast registration method for a combined simplified point cloud, which uses k-nearest neighbors to identify the boundary features of a given scattered point cloud, distinguishes between feature and non-feature areas through curvature, divides the feature area into a cubic grid, and then compares the uniform curvature of the neighborhood of each point with the average curvature of the entire point cloud. The non-feature area is simplified based on the centroid method, and the PCA algorithm is used for coarse registration of the point cloud. The geometric features of the roughly registered point cloud are extracted, and then the iterative nearest point algorithm is used for fine registration.

[0005] However, the iterative closest point algorithm is highly dependent on the initial transformation. If the initial registration is not good, the iteration is prone to fall into the local optimal solution. The above technical solution is based on geometric features and uses PCA principal component analysis to perform preliminary alignment through the geometric characteristics of the point cloud. In complex logistics scenarios, these initialization methods have some shortcomings because the distribution of point clouds is affected by factors such as shelf layout and cargo stacking. The method based on geometric features is effective when the geometric structure of the point cloud is obvious, but for complex and irregular point clouds, especially in logistics scenarios where goods have different shapes and are severely occluded, it is impossible to accurately extract effective geometric features, resulting in initial position deviation, which easily causes the iterative closest point algorithm to deviate from the global optimum and fall into the local optimum, affecting the accuracy of point cloud registration, and thus failing to provide a reliable model basis for logistics management, and it is difficult to effectively support accurate operational decisions and efficient resource allocation. Summary of the invention

[0006] In order to solve the problem that the method of preliminary alignment by geometric characteristics of point cloud in modeling three-dimensional logistics center is easy to cause initial position deviation, it is easy to make the iterative closest point algorithm deviate from the global optimum, and affect the accuracy of point cloud registration and the reliability of modeling. The present invention proposes a three-dimensional logistics center modeling method based on digital twin, including: Collect point cloud and video data of the logistics center. The point cloud is voxelized to form multiple voxel grids. The initial features of each voxel grid are formed by the number of point clouds, centroid, and eigenvalues ​​of covariance matrix decomposition of each voxel grid, and dynamic target objects in the video data are identified; Construct a multi-layer point cloud pyramid, determine the fusion features of each voxel grid based on the inter-layer distance and initial feature difference between each voxel grid and each voxel grid in its adjacent layer, calculate the feature similarity of each voxel grid and the voxel grids in the layers above it based on the fusion features and inter-layer distance, and construct a mapping relationship to determine the voxel grids corresponding to each voxel grid in the layers above it based on the principle of maximizing feature similarity; Update the fusion features of the voxel grids corresponding to each voxel grid in each layer above it, calculate the inter-layer propagation factor of each voxel grid according to the updated fusion features, take the center of the voxel grid whose inter-layer propagation factor is greater than the threshold as a significant feature point, determine the initial matching point pair set of the source point cloud and the target point cloud based on all the significant feature points, and use the iterative closest point algorithm to initialize and register the three-dimensional point cloud model based on the initial matching point pair set; By using 3D reconstruction technology and spatial transformation algorithm, the 2D image information of the dynamic target object is mapped to the 3D point cloud model, and the registration and fusion of the dynamic target object and the point cloud model are achieved, thus completing the 3D modeling.

[0007] The above technical solution structures the complex point cloud through voxelization to form a voxel grid. This division method helps to reduce the complexity of the data and make it easier to manage and analyze. The initial features can describe the characteristics of the voxel grid from multiple dimensions. Furthermore, the construction of a multi-layer point cloud pyramid realizes the hierarchical representation of the point cloud, and can analyze the characteristics of the point cloud at different scales. This multi-scale analysis method helps to capture the details and overall structural information of the point cloud at different levels. The fusion features of the voxel grid are determined by fully combining the spatial position and its own characteristics of the point cloud, making the fusion features more representative and distinguishable. The feature similarity is calculated based on the fusion features and the inter-layer distance, and the mapping relationship is constructed according to the maximization principle. The connection between the starting point clouds at different scales is established, which provides a more accurate matching basis for subsequent processing. Furthermore, updating the fusion features and calculating the inter-layer propagation factor further optimizes the extraction and utilization of point cloud features. The inter-layer propagation factor reflects the degree of association and importance between different layers of the voxel grid. By setting a threshold to screen out significant feature points, the key information in the point cloud can be highlighted, and unnecessary calculations and noise interference can be reduced. Based on these significant feature points, the initial matching point pair set is determined, which provides a reliable initial condition for the iterative closest point algorithm, and can guide the iterative process in a direction closer to the global optimal solution, solving the problem of the traditional iterative closest point algorithm deviating from the global optimal solution due to the initial position deviation, and improving the accuracy of point cloud registration. Furthermore, by mapping the two-dimensional image information of the dynamic target object to the three-dimensional point cloud model, the three-dimensional model is given dynamic characteristics. This process is the key embodiment of the digital twin concept in the modeling of the logistics center, which can show the actual operation status of the logistics center in real time in the model, making the model closer to the real logistics scene, and providing more intuitive, comprehensive and real-time data support for the management, optimization and decision-making of the logistics center.

[0008] Preferably, the fusion characteristics of each voxel grid satisfy the following relationship: ; In the formula, , They are Layer Voxel Grid The fusion features and initial features of 、 Respectively Layer, Layer to layer The influence weight of the layer, , , It is Layer A voxel grid, It is Layer A voxel grid, , They are , The initial characteristics of for and The similarity weight of yes and The similarity weight of and The calculation method is consistent with , , They are Layer, Layer, The total number of voxel grids each layer contains.

[0009] The above technical solution combines the initial features of each voxel grid with the feature information of the adjacent layers, fully considering the point cloud features at different scales. This fusion method is not limited to the information of the current layer, but integrates the features of the upper and lower layers, making the feature description of each voxel grid more comprehensive and rich, and can capture the details and overall features of the point cloud at different hierarchical structures.

[0010] Preferably, the calculation formula for the feature similarity between each voxel grid and the voxel grids of each layer above it is: ; In the formula, It is Layer The voxel grid and Layer The feature similarity between individual voxel grids, It is Layer The voxel grid and Layer The inter-layer distance of the voxel grid is Layer The centroid of the voxel grid and the Layer The Euclidean distance between the centroids of the voxel grid, It is Layer Fusion features of voxel grids and Layer Fusion features of voxel grids The cosine similarity between .

[0011] The above technical solution combines the spatial position relationship and intrinsic feature information of voxel grids. The inter-layer distance reflects the relative position distribution of voxel grids in three-dimensional space, and the cosine similarity of the fused features measures the similarity between two voxel grids from the perspective of data features. This means that when evaluating the similarity between voxel grids, both their spatial proximity and the consistency of their features are taken into account, avoiding the limitation of relying solely on a single factor for similarity judgment, thereby providing a more comprehensive and accurate similarity metric.

[0012] Preferably, the method for determining the voxel grid corresponding to each voxel grid in each layer above it is: For Layer Voxel Grid , will Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , then, As a new starting point, calculate With The feature similarity of each voxel grid in the layer Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , continue with As a new starting point, continue to iterate until the highest level, and finally Until Found in the layer The corresponding voxel grid is constructed The mapping relationship between the voxel grids of the layers above it is obtained In the The corresponding voxel grid in the layers above the layer.

[0013] The above technical solution adopts an iterative approach to gradually advance from adjacent layers to higher layers to find the corresponding voxel grids, and can gradually establish a complete mapping relationship along the hierarchical structure of the point cloud. This layer-by-layer processing process is consistent with the idea of ​​building a multi-layer point cloud pyramid, and fully utilizes the advantages brought by data stratification, so that the voxel grids of each layer can find the most suitable corresponding voxel grids in the upper layer in an orderly and systematic manner based on the existing corresponding relationships, avoiding the occurrence of matching situations with too large a span and lack of coherence, and ensuring the systematic and accurate construction of the mapping relationship.

[0014] Preferably, the method for updating the fusion features of the voxel grids corresponding to the voxel grids in the layers above each voxel grid is: ; In the formula, For the Layer Voxel Grid exist The corresponding voxel grid in the layer, for The fusion characteristics of for Updated fusion features, For the Layer Voxel Grid On top of it The corresponding voxel grid in the layer and The corresponding voxel grid in the layer The similarity of features between and They are In the The sequence number in the layer and In the The sequence number within the layer.

[0015] The above technical solution updates the fusion features based on the feature similarity between voxel grids, which reflects a dynamic and highly correlated idea. By integrating the features of the voxel grids corresponding to the adjacent layers into the updating process of the fusion features of the voxel grids corresponding to the current layer according to the similarity weights, the fusion features can be continuously adjusted and optimized with the inter-layer relationship, better reflecting the correlation between point clouds at different levels and the overall structural characteristics, so that the fusion features are more in line with the distribution and change laws of point clouds in actual logistics center scenarios.

[0016] Preferably, the calculation formula of the interlayer propagation factor of each voxel grid is:

[0017] In the formula, For the Layer Voxel Grid The inter-layer propagation factor, for The corresponding voxel grid in the top level, for Fusion features and Updated fusion features The degree of difference, For the Layer Voxel Grid On top of it The layer above The fused features of the corresponding voxel grid in the layer and The corresponding voxel grid in the highest layer is updated with the fused features The degree of difference is calculated based on the Euclidean distance.

[0018] The above technical scheme uses the degree of feature difference to define the inter-layer propagation factor. The greater the difference in features between a voxel grid and the voxel grid corresponding to the highest layer, the higher the "influence" or "value" of the voxel grid in the process of information propagation and feature transfer. Correspondingly, the greater the inter-layer propagation factor of the voxel grid. This method of measuring the propagation factor based on the degree of feature difference is in line with the inherent requirement of point cloud processing to highlight the parts that are strongly related to the overall goal, so as to more effectively screen key information and perform accurate modeling in the future. It can reasonably quantify the importance of each layer of voxel grids in the entire modeling information flow.

[0019] Preferably, the method for determining the initial matching point pair set of the source point cloud and the target point cloud based on the significant feature points of each layer is as follows: set the source point cloud pyramid A1 and the target point cloud pyramid A2, and for the significant feature point A of the highest layer of A1, perform the following operations in the highest layer of the target point cloud A2: calculate the Euclidean distance between each significant feature point of the highest layer of A2 and A, and obtain the cosine similarity of the fusion feature of each significant feature point and A, and then select the significant feature point B with the smallest Euclidean distance and the largest cosine similarity with A in the highest layer of A2 as the matching point of A, A and B form a matching point pair, and then obtain the matching point pair set of the highest layer of A1 and the highest layer of A2; for the next layer of the highest layer of A1, mark the point pairs that have been matched in the previous layer, and for the unmatched point pairs, use the same method to determine the matching point pairs at the highest layer, and so on, match layer by layer, and use the initial matching point pair set of the lowest layer as the initial matching point pair set of the source point cloud and the target point cloud.

[0020] The above technical solution adopts a hierarchical matching strategy that starts from the highest layer, marks the matched point pairs, and then matches the unmatched point pairs layer by layer in the same way. It is consistent with the idea of ​​building a point cloud pyramid and the entire multi-layer point cloud processing. In the multi-layer point cloud structure, high-level data often reflects more macro and overall characteristics. Matching at the high level first can determine the large corresponding framework, and then refine the matching at the lower level based on this framework. In this way, matching operations can be performed in an orderly manner along the hierarchical structure of the point cloud, making full use of the information characteristics of each layer, ensuring the coherence and systematicness of the matching process, reducing the blindness of matching, and making the entire matching process more in line with the internal logic of the point cloud from the whole to the part, from the macro to the micro, thereby improving the accuracy and reliability of matching.

[0021] Preferably, after completing the three-dimensional modeling, it also includes: establishing a business operation process model of the logistics center: decomposing the business process of the logistics center into multiple operation steps and events, and identifying the various entities involved and their interaction relationships, the various entities including: equipment, personnel, and goods; creating a logical model of each operation step and event to obtain the execution sequence structure, branch path and loop mode of each operation step and event; defining the categories and attributes of each entity involved in each operation step and event to describe the attributes and state changes of each entity; setting the interaction rules and event-driven mechanism between various entities based on the logical model of each operation step and event to simulate the state changes of various entities in the operation process of the logistics center.

[0022] Preferably, after completing the three-dimensional modeling, the method further includes: Virtual data integration: collect dynamic data of dynamic target objects through various sensors, and bind and associate the dynamic data of dynamic target objects with the corresponding areas in the 3D model; Business operation process model integration: associate and bind various entities involved in the business operation process model of the logistics center with the corresponding areas in the 3D model to simulate the business operation process in the 3D model.

[0023] The present invention also provides a three-dimensional logistics center modeling system based on digital twins, the modeling system comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any one of the modeling methods.

[0024] The present invention has the following effects: The solution of the present invention provides reliable initial conditions for the iterative closest point algorithm, can guide the iterative process in a direction closer to the global optimal solution, effectively solves the problem of the traditional iterative closest point algorithm deviating from the global optimal due to initial position deviation, improves the accuracy of point cloud registration, and deeply associates and binds the business operation process model of the logistics center with the three-dimensional point cloud model, and integrates the virtual data of various entities, so that the constructed three-dimensional model is no longer just a static geometric presentation, but a digital twin model that corresponds to the physical entity of the logistics center in real time and dynamically, providing reliable data support for the intelligent management of the logistics center. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0027] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0028] Reference Figure 1 The three-dimensional logistics center modeling method based on digital twin provided by the present invention includes steps S1 to S8: S1: Collect point cloud and video data of the logistics center and perform preprocessing.

[0029] Point clouds can accurately describe the three-dimensional spatial geometry of a logistics center, including the shape, location, and size of fixed facilities such as buildings, shelves, and equipment. Through point clouds, very accurate spatial layout information can be obtained, such as the width of the aisles between shelves, the height of the warehouse, the footprint of the equipment, etc. This information is crucial for the planning of logistics centers, the optimization of equipment layout, and the evaluation of space utilization. Therefore, by using a ground-based three-dimensional laser scanner or a vehicle-mounted laser scanning method, the three-dimensional coordinates of the scanning point are obtained by emitting a laser beam and measuring the time it takes for the laser to reflect back to the instrument. Inside the logistics center, the scanner can scan along the aisles, around the shelves and equipment, and obtain their geometric shape and location information. This method can provide high-precision point cloud data, which is particularly suitable for detailed modeling of fixed facilities.

[0030] Video data provides dynamic information about the logistics center, such as personnel activities, vehicle driving, and cargo handling. It can record details of the operation process, including the operation mode of the equipment, the loading and unloading process of the cargo, and the collaboration between personnel. Video data can capture the time series information of the operation process of the logistics center, so that managers can intuitively observe how the business process progresses over time. Therefore, panoramic cameras are installed at key locations in the logistics center, such as every corner of the warehouse, the intersection of passages, and the loading and unloading area to collect video data. In order to accurately integrate the video data with the point cloud, all video acquisition devices need to be configured for time synchronization, which can be achieved through the Network Time Protocol (NTP) or other high-precision time synchronization technologies to ensure that the video frames and point clouds collected at the same time can match each other, which is convenient for subsequent data processing and analysis.

[0031] Next, preprocess the point cloud and video data: Voxel grid representation is a method of modeling three-dimensional objects in which space is divided into a regular grid of cubes called voxels (volume elements). Each voxel in the grid can store information such as color, density, or material properties, allowing for a detailed volumetric representation of the object. The technique is particularly useful for representing complex geometries and internal structures that may be difficult to capture with surface-based methods such as grids. A fundamental advantage of voxel grids is that they are simple to implement and easy to understand, as they extend the concept of a two-dimensional pixel to three dimensions. This makes voxel grids particularly useful in applications that naturally generate volumetric data.

[0032] Therefore, this step first collects the point cloud of the logistics center and performs voxelization on the point cloud, that is, the original point cloud is regarded as a large number of discrete point sets scattered in the three-dimensional space. Voxelization is to construct a regular three-dimensional grid (voxel grid), and according to the set voxel grid size s (the side length of each voxel grid is s), s is set to 1 meter (empirical value), these discrete point clouds are orderly classified into the corresponding voxel grids.

[0033] Next, a deep learning-based target detection algorithm is used to identify dynamic target objects in each frame of the video data, such as moving trucks, operators, etc.

[0034] The point cloud and video data of the logistics center complement each other to jointly build a complete logistics center model. The point cloud provides a static spatial framework, and the video shows dynamic activities within this framework. The combination of the two can achieve a full-scale and dynamic digital presentation of the logistics center, better supporting the construction and subsequent application of the digital twin model.

[0035] S2: Extract initial features of each voxel grid.

[0036] Get the covariance matrix composed of all point clouds (a point cloud is a three-dimensional coordinate) in each voxel grid, decompose the covariance matrix to get the eigenvalues, and combine the number of point clouds in the voxel grid, the centroid of the voxel grid and the eigenvalues ​​decomposed from the covariance matrix as the initial feature of the voxel grid, which is a multidimensional vector.

[0037] In a specific example: For Voxel Grid , let the number of point clouds it contains be , the centroid is And the covariance matrix is , the covariance matrix is ​​usually 3 3 (because the point cloud is three-dimensional), the covariance matrix is:

[0038] In the formula, Represents the first The coordinates of the point cloud, is a three-dimensional vector, representing The offset relative to the centroid of the voxel grid, yes The covariance matrix is ​​obtained by performing the above calculations on all points in the voxel grid and summing them up. , this matrix is ​​a 3 3, whose elements contain the variance of the point cloud in each dimension and the covariance information between dimensions, and fully describes the distribution characteristics of the point cloud within the voxel relative to the center of mass.

[0039] Next, the covariance matrix Perform eigendecomposition:

[0040] in, is a eigenvector matrix with 3 rows and 3 columns, and each element is an eigenvector. is a diagonal matrix, , In the matrix field, we create a diagonal matrix whose diagonal elements (from the upper left corner to the lower right corner) are , the rest of the elements are 0, the above is the covariance matrix The eigenvalues ​​of , which reflect the degree of discreteness (variance) of the point cloud in the voxel grid in different directions. is the eigenvector matrix The eigenvalue of the first column eigenvector of is the eigenvector matrix The eigenvalue of the second column eigenvector of is the eigenvector matrix The eigenvalues ​​of the eigenvector in the third column are key parameters for describing the geometric characteristics of the point cloud. They are of great significance for subsequent point cloud processing (such as building feature descriptors and performing point cloud registration), and help to accurately build a three-dimensional model of the logistics center and analyze the object structure and spatial layout in the logistics scene.

[0041] for In general, the number of point clouds contained in each voxel grid can intuitively present the density of the point cloud in the voxel grid, and is the basic indicator for describing the distribution of point clouds. When modeling a logistics center, sparse areas may correspond to channels or open spaces, and dense areas may be cargo storage areas or equipment gathering places. It provides key clues for scene recognition and spatial layout analysis, helps to grasp the structural outline of the logistics center from a macro perspective, and preliminarily build a functional zoning framework.

[0042] for For example, the centroid of each voxel grid is the weighted mean of all point clouds (each point cloud is a three-dimensional coordinate) in the voxel grid. As the "center of gravity" of the point cloud distribution, it accurately locates the center of the voxel grid in three-dimensional space. The centroid is the core element for determining the relative position and posture of an object. For example, calculating the centroid of the voxel grid in the shelf area can tell the overall position of the shelf. The dynamic changes of the centroids of multiple voxel grids can monitor the layout adjustments caused by changes in cargo handling and storage status, providing accurate spatial positioning basis for logistics operation monitoring and optimization, and ensuring visualization and controllability of the operation process.

[0043] for For a voxel grid, the covariance matrix of each voxel grid can fully characterize the discrete characteristics and directional change trends of the centroid of the voxel grid. The covariance matrix contains the variance values ​​of the point cloud in each dimension and the correlation information between the dimensions, reflecting the morphological differences of the point cloud in three-dimensional space, and is the core tool for analyzing the geometric structure of the point cloud. The eigenvalues ​​obtained by eigendecomposition of the covariance matrix quantify the degree of discreteness of the point cloud in the direction of the corresponding eigenvector (the size of the eigenvalue corresponds to the variance of the point cloud in the direction of the eigenvector), characterize the extension or aggregation of the point cloud, and reflect the main contour or layout extension trend of the object.

[0044] Therefore, this step comprehensively considers the above contents and constructs the initial features of each voxel grid, as shown in the first Voxel Grid The initial characteristics .

[0045] S3: Build a multi-layer point cloud pyramid and obtain the fusion features of each voxel grid.

[0046] The logistics center scene is complex and changeable, containing many objects of different shapes and positions as well as complex spatial layouts. After the original point cloud is voxelized, the initial features of each voxel grid can reflect certain local information, but they are relatively isolated and limited, making it difficult to fully and accurately describe the entire scene.

[0047] Therefore, this step will construct a multi-layer point cloud pyramid, comprehensively consider the spatial distance and initial feature differences between the voxel grid and its adjacent layer voxel grid, to obtain the fusion features of each voxel grid, effectively integrate information at different scales, enrich the feature expression of the voxel grid, and make it more representative and distinguishable. This not only helps to improve the ability to understand complex scenes, such as more accurately identifying the type of goods, shelf structure, etc., but also enhances the robustness of the model to noise, data loss and dynamic changes of objects, and reduces modeling errors caused by inaccurate local features.

[0048] Specifically, first build a point cloud pyramid: This step constructs a multi-layer point cloud pyramid in a bottom-up manner. Each layer represents the abstraction and organization of point clouds at different resolutions. The bottom layer has the highest resolution and the smallest voxel grid size, which can accurately capture the details of the original point cloud, such as the subtle structure of the shelves in the logistics center, the precise contours of the goods, and other microscopic features; the voxel grid size of the high-level layers increases step by step, the data is more abstract, and a macroscopic structure is presented.

[0049] When constructing a point cloud pyramid for a complex building scene, the bottom-level point cloud voxel grid is very small, and can accurately capture every tiny texture and detail on the building surface, with extremely high resolution. As higher levels are constructed, multiple bottom-level voxels are merged into a larger voxel grid to form a more abstract representation. At this time, the voxel grid becomes larger and the resolution decreases. The high-level structure highlights the overall layout of the logistics center, the overall situation of the distribution of functional areas, and the formulation of macro strategies. This structure embodies the multi-scale characteristics from micro-detailed depiction to macro-pattern control.

[0050] In a specific example, construct Layer source point cloud pyramid, is the total number of layers, Indicates the sequence number of each layer, Layer means the bottom layer. Layer means the highest layer. In each layer, the size of each voxel grid .

[0051] When constructing the point cloud pyramid from the bottom up, as the number of layers increases, the data becomes more abstract, the voxel grid becomes larger, and the resolution decreases. This is the "downward" trend of the data level. The bottom layer can accurately present the details of the parts on the shelves of the logistics center, while the upper layer focuses on the overall layout of the region. Therefore, the first Layer The next layer, Layer The layer above the layer, Layer The two layers above the layer, and so on.

[0052] Specifically, after the point cloud pyramid is constructed, the fusion features of each voxel grid in each layer are determined according to the spatial distance between each voxel grid in each layer and the voxel grid in the adjacent layer, as well as the difference in initial features: For Layer Voxel Grid , whose initial characteristics are ,but The fusion features satisfy the following relationship: ; In the formula, It is Layer Voxel Grid The fusion characteristics of It is Layer Voxel Grid The initial characteristics of For the Layer to layer The influence weight of the layer, For the Layer to layer The influence weight of the layer, It is Layer A voxel grid, It is Layer A voxel grid, yes The initial characteristics of yes The initial characteristics of for and The similarity weight of yes and The similarity weight of and The calculation method is consistent.

[0053] The core of this formula is to organically combine the initial features of each voxel grid with the feature information of the adjacent layers. In the modeling of a three-dimensional logistics center, the multi-layer structure of the point cloud pyramid reflects the spatial information of the point cloud at different scales. Through this fusion method, it is possible to transcend the limitations of a single layer and comprehensively consider the features at the macro and micro levels. For example, when describing a shelf, the first The first layer may present the overall frame structure of the shelf, while the second Layer and The layers provide information about the details of the shelf (such as crossbars and partitions) and the relative position relationship with other surrounding objects, so that the fused features can more completely represent the actual status of the shelf in the entire logistics scene. This fusion is not just a simple addition of data, but through carefully designed weights and similarity metrics, it achieves the complementarity of features at different levels, making the features of the voxel grid more comprehensive and representative, and better able to capture the object features and spatial layout in the complex scenes of the logistics center.

[0054] In this formula, the method for obtaining the influence weight is:

[0055] in, , , They are Layer, Layer, The total number of voxel grids contained in each layer. The more voxel grids a layer has, the richer its description of the overall space and the more detailed it is, and the more critical it is for building a complete and accurate model. Here, the ratio of the number of voxel grids is used as the influence weight, which is similar to assigning corresponding influence levels according to the richness of information in each layer. Layers with more voxel grids should play a dominant role in the process of feature fusion because they contain rich key details, thereby preventing important details from being lost due to improper weighting or simple averaging, and effectively ensuring that the constructed model can accurately reproduce the high complexity of various objects and spatial layouts in logistics scenarios.

[0056] In this formula, the similarity weight , It is used to control the relative importance of the two adjacent layers when fusing the features of the voxel grid of the current layer, and is determined when calculating the fusion features. Each voxel grid feature in the layer has an impact on the current layer ( The contribution ratio of the fused features of the voxel grid of the layer, The calculation of is based on the spatial distance of the voxel grid and the degree of difference of the initial features, ensuring that the initial features with high similarity play a greater role in the fusion process.

[0057] Specifically, The calculation formula is:

[0058] In this formula, Voxel grid With voxel grid The spatial distance for The initial characteristics and The initial characteristics The specific value is and The Euclidean distance of is a natural exponential function. The denominator is the sum of the similarities between all voxel grids in the lower layer and the voxel grids in the current layer. Its core function is to normalize so that the sum of the contribution weights of all voxel grids in the lower layer to the fusion features of the voxel grids in the current layer is 1, ensuring that the contribution ratio of each voxel grid in the fusion feature calculation is reasonably distributed. The exponential function form can highlight the positive contribution of voxel grids with close distances and small feature differences to the weight, that is, the higher the similarity, the greater the weight.

[0059] Specifically, The calculation formula is:

[0060] In this formula, Voxel grid With voxel grid The spatial distance for The initial characteristics and The initial characteristics The specific value is and The Euclidean distance of is a natural exponential function.

[0061] Spatial distance is used to quantify the position difference of different layers of voxel grids in three-dimensional space. It is one of the important factors in calculating feature similarity weights. Spatial distance reflects macro layout associations. For example, shelf spacing affects cargo handling path planning. The closer the distance, the closer the voxel grids are in space, and the higher their feature similarity weights are, and the greater the impact on the results during fusion. Difference measurement between feature descriptors. Feature descriptor differences focus on micro-geometric characteristics. Differences such as cargo shape and shelf structure determine matching accuracy. By comparing the differences in feature descriptors of different voxel grids, their similarity in geometric features can be more accurately measured.

[0062] In particular, when At this time, the top layer of the point cloud pyramid is The calculation formula of the fusion feature of the voxel grid is:

[0063] The characters in this formula have the same meanings as before and are not repeated here.

[0064] Furthermore, after obtaining the fusion features of each voxel grid of each layer, normalization operations are performed on the dimensional values ​​of the fusion features of all voxel grids of all layers (such as using the maximum and minimum value normalization method) to obtain the normalized fusion features of each voxel grid of each layer. , Depend on The normalized values ​​of the eigenvalues ​​of these five dimensions are composed of numerical values.

[0065] S4: Based on the fusion features and inter-layer distances, the feature similarity of each voxel grid and the voxel grids of each layer above it is calculated, and the mapping relationship is constructed according to the principle of maximizing feature similarity.

[0066] Since the point cloud pyramid is divided into L layers from bottom to top, the bottom layer is the most detailed point cloud. As the number of layers increases, the point cloud becomes more abstract with each layer. Layer ratio The layer is more abstract, The layer is A generalization and simplification of the point cloud data of the layer.

[0067] In from Layer to During the process of multiple layers The voxel grids of the first layer are merged and fused according to certain rules to obtain the layer, which means that the A voxel grid of a layer usually contains Multiple voxel grids of layers can thus be constructed. The voxel grid of the layer and the The mapping relationship between the voxel grids of the layers.

[0068] Specifically, first obtain the Layer and The centroid of each voxel grid of the layer, such as Layer The coordinates of all point clouds within the voxel grid are averaged to obtain the The centroid of the individual voxel grid, based on which the first Each voxel grid of the layer and the The inter-layer distance between each voxel grid of the layer, i.e. the Euclidean distance between the centroids.

[0069] Next, considering that the point cloud pyramid is from bottom to top, each layer is an abstraction and generalization of the data of the next layer, that is, the upper layer is a more macroscopic representation obtained after a certain processing of the lower layer. In this hierarchical structure, the information transmission and feature association from the lower layer to the upper layer are more in line with the logical order of data processing. Therefore, choosing to calculate the feature similarity between a certain voxel grid of a certain layer and the voxel grids of the layers above this layer can reflect the changes and similarities of the features of the voxel grids as the data is continuously abstracted, which helps to establish the connection between different abstract levels. In addition, the subsequent analysis and operation are often based on the generalization and guidance of the upper layer to the lower layer. By calculating the feature similarity of each voxel grid with the voxel grids of the layers above, the feature association and mapping from macro to micro can be better realized, which is convenient for overall to local analysis and processing.

[0070] Therefore, this step calculates the feature similarity of each voxel grid with the voxel grids of each layer above it based on the fusion features of each voxel grid and the inter-layer distance:

[0071] In the formula, It is Layer The voxel grid and Layer The feature similarity between individual voxel grids, For the The ordinal number of the voxel grid of the layer, For the The ordinal number of the voxel grid of the layer, It is Layer The voxel grid and Layer The inter-layer distance of the voxel grid, i.e. Layer The centroid of the voxel grid and the Layer The Euclidean distance between the centroids of the voxel grid, It is Layer Fusion features of voxel grids and Layer Fusion features of voxel grids The cosine similarity between .

[0072] This formula is calculated by Layer The voxel grid and Layer The feature similarity between voxel grids cleverly combines spatial distance and feature information. In the modeling of three-dimensional logistics centers, only considering the spatial distance or only considering the cosine similarity of features is not enough to accurately describe the similarity between voxel grids. For example, in a complex logistics scenario, two voxel grids may be close in space, but their features (such as the type of goods they represent, shelf structure, etc.) may be very different; conversely, voxel grids with similar features may be far apart in space. By combining the two, the relationship between them can be more comprehensively evaluated.

[0073] In this formula, The part shows that the closer the spatial distance of the voxel grids is, the greater the contribution of this part, which means that the voxel grids with closer spatial distance will have a greater weight in the similarity calculation. The part indicates that the voxel grids with more similar fusion features will have a greater weight in the similarity calculation. Multiplying these two parts makes the voxel grids with closer spatial distance and more similar fusion features have greater feature similarity.

[0074] In summary, the quantitative feature similarity calculation in this step provides a comparable and measurable standard for subsequent operations in practical applications. The feature similarity between each voxel grid and the voxel grids in the layers above it is an important link to establish connections between voxel grids at different levels in the multi-layer point cloud pyramid structure. When constructing a digital twin model of a three-dimensional logistics center, it is necessary to associate point cloud information at different levels. By calculating the feature similarity between layers, the voxel grids of adjacent layers can be associated according to the similarity, so that the corresponding voxel grids can be found in point cloud data at different levels to realize information transmission and integration.

[0075] Finally, the mapping relationship is constructed according to the principle of maximizing feature similarity. Specifically: For Layer For a voxel grid, Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , then, As a new starting point, calculate With The feature similarity of each voxel grid in the layer Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , continue with As a new starting point, continue to iterate until the highest level, and finally Layer, Layers, until all are found in layer 1 The corresponding voxel grid is constructed The mapping relationship between the voxel grids of the layers above it is obtained In the The corresponding voxel grid in the layers above the layer.

[0076] By constructing a mapping relationship based on the principle of maximizing feature similarity, a clear logical connection bridge can be built between voxel grids at different levels, so that the voxel grid of each layer can find its corresponding voxel grid in the more abstract upper layer. This correspondence reflects the evolution of data from micro to macro, which helps to better understand the intrinsic correlation of the point cloud of the entire three-dimensional logistics center at different scales.

[0077] Determining the corresponding voxel grid by maximizing feature similarity is a reasonable way that meets data characteristics and actual needs. Because similar features often mean that the voxel grids have a high correlation in terms of the objects, spatial regions, etc. represented, when looking for correspondence from one layer to the layers above it, selecting the voxel grid with the greatest feature similarity as the corresponding voxel grid can ensure to the greatest extent that the constructed mapping relationship is based on the intrinsic similarity of the data, rather than random or unfounded associations, so that the mapping relationship can better reflect the real space and object feature correspondence.

[0078] For example, when judging the correspondence between voxel grids representing shelf areas in different layers, based on the principle of maximizing feature similarity, we can accurately associate voxel grids at different abstract levels but related to the same shelf, and accurately grasp the manifestation of the shelf at different levels of the entire logistics center.

[0079] S5: updating the fusion features of the voxel grids corresponding to the voxel grids in the layers above each voxel grid, and calculating the inter-layer propagation factor of each voxel grid.

[0080] In a complex and ever-changing scenario like a logistics center, the fusion features of the voxel grids are updated based on feature similarity, so that the voxel grids of each layer can establish a close and reasonable connection with the corresponding voxel grids of the adjacent upper layer, ensuring that the information transmission and fusion process is carried out according to the actual degree of feature correlation between the voxel grids, so that the fusion features of each layer of the multi-layer point cloud pyramid form a coherent and organic whole.

[0081] Updating fused features from bottom to top helps maintain the consistency of feature descriptions of logistics centers at different abstract levels. For example, when a certain area of ​​a logistics center undergoes cargo adjustments or layout changes, this update method can rely on feature similarity to pass the change information first perceived by the bottom layer upward and update the fused features of the corresponding voxel grids of each layer in sequence, so that the entire multi-layer point cloud pyramid can reflect this change in a timely and coordinated manner from bottom to top, ensuring the consistency between features at each layer. No matter at which abstract level the logistics center is observed, its feature description can remain logically consistent and consistent with the actual situation, providing a stable and coherent information basis for subsequent analysis, modeling, and decision-making based on these features.

[0082] Specifically, the method for updating the fusion features of the voxel grids corresponding to each voxel grid in each layer above it is: The first Layer Voxel Grid On top of it The corresponding voxel grid in the layer is denoted as , according to the following formula To update:

[0083] In the formula, For the Layer Voxel Grid In the The corresponding voxel grid in the layer, for The fusion characteristics of for Updated fusion features, For the Layer Voxel Grid On top of it The corresponding voxel grid in the layer and The corresponding voxel grid in the layer The similarity of features between and They are In the The sequence number in the layer and In the The sequence number within the layer.

[0084] This formula constructs a reasonable cross-layer information transmission mechanism. According to the feature similarity between the corresponding voxel grids of each layer, the relatively lower-level and more detailed features are transmitted hierarchically to the upper layer, so that the fusion features of the upper voxel grids are no longer limited to the original abstract information of the layer itself, but incorporate the richer details of the lower layer, realizing the upward integration of multi-scale information in the pyramid structure, so that the fusion features of each layer can integrate information at different levels and more comprehensively characterize the real status of objects and space in the logistics center.

[0085] Next, the inter-layer propagation factor of each voxel grid is calculated:

[0086] In the formula, For the Layer Voxel Grid The inter-layer propagation factor, for The corresponding voxel grid in the top level, for Fusion features and Updated fusion features The degree of difference, For the Layer Voxel Grid On top of it The layer above The fused features of the corresponding voxel grid in the layer and The corresponding voxel grid in the highest layer is updated with the fused features The degree of difference is calculated based on the Euclidean distance.

[0087] This formula is based on the Layer The voxel grid of each voxel is relative to its corresponding voxel grid at the top level. The relative influence of the fusion feature changes in the entire multi-layer structure, that is, Layer The influence of the fusion features of the voxel grid on the inter-layer propagation process. Layer The larger the inter-layer propagation factor of a voxel grid, the more significant the change in the fusion features of the voxel grid when it propagates from its own layer to the highest layer; conversely, a smaller value indicates that its influence in inter-layer propagation is relatively weaker.

[0088] For smaller inter-layer propagation factors: it usually means that the objects or areas represented by the corresponding voxel grids have relatively common and general characteristics in the logistics center, and their fusion characteristics between different layers do not change much. In the process of propagating from the layer where they are located to the highest layer, the difference in the fusion characteristics of the corresponding voxel grids in the highest layer accounts for a small proportion. The presented features are relatively "ordinary" but indispensable parts of the entire logistics scene, such as the voxel grids corresponding to the repetitive frame parts in the conventional shelf structure, whose features are highly similar between layers. For example, the voxel grid features of the same type of standard goods stacked in large quantities in the logistics center are relatively consistent in overall performance, whether in the more refined bottom layer or in the gradually abstract upper layer, and there will be no obvious feature changes between layers, so their inter-layer propagation factors are relatively small. This type of voxel grid has a weak influence on the features of other layers in the information transmission and fusion process of the entire multi-layer point cloud pyramid structure. When constructing the logistics center model, although they are also part of the overall data, due to their relatively stable characteristics and lack of prominent changes, they have little impact on the integration of features between different layers and the model's restoration effect on complex scenarios. They are more of a conventional and auxiliary element in the model. When considering the information at each level, their weight is relatively low.

[0089] For larger inter-layer propagation factors: the objects or areas generally represented have unique layouts, important functions, or are in key positions in the logistics center scene. For example, the exclusive area for storing special goods in the logistics center, the corresponding voxel grid carries the unique shape, special storage location and other feature information of these special goods. When propagating from the layer to other layers, there will be obvious differences compared with the fusion features of the corresponding voxel grid in the highest layer, resulting in a larger propagation factor. In the process of information transmission and fusion, it has a strong influence on the feature shaping of each layer of the entire multi-layer point cloud pyramid. When constructing the logistics center model, the feature information carried by these voxel grids is like a key clue, which can guide different layers to more accurately depict the related areas or objects, and provide important and distinguishing information for the feature fusion between layers. The lack of them may cause the model to have missing or deviations in the feature description of key parts, affecting the accurate restoration of the entire logistics center scene, so their influence needs to be considered in model construction and feature integration. For example, the voxel grid corresponding to the area where the newly introduced automated sorting equipment is located in the logistics center has obvious differences in function and structure from the surrounding traditional shelves, channels, etc., and the features change significantly when propagating between layers.

[0090] S6: Screen significant feature points based on the size of the inter-layer propagation factor, and perform iterative closest point algorithm to initialize and align the 3D point cloud model.

[0091] In order to accurately register the point cloud, a relatively high inter-layer propagation factor threshold of 0.6 (empirical value) is set here, and the significant feature points are strictly screened to ensure that the initial matching point pairs are all point clouds with strong influence and very prominent features. In each layer, the voxel grid with an inter-layer propagation factor greater than the inter-layer propagation factor threshold is taken as the significant voxel grid, and the center of the significant voxel grid is taken as the significant feature point of the layer, and finally all the significant feature points in each layer are obtained.

[0092] Next, the initial matching point pair set of the source point cloud and the target point cloud is determined based on all the significant feature points: In the context of 3D modeling, the source point cloud pyramid and the target point cloud pyramid are multi-level structures constructed based on the point cloud of the logistics center. The source point cloud pyramid is processed by the original point cloud collected from the logistics center. The target point cloud pyramid is constructed in a similar principle. It is generated by processing the point cloud collected from different periods or different perspectives of the same logistics center. It is used to monitor changes in facility layout, cargo displacement, etc., and can provide a reference for model matching and similarity analysis.

[0093] Therefore, in the present invention, the same processing method is adopted for the target point cloud pyramid and the source point cloud pyramid to obtain the significant feature points of each layer, and the symmetrical processing ensures consistency, ensuring that the feature extraction standards of the two are unified and the feature spaces are comparable. In the logistics scene, the source point cloud and the target point cloud are affected by similar environmental factors. This method fits the data characteristics, makes the feature extraction scale consistent, and enhances the reliability and stability of point cloud registration.

[0094] Next, the method for determining the initial matching point pair set of the source point cloud and the target point cloud based on the salient feature points of each layer is as follows: First, set the source point cloud pyramid A1 and the target point cloud pyramid A2; For the salient feature point A in the highest layer of A1, the following operations are performed in the highest layer of the target point cloud A2: the Euclidean distance between each salient feature point in the highest layer of A2 and A is calculated, and the cosine similarity between each salient feature point and the fusion feature of A is obtained. Then, the salient feature point B in the highest layer of A2 with the smallest Euclidean distance and the largest cosine similarity to A is selected as the matching point of A. A and B form a matching point pair, and then a set of matching point pairs of the highest layers of A1 and A2 is obtained. For the layer below the highest layer of A1, mark the point pairs that have been matched in the previous layer. For the unmatched point pairs, use the same method to determine the matching point pairs at the highest layer, and so on, match them layer by layer. Finally, use the initial matching point pair set at the bottom layer as the initial matching point pair set of the source point cloud and the target point cloud.

[0095] It should be noted that the reason for using the bottom-level initial matching point pair set as the initial matching point pair set of the source point cloud and the target point cloud is: From the perspective of resolution: the bottom layer has the highest resolution, and the voxel grid size is relatively large, which can accurately capture the details of the original point cloud. In the logistics center scene, this means that the microscopic features such as the subtle structure of the shelf and the precise outline of the goods are fully preserved. The numerous detail feature points increase the number of potential matching points, allowing the bottom layer to form a rich set of matching point pairs.

[0096] From the perspective of feature integrity: the bottom layer retains the most complete feature information of the original point cloud. This integrity makes the bottom layer feature points more unique and recognizable when searching for matching points, making it easier to find corresponding matching points in the target point cloud. For example, the complex equipment structure of a logistics center or the irregular shape of goods stacked can be presented in its entirety in the bottom layer, while the upper layer may only retain the general outline.

[0097] In summary, this step uses multi-level fine screening, from the macro-anchoring at the top of the pyramid to the micro-verification at the bottom. Each layer collaboratively explores the deep features of logistics facilities and goods to form a coarse and fine combination and hierarchical progressive matching architecture. The final set of initial matching point pairs can fully capture the feature associations between the target point cloud and the source point cloud, accurately capture the feature associations to construct high-quality initial matching, and provide a precise start for the initialization of the iterative closest point algorithm, reducing the dependence on the initial transformation, reducing the risk of falling into local optimality, and improving the registration accuracy and efficiency.

[0098] After obtaining an accurate set of initial matching points, an iterative closest point algorithm is used to perform initial transformation estimation based on the accurate set of initial matching points, including: The weighted centroid and covariance matrix are calculated and decomposed from the initial matching point set of the source point cloud and the target point cloud to obtain the initial rotation and translation matrix, completing the initialization process of the iterative closest point algorithm. The iterative closest point algorithm calculates the rigid body transformation that makes the source point cloud and the target point cloud match the highest degree according to the initial matching point set found. This transformation can be solved by minimizing the error function. The most commonly used error function is the sum of the squared distances between point pairs. By solving this minimized error function, the optimal rotation and translation transformation parameters can be obtained. Based on the optimal rotation and translation transformation parameters, the best registration effect is obtained, and the point cloud registration is completed to obtain a three-dimensional point cloud model.

[0099] S7: Align and fuse the spatial position of dynamic target objects in video data with the point cloud model.

[0100] Based on the coordinate system of the registered 3D point cloud model, the 2D image information of the dynamic target objects in the video data is mapped to the 3D point cloud model using 3D reconstruction technology and spatial transformation algorithm, realizing the spatial position registration and fusion of the dynamic target objects and the point cloud model. The point cloud is a static spatial structure description of the logistics center, while the dynamic target objects in the video data contain real-time operational activity information such as vehicle driving, personnel operations, and cargo handling. Through registration and fusion, static and dynamic elements can be organically combined in one model to construct a complete digital twin model that fully reflects the actual status of the logistics center. Whether it is a fixed spatial structure or a constantly changing operational process, it can be presented in this model.

[0101] S8: Perform virtual data integration and business operation process model integration on the 3D point cloud model.

[0102] Integrating virtual data with the 3D point cloud model enables the model to reflect the dynamic data of the actual operation status of the logistics center in real time. Integrating the business operation process model with the 3D model can simulate logistics operations in different scenarios, which helps to discover potential problems and optimization space in the process.

[0103] S81: Virtual data integration.

[0104] The dynamic data of all dynamic target objects involved in the video data are collected through various sensors, and these data are accurately bound and associated with the corresponding areas of the dynamic target objects in the three-dimensional point cloud, so that the three-dimensional point cloud model can display the dynamic changes of each dynamic target object in real time. Specifically, the following steps are included: S811: Deploy sensors.

[0105] Select appropriate sensors based on the characteristics of different dynamic target objects and the types of dynamic information that need to be monitored. For example, in a logistics center, goods are in constant flow and state change, and are a type of dynamic target object. Radio frequency identification (RFID) tags and weight sensors can be deployed to monitor the location and weight changes of goods; the global positioning system (GPS) and on-board odometers can be used to monitor the running trajectory and speed of transport vehicles; and indoor positioning systems (such as ultra-wideband positioning technology UWB) and wearable devices (such as smart bracelets with motion sensors) can be used to monitor the movement trajectory and operation status of personnel. These sensors are reasonably distributed at various key locations in the logistics center to ensure that the dynamic data of all dynamic target objects in the actual business operation process can be fully and accurately obtained.

[0106] S812: Data collection and transmission.

[0107] Various sensors collect dynamic data of corresponding entities in real time according to preset frequency and accuracy requirements. For example, RFID tags will continuously send identification codes and location information of goods, and weight sensors will feedback weight change values ​​of goods at regular intervals. These collected data are transmitted to a unified data processing platform through wired (such as Ethernet) or wireless (such as Wi-Fi, Bluetooth, Zigbee, etc.) communication methods to ensure the timeliness and integrity of the data.

[0108] S813: Data is bound and associated with the 3D point cloud.

[0109] Through the pre-set spatial coordinate mapping algorithm and recognition rules, the dynamic data of each dynamic target object received is bound and associated with their corresponding areas in the 3D point cloud model. For example, if the actual physical coordinates of a certain cargo in the logistics center are known, the coordinate conversion formula is used to correspond it to a specific voxel grid area in the 3D point cloud model, and the dynamic data such as the weight and status of the cargo are also associated with this area. In this way, when the weight of the cargo changes or the position moves, the corresponding area in the 3D point cloud model can be updated in real time to display these dynamic changes, truly realizing the integration of virtual process data and providing strong data support for subsequent real-time monitoring, analysis and decision-making.

[0110] S82: Business operation process model integration.

[0111] First, we need to establish the business operation process model of the logistics center, including the following steps: S821: Business process decomposition.

[0112] Analyze the daily operations of the logistics center and decompose the entire logistics center business process into multiple clear and specific operation steps and events according to function and sequence. For example, for the goods warehousing process, it can be subdivided into arrival acceptance, unloading, counting quantity, quality inspection, warehousing registration, shelf storage and other operation steps; for the goods outbound process, it can be divided into order processing, picking, packaging, review, loading and shipping and other steps. At the same time, identify the various entities involved in each operation step and event, including equipment (such as forklifts, conveyors, shelves, barcode scanners, etc.), personnel (warehouse managers, porters, quality inspectors, etc.), and goods, and sort out the interaction between them, such as the loading and unloading collaboration between forklifts and shelves, and personnel operating barcode scanners to enter information about goods, etc., to lay the foundation for subsequent model construction.

[0113] S822: Logical model creation.

[0114] For each decomposed operation step and event, a corresponding logical model is created to clearly present the execution sequence structure, branch path and loop mode of each operation step and event. Taking the cargo sorting process as an example, its execution sequence structure includes: first receiving the sorting task, then conveying the cargo to the corresponding sorting area through the conveyor according to the destination, category and other information of the cargo, and then the sorting personnel placing the cargo in the corresponding shipping channel; and one branch path is: when the size of the cargo exceeds a certain standard, it needs to be manually transported to the large-piece sorting area for separate processing; the loop mode may appear in the same batch of cargo There are multiple identical sorting requirements, and a certain sorting operation will be repeated many times. Through the creation of such a logical model, it is possible to accurately grasp the various complex logical relationships in the business operation process, making the model closer to the actual logistics operation situation.

[0115] S823: Entity attribute definition.

[0116] Define in detail the categories and attributes of various entities involved in each operation step and event, and fully describe the attributes and state changes of the entities. For the cargo entity, its category can be divided according to product type, size, weight, etc., and the attributes include cargo number, name, inventory quantity, storage location, current status (in storage, on the shelf, waiting to be shipped out, etc.); for equipment entities, such as forklifts, the attributes include equipment number, model, location, working status (idle, in operation, faulty, etc.); the attributes of personnel entities include employee number, name, position, current task, etc. Clarifying the attributes and state changes of these entities can provide detailed parameter basis for simulating the real state changes in the operation process of the logistics center, ensuring that the model can accurately reflect the actual situation of various entities during operation.

[0117] S824: Interaction rules and event-driven mechanism settings.

[0118] According to the logical model of each operation step and event, the interaction rules and event-driven mechanism between various entities are strictly set to simulate the real state changes in the operation process of the logistics center. For example, when the goods arrive at the designated sorting area (this is an event), the sorting equipment is triggered to start and sort according to the preset sorting rules (this is the event-based interaction rules and driving mechanism); for another example, when the warehouse manager scans the barcode of the goods to complete the warehouse registration (event), the system automatically updates the inventory database and sends the forklift driver a shelf instruction (interaction rules and driving mechanism). Through such settings, different entities can interact in different operation scenarios according to the rules of actual logistics operations, so that the entire business operation process model can dynamically and realistically simulate the operation status of the logistics center, and then combine with the three-dimensional point cloud model to realize the digital twin presentation and management of the logistics center in all directions and throughout the process.

[0119] Then, various entities involved in the business operation process model of the logistics center are associated and bound to the corresponding areas in the three-dimensional model, so that the business operation process can be visualized in the three-dimensional model.

[0120] Finally, based on the pre-built detailed business operation process model, various actual business scenarios can be simulated in the three-dimensional model, including standard processes and sudden abnormal situations (such as equipment failure, cargo loss, emergency order insertion, etc.). Employees can conduct practical exercises in these simulated scenarios, operate according to standard operating procedures and response strategies, learn how to make correct decisions and responses in different situations, and improve their ability to deal with complex problems and practical operation skills.

[0121] To sum up, through this step, a three-dimensional model of the logistics center based on digital twins that is closely integrated with reality can be obtained, providing a solid basic support for the efficient operation, optimized management and intelligent decision-making of the logistics center.

[0122] The present invention also provides a three-dimensional logistics center modeling system based on digital twins, the modeling system comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any one of the modeling methods.

[0123] In the description of this specification, "multiple" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0124] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A three-dimensional logistics center modeling method based on digital twins, characterized in that: include: Collect point cloud and video data of the logistics center. The point cloud is voxelized to form multiple voxel grids. The initial features of each voxel grid are formed by the number of point clouds, centroid, and eigenvalues ​​of covariance matrix decomposition of each voxel grid, and dynamic target objects in the video data are identified; Construct a multi-layer point cloud pyramid, determine the fusion features of each voxel grid based on the inter-layer distance and initial feature difference between each voxel grid and each voxel grid in the adjacent layer, and calculate the feature similarity of each voxel grid with the voxel grids in the layers above it based on the fusion features and inter-layer distance. According to the principle of maximizing feature similarity, a mapping relationship is constructed to determine the corresponding voxel grids in each layer above each voxel grid; Update the fusion features of the voxel grids corresponding to each voxel grid in each layer above it, calculate the inter-layer propagation factor of each voxel grid according to the updated fusion features, take the center of the voxel grid whose inter-layer propagation factor is greater than the threshold as a significant feature point, determine the initial matching point pair set of the source point cloud and the target point cloud based on all the significant feature points, and use the iterative closest point algorithm to initialize and register the three-dimensional point cloud model based on the initial matching point pair set; By using 3D reconstruction technology and spatial transformation algorithm, the 2D image information of the dynamic target object is mapped to the 3D point cloud model, and the registration and fusion of the dynamic target object and the point cloud model are achieved, thus completing the 3D modeling.

2. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The fusion characteristics of each voxel grid satisfy the following relationship: ； In the formula, , They are Layer Voxel Grid The fusion features and initial features of 、 Respectively Layer, Layer to layer The influence weight of the layer, , , It is Layer A voxel grid, It is Layer A voxel grid, , They are , The initial characteristics of for and The similarity weight of yes and The similarity weight of and The calculation method is consistent with , , They are Layer, Layer, The total number of voxel grids each layer contains.

3. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The calculation formula for the feature similarity between each voxel grid and the voxel grids of each layer above it is: ； In the formula, It is Layer The voxel grid and Layer The feature similarity between individual voxel grids, It is Layer The voxel grid and Layer The inter-layer distance of the voxel grid is Layer The centroid of the voxel grid and the Layer The Euclidean distance between the centroids of the voxel grid, It is Layer Fusion features of voxel grids and Layer Fusion features of voxel grids The cosine similarity between .

4. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The method for determining the voxel grid corresponding to each voxel grid in the layers above it is: For Layer Voxel Grid , will Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , then, As a new starting point, calculate With The feature similarity of each voxel grid in the layer Layer The voxel grid with the largest feature similarity is In the The corresponding voxel grid in the layer , continue with As a new starting point, continue to iterate until the highest level, and finally Found in all levels until 1 The corresponding voxel grid is constructed The mapping relationship between the voxel grids of the layers above it is obtained In the The corresponding voxel grid in the layers above the layer.

5. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The method for updating the fusion features of the corresponding voxel grids in the layers above each voxel grid is: ； In the formula, For the Layer Voxel Grid exist The corresponding voxel grid in the layer, for The fusion characteristics of for Updated fusion features, For the Layer Voxel Grid On top of it The corresponding voxel grid in the layer and The corresponding voxel grid in the layer The similarity of features between and They are In the The sequence number in the layer and In the The sequence number within the layer.

6. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The calculation formula of the inter-layer propagation factor of each voxel grid is: ； In the formula, For the Layer Voxel Grid The inter-layer propagation factor, for The corresponding voxel grid in the top level, for Fusion features and Updated fusion features The degree of difference, For the Layer Voxel Grid In the The fused features of the corresponding voxel grid in the layer and The corresponding voxel grid in the highest layer is updated with the fused features The degree of difference is calculated based on the Euclidean distance.

7. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: The method for determining the initial matching point pair set of the source point cloud and the target point cloud based on the significant feature points of each layer is: Set the source point cloud pyramid A1 and the target point cloud pyramid A2. For the salient feature point A in the highest layer of A1, perform the following operations in the highest layer of the target point cloud A2: calculate the Euclidean distance between each salient feature point in the highest layer of A2 and A, and obtain the cosine similarity between each salient feature point and the fusion feature of A. Then select the salient feature point B in the highest layer of A2 with the smallest Euclidean distance and the largest cosine similarity as the matching point of A. A and B form a matching point pair, and then obtain the matching point pair set of the highest layer of A1 and the highest layer of A2. For the layer below the highest layer of A1, mark the point pairs that have been matched in the previous layer. For the unmatched point pairs, use the same method to determine the matching point pairs at the highest layer, and so on, match layer by layer, and use the initial matching point pair set at the bottom layer as the initial matching point pair set of the source point cloud and the target point cloud.

8. The three-dimensional logistics center modeling method based on digital twin according to claim 1 is characterized in that: After completing the 3D modeling, it also includes: establishing the business operation process model of the logistics center: Decompose the business process of the logistics center into multiple operation steps and events, and identify the various entities involved and their interactions, including equipment, personnel, and goods; Create a logical model of each operation step and event to obtain the execution sequence structure, branch path and loop mode of each operation step and event; Define the categories and attributes of various entities involved in each operation step and event to describe the attributes and state changes of various entities; The interaction rules and event-driven mechanisms among various entities are set according to the logical models of various operation steps and events to simulate the state changes of various entities in the operation process of the logistics center.

9. The three-dimensional logistics center modeling method based on digital twin according to claim 8 is characterized in that: After completing the 3D modeling, it also includes: Virtual data integration: collect dynamic data of dynamic target objects through various sensors, and bind and associate the dynamic data of dynamic target objects with the corresponding areas in the 3D model; Business operation process model integration: associate and bind various entities involved in the business operation process model of the logistics center with the corresponding areas in the 3D model to simulate the business operation process in the 3D model.

10. The three-dimensional logistics center modeling system based on digital twin is characterized by: The modeling system comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the steps of the modeling method according to any one of claims 1 to 9.

Images