3D storage yard measurement method, system and equipment based on unmanned aerial vehicle and medium

Abstract

The invention discloses a 3D storage yard measurement method, system and equipment based on an unmanned aerial vehicle, and a medium, and relates to the field of storage yard measurement. In the method, an unmanned aerial vehicle is controlled to fly according to a preset flight path to collect measurement images and point cloud data of each target cargo pile in a port, the measurement images and the point cloud data are registered, and the registered measurement images and the point cloud data are fused to generate a fused data set; inputting the data set into a preset deep convolutional neural network, generating a preliminary three-dimensional model of the target cargo pile, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; obtaining cargo characteristic data and environmental factor data of the target cargo pile, and constructing a density model of the target cargo pile according to the cargo characteristic data and the environmental factor data; and calculating quality data of the target cargo pile according to the target three-dimensional model and the density model. By implementing the technical scheme provided by the invention, the cargo measurement of the port storage yard can be completed more accurately and efficiently.

Description

Technical Field

[0001] This application relates to the technical field of yard measurement, and particularly relates to a 3D yard measurement method, system, device, and medium based on unmanned aerial vehicles (UAVs). Background Art

[0002] In the field of port management, the throughput of port goods is increasing day by day, and cargo stack management has become an important link in port operation. Accurate cargo stack measurement is of crucial significance for reasonably arranging storage space, improving cargo allocation efficiency, and ensuring the safe and stable operation of the port. Accurately grasping the actual situation of the cargo stack helps to optimize the utilization of port resources, reduce space waste, and also provides a scientific basis for cargo handling, transportation, and other links, thereby improving the overall operation efficiency and economic benefits of the port.

[0003] Traditional port cargo stack measurement mainly relies on manual on-site measurement. Manual on-site measurement usually involves staff carrying simple measurement tools, such as tape measures and measuring rods, and directly going to the cargo stack site for size measurement. This method requires staff to be in close contact with the cargo stack, the operation is relatively cumbersome, and it is easily affected by factors such as the irregular shape of the cargo stack and the complex on-site environment.

[0004] However, these traditional measurement methods have obvious defects. Manual on-site measurement and total station measurement methods not only consume a large amount of manpower and time, but also have low measurement accuracy, making it difficult to meet the large-scale and high-efficiency operation requirements of modern ports. Especially when facing cargo stacks with complex shapes and wide distributions, the measurement accuracy of traditional methods will be further reduced. In addition, traditional methods cannot fully consider the influence of cargo characteristics and environmental factors on the density and state of the cargo stack, resulting in measurement results that cannot truly reflect the actual situation of the cargo stack, thereby affecting port management decisions and operational benefits. Summary of the Invention

[0005] This application provides a 3D yard measurement method, system, device, and medium based on UAVs, which can complete the measurement of port yard goods more accurately and efficiently by comprehensively considering various factors.

[0006] In the first aspect of this application, a 3D yard measurement method based on UAVs is provided, which is applied to a port management platform. The method includes: Controlling the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo stack in the port, registering the measurement images and the point cloud data, and fusing the registered measurement images and the point cloud data to generate a fused data set; Inputting the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo stack, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; Obtain the cargo characteristic data and environmental factor data of the target cargo stack, and construct a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data. The cargo characteristic data includes cargo source, transportation mode, storage time, and stacking method. The environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; Calculate the mass data of the target cargo stack according to the target three-dimensional model and the density model.

[0007] Optionally, the registering the measurement image and the point cloud data and fusing the registered measurement image and the point cloud data to generate a fused data set includes: Extract the image features of the measurement image. The image features include edges, corners, and textures. Extract the point cloud features of the point cloud data. The point cloud features include curvature, normal vector, and local density; Calculate the Euclidean distance between the image features and the point cloud features, and determine the feature point pairs with the Euclidean distance less than a preset threshold as matching pairs. The feature point pairs include one image feature and one point cloud feature; Calculate the registration transformation matrix between the measurement image and the point cloud data according to the matching pairs, obtain the optimal solution of the registration transformation matrix by the least squares method, and align the measurement image and the point cloud data according to the optimal solution; Fuse the aligned measurement image and the point cloud data.

[0008] Optionally, the inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model includes: Construct a three-dimensional model feature table and encode the feature elements of the three-dimensional model. The feature elements include geometric features and topological structures, and each feature element has a unique index; When receiving the preliminary three-dimensional model, look up the index of the feature elements of the preliminary three-dimensional model in the three-dimensional model feature table, retrieve the corresponding embedding vectors from a preset embedding matrix using the index, and construct an embedding vector sequence according to the embedding vectors; For the embedding vector sequence, calculate the correlation scores between each feature element and other feature elements, construct an attention weight matrix according to the correlation scores, and perform weighted summation on the embedding vector sequence according to the attention weight matrix to obtain the context representation of each feature element; Use graph embedding technology to incorporate the context representation into the generative adversarial network, and generate a target three-dimensional model through the generative adversarial network.

[0009] Optionally, the generating a target three-dimensional model through the generative adversarial network includes: Generate a sample model through the generator of the generative adversarial network, and input both the sample model and the preliminary 3D model into the discriminator of the generative adversarial network; Calculate the loss functions of the generator and the discriminator according to the discrimination results of the discriminator on the sample model and the preliminary 3D model, update the parameters of the generator and the discriminator through the backpropagation algorithm, and perform a new round of iteration through the updated generator and discriminator; When the generative adversarial network reaches the preset number of iterations or performance convergence, the generator generates an intermediate 3D model; According to the curvature magnitude of the surface triangular patches of the intermediate 3D model, remove the redundant triangular patches in the intermediate 3D model, identify the topological structure features in the intermediate 3D model using topology principles to determine the location and type of defects, and adopt a repair algorithm based on topological transformation to repair the detected defects to obtain the target 3D model.

[0010] Optionally, the constructing the density model of the target cargo stack according to the cargo characteristic data and the environmental factor data includes: Obtain the data correlation between the cargo characteristic data and the environmental factor data through the principal component analysis method; Respectively construct the first influence models of each factor in the cargo characteristic data on the cargo density, and respectively construct the second influence models of each factor in the environmental factor data on the cargo density; Integrate the first influence model and the second influence model according to the data correlation to construct a density model.

[0011] Optionally, the integrating the first influence model and the second influence model according to the data correlation to construct a density model includes: According to the results of the principal component analysis, assign weights to each factor in the cargo characteristic data and the environmental factor data; Determine the first contribution values of each factor in the cargo characteristic data to the cargo density according to the first influence model, and determine the second contribution values of each factor in the environmental factor data to the cargo density according to the second influence model; Perform weighted summation on the cargo characteristic data and the environmental factor data according to the weights, the first contribution values, and the second contribution values to obtain a comprehensive density prediction value to construct a density model.

[0012] Optionally, the calculating the mass data of the target cargo stack according to the target 3D model and the density model includes: Divide the target 3D model into multiple cubic units, determine the target volume of each cubic unit, determine the predicted density of each cubic unit according to the density model, and obtain the target mass based on the target volume and the predicted density. Add up the multiple target masses to obtain the total mass of the target cargo stack.

[0013] In a second aspect of the present application, a 3D yard measurement system based on an unmanned aerial vehicle is provided, including an acquisition module, a model module, a density module, and a calculation module, where: The acquisition module is configured to control the unmanned aerial vehicle to fly along a preset flight path to acquire measurement images and point cloud data of each target cargo stack in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and the point cloud data to generate a fused data set. The model module is configured to input the data set into a preset deep convolutional neural network to generate a preliminary 3D model of the target cargo stack, and input the preliminary 3D model into a generative adversarial network to generate a target 3D model. The density module is configured to obtain the cargo characteristic data and environmental factor data of the target cargo stack, and construct a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data. The cargo characteristic data includes the cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity. The calculation module is configured to calculate the mass data of the target cargo stack according to the target 3D model and the density model.

[0014] In a third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, and both the user interface and the network interface are used to communicate with other devices. The processor is used to execute the instructions stored in the memory so that the electronic device executes the method described in any one of the above.

[0015] In a fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are executed, the method described in any one of the above is executed.

[0016] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. By flying the drone along a preset path, full-coverage and high-efficiency data collection can be achieved, avoiding the limitations and subjectivity of manual measurement, while reducing labor costs and safety risks. The images providing texture and color information are registered and fused with the point cloud providing precise geometric information to generate a more complete three-dimensional data set, making up for the deficiencies of a single data source and improving the model accuracy; 2. Use a deep convolutional neural network to extract features from the fused data set and quickly generate a preliminary three-dimensional model, retaining the geometric shape and surface details of the cargo stack. Refine the preliminary model through a generative adversarial network to generate a target three-dimensional model with higher resolution and more authenticity, reducing noise and distortion and improving the model quality; 3. Combine cargo characteristics (such as source, transportation method, storage time) and environmental factors (such as temperature, humidity) to construct a dynamic density model to reflect the density changes of the cargo stack under different conditions (such as grain swelling due to moisture absorption, metal oxidation, etc.). Based on the target three-dimensional model and the density model, calculate the mass of the cargo stack, avoiding the limitations of traditional weighing methods (such as being unable to directly measure large cargo stacks or dangerous goods); 4. The real-time and accurate cargo stack mass data supports the port to dynamically adjust the inventory strategy, reducing the risks of overloading or underloading. Monitor the stability of the cargo stack through the density model (such as the risk of landslide caused by too high humidity), and analyze the structural safety in combination with the three-dimensional model. The accurate mass data helps the port to reasonably allocate resources such as loading and unloading equipment and transport vehicles, improving the operation efficiency. Description of the Drawings

[0017] Figure 1 is a schematic flowchart of the drone-based 3D yard measurement method disclosed in the embodiments of the present application; Figure 2 is a schematic module diagram of the drone-based 3D yard measurement system disclosed in the embodiments of the present application; Figure 3 is a schematic structural diagram of an electronic device disclosed in the embodiments of the present application.

[0018] Description of the reference numerals: 201, acquisition module; 202, model module; 203, density module; 204, calculation module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. Detailed Embodiments

[0019] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments.

[0020] In the description of the embodiments of the present application, words such as "for example" or "for illustration" are used to give examples, illustrations or explanations. Any embodiment or design solution described as "for example" or "for illustration" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "for example" or "for illustration" is intended to present relevant concepts in a specific manner.

[0021] In the description of the embodiments of the present application, the term "plurality" means two or more. For example, a plurality of systems means two or more systems, and a plurality of screen terminals means two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0022] This embodiment discloses a 3D yard measurement method based on an unmanned aerial vehicle. Figure 1 It is a schematic flowchart of a 3D yard measurement method based on an unmanned aerial vehicle disclosed in the embodiments of the present application, and is applied to a port management platform, as Figure 1 shown. The method includes the following steps: S101. Control the unmanned aerial vehicle to fly according to a preset flight path to collect measurement images and point cloud data of each target cargo stack in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and the point cloud data to generate a fused data set; S102. Input the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo stack, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; S103. Obtain the cargo characteristic data and environmental factor data of the target cargo stack, and construct a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data. The cargo characteristic data includes the cargo source, transportation mode, storage time and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure and light intensity; S104. Calculate the mass data of the target cargo stack according to the target three-dimensional model and the density model.

[0023] Control the drone to fly in the port area according to the pre-set flight path. During the flight, the drone uses the on-board sensors (such as cameras, lidar, etc.) to collect the measurement images and point cloud data of each target cargo stack in the port. The measurement images can usually provide visual information such as the appearance and color of the cargo stack, while the point cloud data can accurately describe the three-dimensional geometric shape and spatial position information of the cargo stack surface. Due to differences in acquisition equipment, perspectives, time, etc., the collected measurement images and point cloud data may be inconsistent in spatial position. Therefore, it is necessary to perform a registration operation on these data, that is, to align the measurement images and point cloud data in space so that they can accurately correspond to each other for subsequent fusion processing. Fuse the registered measurement images and point cloud data. The fused dataset combines the advantages of image and point cloud data, including both rich visual information and accurate three-dimensional geometric information, and can more comprehensively and accurately describe the characteristics of the target cargo stack. Input the fused dataset into a pre-set deep convolutional neural network. Through learning a large amount of data, the deep convolutional neural network can extract the feature information of the cargo stack from the input dataset and generate a preliminary three-dimensional model of the target cargo stack based on this feature information. This preliminary three-dimensional model is a preliminary estimate of the shape and structure of the target cargo stack. Input the preliminary three-dimensional model into a generative adversarial network (GAN). The generative adversarial network consists of a generator and a discriminator. The generator attempts to generate realistic data, while the discriminator attempts to distinguish between the generated data and real data. Through the adversarial training between the generator and the discriminator, the generator can continuously optimize the generated three-dimensional model to make it more realistic and accurate, and finally generate the target three-dimensional model. Obtain the cargo characteristic data and environmental factor data of the target cargo stack. The cargo characteristic data includes the cargo source, transportation method, storage time, and stacking method. Goods from different sources may vary in material, density, etc. Bumps, squeezes, etc. during transportation may affect the stacking state and density distribution of the goods. The longer the storage time, the more likely the goods are to change in density due to moisture, deterioration, etc. Different stacking methods will affect the void ratio and density distribution inside the cargo stack. The environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity, etc. These environmental factors may affect the physical properties of the goods and thus affect the density of the cargo stack. Based on the obtained cargo characteristic data and environmental factor data, construct a density model of the target cargo stack. This model can be a mathematical formula, algorithm, or a machine learning-based model, used to describe the relationship between the cargo characteristic data and environmental factor data and the density of the cargo stack. Through this density model, the density distribution of the target cargo stack can be calculated according to the given cargo characteristics and environmental factor data. According to the generated target three-dimensional model and the constructed density model, calculate the mass data of the target cargo stack. The target three-dimensional model provides the volume information of the cargo stack, while the density model provides the density information of the cargo stack.By multiplying the volume of the cargo stack by the corresponding density and considering the density differences in different parts within the cargo stack (if the density model is distributed), the mass data of the target cargo stack can be calculated. This mass data can provide important reference bases for cargo management and transportation scheduling in the port.

[0024] Optionally, the registering of the measurement image and the point cloud data and the fusing of the registered measurement image and the point cloud data to generate a fused data set include: Extracting the image features of the measurement image, where the image features include edges, corner points, and textures, and extracting the point cloud features of the point cloud data, where the point cloud features include curvature, normal vectors, and local density; Calculating the Euclidean distance between the image features and the point cloud features, and determining the feature point pairs with the Euclidean distance less than a preset threshold as matching pairs, where the feature point pairs include one image feature and one point cloud feature; Calculating the registration transformation matrix between the measurement image and the point cloud data according to the matching pairs, obtaining the optimal solution of the registration transformation matrix by the least squares method, and aligning the measurement image and the point cloud data according to the optimal solution; Fusing the aligned measurement image and the point cloud data.

[0025] Extract image features from the measured image. These features mainly include edges, corners, and textures. Edge: It is the area where the gray level in the image changes sharply, which can reflect the contour information of the object. For example, in the measured image of the cargo stack, the edge can clearly outline the boundary of the cargo stack, helping to determine the position and shape of the cargo stack in the image subsequently. Corner: It is the point with a relatively large local curvature in the image, usually located at the intersection of two edges. Corners have good stability and distinguishability, and can provide important reference points for image registration. For example, there may be corners at some prominent parts or edge turning points of the cargo stack. Texture: It reflects the spatial distribution pattern of pixel gray levels in the image, and different object surfaces have different texture characteristics. By extracting texture features, different parts of the cargo stack or different types of goods can be further distinguished. Extract point cloud features from the point cloud data, including curvature, normal vector, and local density. Curvature: It describes the degree of bending of the point cloud surface at a certain point. In the point cloud data of the cargo stack, the curvature can reflect the undulation changes of the cargo surface. For example, the convex or concave parts on the cargo surface have different curvature characteristics. Normal vector: It is the vector perpendicular to the tangent plane of the point cloud surface at a certain point, which can represent the surface direction of that point. The normal vector information is very important for determining the orientation and spatial pose of the cargo stack surface. Local density: It reflects the density of points within a certain range around a certain point in the point cloud. The local density of different parts of the cargo stack may be different. For example, the local density is relatively large in the area where the goods are stacked tightly, while it is relatively small in the area with gaps. Calculate the Euclidean distance between the extracted image features and point cloud features. The Euclidean distance is a commonly used distance metric method for measuring the similarity between two feature points. For each image feature, calculate its Euclidean distance from all point cloud features. Determine the feature point pairs with the Euclidean distance less than the preset threshold as matching pairs. The preset threshold is set according to the actual situation and is used to screen out the feature point pairs that may have corresponding relationships. A matching pair includes an image feature and a point cloud feature, and these matching pairs provide the basis for calculating the registration transformation matrix subsequently. According to the determined matching pairs, calculate the registration transformation matrix between the measured image and the point cloud data. The registration transformation matrix is a mathematical model that describes how to map the points in the measured image to the corresponding points in the point cloud data, or vice versa. Obtain the optimal solution of the registration transformation matrix through the least squares method. The least squares method is a commonly used optimization method that finds the optimal parameter values by minimizing the sum of the squares of the errors. Here, it is to solve the optimal solution of the registration transformation matrix by minimizing the errors between the matching pairs, so that the difference between the registered measured image and the point cloud data is minimized. Align the measured image and the point cloud data according to the obtained optimal solution. After alignment, the measured image and the point cloud data are in agreement in terms of spatial position, preparing for subsequent data fusion. Fuse the aligned measured image and point cloud data.The fused dataset combines the advantages of image and point cloud data, including both the visual information of images (such as color, texture, etc.) and the precise three-dimensional geometric information of point cloud data. Such a fused dataset can more comprehensively and accurately describe the characteristics of the target cargo stack, providing a more reliable data basis for subsequent tasks such as three-dimensional model generation and quality calculation.

[0026] Extracting image features such as edges, corners, and textures of the measurement image, as well as point cloud features such as curvature, normal vectors, and local density of the point cloud data, can comprehensively describe the geometric and texture information of the measurement image and point cloud data from different dimensions. Different types of features complement each other, helping to more accurately find the corresponding relationship between the measurement image and point cloud data in the subsequent registration process. Calculate the Euclidean distance between the image features and point cloud features, and determine the feature point pairs with Euclidean distance less than the preset threshold as matching pairs. This method can quickly screen out the potentially matching feature point pairs, reducing the subsequent calculation amount. Using the Euclidean distance for feature matching has a certain robustness and can adapt to data changes in different scenarios. Even if there is a certain amount of noise, occlusion, or deformation in the measurement image and point cloud data, as long as the Euclidean distance between the feature points meets the threshold condition, they can still be determined as matching pairs, thus ensuring the reliability of the matching. The calculation method of the registration transformation matrix can adapt to scenarios of different complexity levels. Whether it is a simple regular cargo stack or a complex irregular-shaped cargo stack, as long as enough matching pairs can be extracted, an accurate registration transformation matrix can be calculated through this method to achieve effective alignment of the measurement image and point cloud data. Fusing the aligned measurement image and point cloud data combines the advantages of image and point cloud data. The fused dataset contains both the rich visual information of the image and the precise three-dimensional geometric information of the point cloud data, and can more comprehensively and accurately describe the characteristics of the target cargo stack. The fused dataset provides a better data basis for subsequent tasks such as three-dimensional model generation and quality calculation.

[0027] Optionally, the step of inputting the preliminary three-dimensional model into the generative adversarial network to generate the target three-dimensional model includes: Construct a three-dimensional model feature table and encode the feature elements of the three-dimensional model. The feature elements include geometric features and topological structures, and each feature element has a unique index. When receiving the preliminary three-dimensional model, look up the index of the feature elements of the preliminary three-dimensional model in the three-dimensional model feature table, retrieve the corresponding embedding vectors from the preset embedding matrix using the index, and construct an embedding vector sequence based on the embedding vectors. For the sequence of embedding vectors, calculate the correlation scores between each feature element and other feature elements, construct an attention weight matrix according to the correlation scores, and perform weighted summation on the sequence of embedding vectors according to the attention weight matrix to obtain the context representation of each feature element; Use graph embedding technology to incorporate the context representation into the generative adversarial network, and generate a target 3D model through the generative adversarial network.

[0028] To effectively manage and process the features of a 3D model, a 3D model feature table is first constructed. This feature table can be regarded as a database for storing various feature information of the 3D model. Determine the feature elements of the 3D model, mainly including geometric features (such as the shape, size, curvature, etc. of the model, which describe the specific form of the model in 3D space) and topological structures (such as the connection relationships of the model, the number of holes, etc., which reflect the logical relationships between parts of the model). Encode each feature element and assign a unique index to each feature element. The purpose of doing this is to be able to quickly and accurately locate and reference these feature elements in subsequent processing. For example, when processing a complex 3D cargo stack model, different geometric features (such as the top shape and side profile of the cargo stack) and topological structures (such as whether there are voids inside the cargo stack) can be identified by unique indexes. When receiving the initial 3D model, it is necessary to analyze the feature elements of the model. By looking up the indexes of the feature elements of the initial 3D model in the 3D model feature table, the positions of these feature elements in the feature table can be determined. Use the found indexes to retrieve the corresponding embedding vectors from a preset embedding matrix. The embedding matrix is a pre-trained matrix that maps each feature element to a high-dimensional embedding vector, and these embedding vectors contain the semantic and geometric information of the feature elements. According to the retrieved embedding vectors, construct an embedding vector sequence. This sequence is arranged in the order of the feature elements in the initial 3D model, providing a basis for subsequent feature processing. For example, for a cargo stack model, the embedding vectors corresponding to the geometric features and topological structures of different parts of the cargo stack will be arranged in sequence according to the construction order of the cargo stack. For the constructed embedding vector sequence, calculate the correlation score between each feature element and other feature elements. The correlation score reflects the degree of association between feature elements. For example, in a cargo stack model, there may be a certain geometric association between the top shape and the side profile of the cargo stack, and their correlation score will be relatively high. The method for calculating the correlation score can adopt common similarity measurement methods such as cosine similarity and dot product. According to the calculated correlation scores, construct an attention weight matrix. Each element in the attention weight matrix represents the attention weight between the corresponding feature elements. The larger the weight, the more important the association between the two feature elements. Then, according to the attention weight matrix, perform a weighted sum on the embedding vector sequence to obtain the context representation of each feature element. The context representation comprehensively considers the information of the feature element itself and the information of other feature elements associated with it, and can more comprehensively describe the role and meaning of the feature element in the model. For example, through the weighted sum, the context representation of a certain feature element in the cargo stack model not only contains its own geometric and topological information, but also integrates the information of other feature elements associated with it, making the representation of this feature element more rich and accurate. Use graph embedding technology to integrate the obtained context representation into a generative adversarial network.Graph embedding technology can represent the feature elements of a 3D model and the relationships between them as a graph structure, and map the nodes (feature elements) and edges (relationships between feature elements) in the graph structure to a low-dimensional space to obtain graph embedding vectors. These graph embedding vectors contain the overall structure and feature information of the model, and can better represent the complex relationships of the 3D model. Inputting the graph embedding vectors into a generative adversarial network can provide richer model information for the generator and help the generator generate more realistic 3D models. The generative adversarial network consists of a generator and a discriminator. The generator generates a 3D model based on the input graph embedding vectors, while the discriminator attempts to distinguish the generated 3D model from the real 3D model. Through the adversarial training between the generator and the discriminator, the generator continuously optimizes the generated 3D model to make it more realistic and accurate. Finally, the target 3D model generated by the generative adversarial network can better reflect the features of the preliminary 3D model and has improvements in both details and overall quality. For example, in the scenario of port cargo stacks, the generated target 3D model can more accurately present the shape, texture, and stacking method of the cargo stacks, providing a more reliable basis for the cargo management of the port.

[0029] Construct a three-dimensional model feature table and encode feature elements such as geometric features and topological structures, providing a standardized representation for the features of the three-dimensional model. This enables the features of preliminary three-dimensional models from different sources and in different formats to be stored and processed in a unified form, facilitating subsequent feature retrieval and operations. Assign a unique index to each feature element. When a preliminary three-dimensional model is received, the corresponding feature element can be quickly located by looking up the index. This efficient retrieval mechanism can significantly reduce the time for feature processing and improve the efficiency of the entire three-dimensional model generation process. Retrieve the corresponding embedding vector from a preset embedding matrix using the index, mapping the feature element from a discrete symbolic representation to a continuous vector space. This semantic mapping can capture the potential semantic relationships between feature elements, enabling subsequent calculations to better understand the meaning of the features. Construct an embedding vector sequence based on the retrieved embedding vectors, providing an ordered data structure for subsequent feature processing. The serialized representation facilitates the use of various sequence processing algorithms to analyze and operate on the features, such as the attention mechanism. Calculate the correlation score between each feature element and other feature elements, which can capture the internal associations between feature elements. Construct an attention weight matrix based on the correlation score and perform weighted summation on the embedding vector sequence according to this matrix to obtain the context representation of each feature element. This dynamic feature weighting method can highlight important feature elements and suppress unimportant feature elements, enabling the model to pay more attention to the features that have a key impact on the generation of the three-dimensional model, thereby improving the accuracy of generating the target three-dimensional model. Incorporate the context representation into the generative adversarial network using graph embedding technology, which can better utilize the structural information of the three-dimensional model. Graph embedding technology can represent the topological structure of the three-dimensional model as a graph structure and embed the feature information into the nodes of the graph, enabling the generative adversarial network to learn the structural features of the three-dimensional model and the relationships between features. By combining graph embedding technology with the generative adversarial network, the generative adversarial network can more accurately learn the distribution features of the three-dimensional model, thereby generating a more realistic and accurate target three-dimensional model.

[0030] Optionally, generating the target three-dimensional model through the generative adversarial network includes: Generate a sample model through the generator of the generative adversarial network and input both the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network; Calculate the loss functions of the generator and the discriminator according to the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, update the parameters of the generator and the discriminator through the backpropagation algorithm, and perform a new round of iteration through the updated generator and discriminator; When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate three-dimensional model; According to the curvature magnitude of the surface triangular patches of the intermediate three-dimensional model, redundant triangular patches in the intermediate three-dimensional model are removed, topological structure features in the intermediate three-dimensional model are identified using the principles of topology to determine the location and type of defects, and a repair algorithm based on topological transformation is used to repair the detected defects to obtain the target three-dimensional model.

[0031] The generator in a generative adversarial network is a neural network model. Its role is to generate sample models similar to real data (in this case, 3D models) based on the input random noise (or other latent variables). Through continuous learning and optimization, the generator attempts to generate increasingly realistic 3D models to deceive the discriminator. The discriminator is also a neural network model, and its task is to distinguish whether the input model is a sample model generated by the generator or a real preliminary 3D model. The discriminator will analyze and judge the input model and output a probability value indicating the likelihood that the model is a real model. By inputting both the sample model and the preliminary 3D model into the discriminator, the discriminator will give their respective discrimination results. Based on the discrimination results of the discriminator for the sample model and the preliminary 3D model, the loss functions of the generator and the discriminator are calculated. The loss functions are used to measure the performance of the generator and the discriminator. The goal of the generator is to generate as realistic sample models as possible, making it difficult for the discriminator to distinguish; while the goal of the discriminator is to accurately judge the source of the input model. The calculation of the loss function is based on the difference between the discrimination result of the discriminator and the real labels (the sample model is false, and the preliminary 3D model is true). Through the backpropagation algorithm, according to the calculated loss function value, the parameters of the generator and the discriminator are updated. The backpropagation algorithm is an optimization algorithm for training neural networks. It can adjust the values of the parameters according to the gradient information of the loss function with respect to the network parameters to reduce the loss function, thereby improving the performance of the generator and the discriminator. After updating the parameters, the generator and the discriminator will perform a new round of iteration. The generator continues to generate new sample models, and the discriminator continues to make discriminations. When the generative adversarial network reaches the preset number of iterations or performance convergence, the iterative process terminates. The preset number of iterations is determined according to actual requirements and computing resources, and performance convergence usually means that the loss function values of the generator and the discriminator no longer change significantly, or the discrimination accuracy of the discriminator reaches a certain threshold. After the iteration terminates, the generator generates an intermediate 3D model. This intermediate 3D model is a relatively realistic 3D model generated by the generator after multiple iterations of optimization, but there may still be some defects or areas that need further optimization. According to the curvature magnitude of the surface triangular patches of the intermediate 3D model, redundant triangular patches are removed. Curvature can reflect the degree of curvature of the surface where the triangular patch is located. Triangular patches with smaller curvatures may contribute less to the shape and structure of the model and can be regarded as redundant patches. Removing redundant patches can reduce the data volume of the model, improve the storage and computing efficiency of the model, and at the same time maintain the main shape features of the model. Use topological principles to identify the topological structure features in the intermediate 3D model. Topology focuses on the invariant properties of objects under continuous deformation. By analyzing the topological structure of the model, the positions and types of possible defects in the model can be determined. For example, holes, cracks, etc. in the topological structure can be identified as defects. Adopt a repair algorithm based on topological transformation to repair the detected defects.Topological transformation can change the topological structure of a model while maintaining the overall shape and features of the model. Based on the topological transformation, the repair algorithm can perform corresponding topological operations on the model according to the type and location of the defects, such as filling holes, repairing cracks, etc., so as to obtain the target 3D model. The target 3D model is a high-quality 3D model after optimization and repair, which can more accurately reflect the actual shape and structure of the port cargo stack.

[0032] The generator generates a sample model and inputs it together with the preliminary 3D model into the discriminator. The discriminator provides a feedback signal to the generator by discriminating between the sample model and the preliminary 3D model. The generator continuously adjusts its own parameters according to the feedback of the discriminator, making the generated sample model closer and closer to the real 3D model (the preliminary 3D model can be regarded as a reference real model), thereby improving the authenticity and accuracy of the generated target 3D model. By calculating the loss functions of the generator and the discriminator and using the backpropagation algorithm to update the parameters, the performance of the generator and the discriminator can be continuously optimized. The design of the loss function makes the sample model generated by the generator more in line with the characteristics of the real 3D model in terms of geometric shape, texture, etc., thereby improving the overall quality of the target 3D model. Through continuous new rounds of iterative training, the generative adversarial network can gradually approach the optimal solution. Each iteration improves the performance of the generator and the discriminator, and the generated intermediate 3D model is also closer and closer to the ideal target 3D model. When the generative adversarial network reaches the preset number of iterations or performance convergence, the intermediate 3D model is generated. This control mechanism can avoid problems such as overtraining or under-training. The preset number of iterations can be adjusted according to the actual situation, and the judgment criteria for performance convergence can also be set according to specific requirements, so as to ensure the effectiveness and controllability of the training process. Removing redundant triangular patches according to the curvature size of the triangular patches on the surface of the intermediate 3D model can simplify the model structure and reduce the storage space and computational complexity of the model. The curvature size reflects the degree of bending of the patch. Removing redundant patches with smaller curvature and little impact on the overall shape of the model can improve the efficiency of the model while ensuring the accuracy of the model. Using topological principles to identify the topological structure features in the intermediate 3D model can accurately determine the location and type of defects. Topological principles focus on the connectivity, holes and other structural features of the model. By analyzing these features, abnormal situations in the model, such as defects like holes and cracks, can be found. Adopting a repair algorithm based on topological transformation to repair the detected defects can make the repaired model maintain its original topological structure and geometric shape. Topological transformation can repair local defects without changing the overall topological properties of the model, thereby obtaining a high-quality target 3D model.

[0033] Optionally, the constructing the density model of the target cargo stack according to the cargo characteristic data and the environmental factor data includes: Obtain the data correlation between the cargo characteristic data and the environmental factor data through principal component analysis; Respectively construct a first influence model of each factor in the cargo characteristic data on the cargo density, and respectively construct a second influence model of each factor in the environmental factor data on the cargo density; Integrate the first influence model and the second influence model according to the data correlation to construct a density model.

[0034] Principal Component Analysis (PCA) is a commonly used method for data dimensionality reduction and feature extraction. Its core idea is to transform the original data into a new set of orthogonal variables, namely principal components, through linear transformation. These principal components are linear combinations of the original variables and are arranged in descending order of variance. The first principal component has the largest variance and can reflect the variation information of the original data to the greatest extent, and the subsequent principal components decrease in turn. In the port cargo stack scenario, there may be complex correlation relationships between cargo characteristic data (such as cargo source, transportation mode, storage time, and stacking method) and environmental factor data (such as temperature, humidity, wind speed, air pressure, and light intensity). Principal Component Analysis can process these multi-dimensional data and find the main change directions and potential correlations in the data. For example, certain cargo sources may correspond to specific transportation modes, and under different storage times, the degree of influence by environmental factors may vary. Principal component analysis can reveal these hidden correlation patterns. By extracting the principal components, the original high-dimensional data can be transformed into a low-dimensional principal component space, reducing the dimension and complexity of the data. This not only reduces the computational amount of subsequent model construction but also avoids the overfitting problem caused by too high data dimensionality and improves the generalization ability of the model. The influencing mechanisms of each factor in the cargo characteristic data (cargo source, transportation mode, storage time, and stacking method) on the cargo density are different. For example, goods from different sources may have different densities due to factors such as raw materials and production processes; jolts and squeezes during transportation may change the stacking state of the goods, thereby affecting the density distribution; the longer the storage time, the goods may change in density due to moisture absorption, deterioration, etc.; different stacking methods will affect the void ratio and density distribution inside the cargo stack. For each factor, a first influence model on the cargo density can be constructed separately. These models can use machine learning methods such as regression analysis, decision trees, and neural networks. For example, for the cargo source factor, density data of goods from different sources can be collected, and a quantitative relationship model between the cargo source and density can be established using regression analysis; for the storage time factor, time series analysis methods can be used to analyze the change trend of density over storage time. Each factor in the environmental factor data (temperature, humidity, wind speed, air pressure, and light intensity) will also affect the cargo density. For example, an increase in temperature may cause the goods to expand and the density to decrease; an increase in humidity may cause the goods to absorb water and the density to increase; factors such as wind speed, air pressure, and light intensity may also indirectly affect the density by affecting the physical properties or chemical reactions of the goods. Similarly, for each environmental factor, a second influence model on the cargo density can be constructed separately. These models can select appropriate modeling methods according to the specific relationship between the environmental factor and the density. For example, for the relationship between temperature and density, if it shows a linear change trend, a linear regression model can be used; if the relationship is more complex, non-linear regression or neural network models can be considered. The data correlations obtained through principal component analysis reflect the internal connections between the cargo characteristic data and the environmental factor data.When constructing the density model, integrating the first influence model and the second influence model based on these data associations can comprehensively consider the combined effects of various factors on the cargo density. For example, if the data association indicates an interaction between the cargo source and temperature, then this interaction's impact on density needs to be considered in the integrated model. The integrated density model can more comprehensively and accurately describe the relationship between the cargo density, the cargo characteristic data, and the environmental factor data. Compared with a single influence model, the integrated model can better capture the complexity and non-linear relationships in the data, thereby improving the accuracy of density prediction. For example, in practical applications, through the integrated model, the density distribution of the target cargo stack can be more accurately calculated based on the given cargo characteristic data and environmental factor data, providing a reliable basis for subsequent tasks such as quality calculation.

[0035] Cargo characteristic data (such as cargo source, transportation method, storage time, and stacking method) and environmental factor data (such as temperature, humidity, wind speed, air pressure, and light intensity) usually contain multiple dimensions, and there may be complex correlations between the dimensions. The principal component analysis method can transform this multi-dimensional data into a few principal components. These principal components retain the main information in the original data while reducing the data dimension. This makes the subsequent analysis of data associations simpler and more efficient, reducing the computational complexity. Separately constructing the first influence model of each factor in the cargo characteristic data on the cargo density and the second influence model of each factor in the environmental factor data on the cargo density can accurately quantify the influence degree of each factor on the cargo density. Separately modeling the cargo characteristic data and the environmental factor data allows for more detailed analysis of different types of data. The cargo characteristic data and the environmental factor data have different characteristics and laws. Separately modeling can better capture the features of each type of data, improving the accuracy and adaptability of the model. Separately constructing the influence model makes it more convenient in the subsequent model adjustment and optimization process. If it is found that the influence of a certain factor on the cargo density does not match the expectation, the influence model corresponding to this factor can be adjusted separately without affecting the influence models of other factors, improving the maintainability of the model. Integrating the first influence model and the second influence model based on the data association to construct the density model can comprehensively consider the combined effects of the cargo characteristic data and the environmental factor data on the cargo density. This comprehensive model can more comprehensively reflect the actual situation, avoid the limitations of the single-factor model, and improve the accuracy and reliability of the density model.

[0036] Optionally, the integrating the first influence model and the second influence model according to the data association to construct the density model includes: According to the results of the principal component analysis, assign weights to each factor in the cargo characteristic data and the environmental factor data; Determine the first contribution value of each factor in the cargo characteristic data to the cargo density according to the first influence model, and determine the second contribution value of each factor in the environmental factor data to the cargo density according to the second influence model; Perform weighted summation on the cargo characteristic data and the environmental factor data according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value, so as to construct a density model.

[0037] Principal component analysis is a dimensionality reduction technique that transforms multiple correlated variables into a few uncorrelated principal components through linear transformation of the original data. In this process, each principal component contains a part of the information of the original data, and the amount of information (variance contribution rate) contained in different principal components is different. Among the cargo characteristic data (such as cargo type, packaging method, storage duration, etc.) and environmental factor data (such as temperature, humidity, air pressure, etc.), the influence degrees of various factors on the cargo density are different. Through principal component analysis, the load (i.e., the correlation coefficient between the factor and the principal component) of each factor in each principal component can be calculated, as well as the variance contribution rate of each principal component. Based on this information, weights are assigned to each factor. Generally speaking, factors with larger loads in the principal component and higher variance contribution rates of the principal component where they are located will be given larger weights because these factors have a more significant impact on the cargo density. For example, if temperature has a large load in a certain principal component and the variance contribution rate of this principal component is high, then the temperature factor will be given a higher weight, indicating that temperature has a greater impact on the cargo density. The first influence model is constructed for each factor in the cargo characteristic data to quantify the influence of each factor on the cargo density. For each factor in the cargo characteristic data, the first contribution value of this factor to the cargo density can be calculated through the first influence model. This contribution value reflects the specific influence degree of this factor on the cargo density when only considering the cargo characteristic factors. For example, for the factor of cargo type, the first influence model may calculate the contribution value of this type of cargo to the overall cargo density according to the physical properties (such as density, porosity, etc.) of different cargo types. Similarly, the second influence model is constructed for each factor in the environmental factor data to quantify the influence of each environmental factor on the cargo density. For each factor in the environmental factor data, the second contribution value of this factor to the cargo density can be calculated through the second influence model. This contribution value reflects the specific influence degree of this factor on the cargo density when only considering the environmental factors. For example, for the factor of humidity, the second influence model may calculate the contribution value of humidity to the cargo density according to the influence of humidity changes on the water absorption, swelling, etc. of the cargo. After obtaining the weights, the first contribution values, and the second contribution values of each factor, the comprehensive density prediction value is calculated by the method of weighted summation. For each factor in the cargo characteristic data, multiply its first contribution value by the corresponding weight; for each factor in the environmental factor data, multiply its second contribution value by the corresponding weight. Then add up the weighted contribution values of all factors to obtain the comprehensive density prediction value. This prediction value comprehensively considers the influence of cargo characteristic factors and environmental factors on the cargo density. Taking the comprehensive density prediction value as the output and the cargo characteristic data and environmental factor data as the input, a density model is constructed.The model can be represented as a mathematical function or algorithm. When new cargo characteristic data and environmental factor data are input, the model can quickly calculate the corresponding predicted value of cargo density based on the pre-calculated weights and influence model. For example, in port cargo stack management, when new cargo is stored in the warehouse, by inputting the characteristic data of the cargo and the current environmental factor data, the density model can predict the density of the cargo stack, thereby providing a basis for the quality calculation and management of the cargo stack.

[0038] Weights are assigned to each factor in the cargo characteristic data and environmental factor data according to the results of principal component analysis, which can quantify the relative importance of different factors on cargo density. Principal component analysis reveals the internal relationships and structures among the data, and the weights determined through this analysis reflect the status and roles of each factor in the overall data. Under different ports, cargo types, and storage conditions, the influence degrees of various factors on cargo density may vary. Reasonable weight assignment can make the density model more in line with the actual situation, improving the pertinence and applicability of the model. Weight assignment provides a clear structural framework for subsequent model construction. By clarifying the weights of each factor, the model can more systematically integrate the cargo characteristic data and environmental factor data, avoiding the disorderly effects of each factor in the model, and helping to improve the stability and interpretability of the model. According to the first influence model, the first contribution value of each factor in the cargo characteristic data to the cargo density is determined, and according to the second influence model, the second contribution value of each factor in the environmental factor data to the cargo density is determined, which can accurately evaluate the specific influence degree of each factor on the cargo density. The determination of this contribution value is based on the calculation results of each influence model, with high accuracy and reliability. The determination of the contribution value provides a clear basis for model debugging and optimization. If it is found that the contribution value of a certain factor does not match the expectation, the corresponding influence model for this factor can be adjusted, or the weight assignment of this factor can be re-examined, thereby gradually optimizing the performance of the model. Clarifying the contribution values of each factor makes the density model more interpretable. In practical applications, managers can, based on the contribution values of each factor, understand which factors have a greater impact on cargo density, and thus take targeted measures to control cargo density and improve the management efficiency of port cargo stacks. By performing weighted summation on the cargo characteristic data and environmental factor data according to the weights, the first contribution value, and the second contribution value, a comprehensive density prediction value is obtained, which can comprehensively consider the influence of multiple factors on cargo density. This comprehensive prediction method avoids the limitations of single-factor models, enabling the density model to more comprehensively and accurately reflect the actual situation.

[0039] Optionally, the calculating of the quality data of the target cargo stack according to the target three-dimensional model and the density model includes: Divide the target three-dimensional model into multiple cubic units, determine the target volume of each cubic unit, determine the predicted density of each cubic unit according to the density model, and obtain the target mass based on the target volume and the predicted density. Add up the multiple target masses to obtain the total mass of the target cargo stack.

[0040] The target three-dimensional model is an accurate representation of the geometric shape of the target cargo stack in three-dimensional space. Since the shape of the cargo stack is usually irregular, it is relatively complex to directly calculate its mass. By dividing it into multiple cubic units, the complex geometric shape problem can be transformed into a problem of dealing with simple geometric bodies (cubes), greatly simplifying the calculation process. In the fields of computer graphics and 3D modeling, there are various algorithms to achieve mesh division of 3D models, such as the octree algorithm. The octree algorithm recursively divides the three-dimensional space into eight subspaces of equal volume until the geometric shape within each subspace is simple enough (such as approximately a cube), thus completing the division of the target three-dimensional model. This division method facilitates subsequent independent mass calculation for each unit, and can flexibly adjust the size and quantity of cubic units according to the complexity of the cargo stack and the requirements of calculation accuracy. The volume of a cubic unit is one of the important parameters for its mass calculation. After dividing the cubic units, according to the geometric characteristics of the cube, its volume can be calculated through a simple formula. For a cubic unit with side length a, its volume V = a 3. In practical applications, since the divided cube units may not be strictly standard cubes, their volumes can be determined by approximate calculation or numerical integration methods. For example, for some irregular approximate cube units, they can be divided into multiple small standard cubes or other numerical methods can be used to estimate their volumes. Accurately determining the volume of each cube unit is one of the key steps to ensure the accuracy of mass calculation. The density model describes the relationship between the cargo density and the cargo characteristic data and environmental factor data. Each cube unit can be regarded as a local area in the cargo stack, and the cargo density in this area may be affected by various factors, such as the type of cargo, storage time, temperature and humidity of the environment where it is located, etc. Through the density model, the cargo density of each cube unit can be predicted according to the relevant data corresponding to it. First, it is necessary to determine the position of each cube unit in the target three-dimensional model and the corresponding cargo characteristic data and environmental factor data. For example, through the method of spatial interpolation, the environmental factor data at the position of this cube unit can be estimated according to the data of the surrounding known points. Then, these data are input into the density model to obtain the predicted density ρ of each cube unit. The accuracy of the predicted density directly affects the accuracy of the final mass calculation, so the construction and parameter determination of the density model are crucial. According to the mass calculation formula in physics m = ρV (where m is the mass, ρ is the density, and V is the volume), for each cube unit, knowing its target volume V and predicted density ρ, the target mass m of this unit can be calculated. Suppose the volume V of a cube unit is 1m 3 , and the predicted density ρ = 800 kg / m 3 , then the target mass m of this unit is m = ρV = 800×1 = 800 kg. This step combines the geometric information and density information to obtain the mass of each local area, laying a foundation for the subsequent calculation of the total mass of the cargo stack. The target cargo stack is composed of multiple cube units, and each unit has its corresponding target mass. Adding up the target masses of all cube units can obtain the total mass of the target cargo stack.

[0041] The target 3D model is divided into multiple cubic units, transforming the originally complex 3D model calculation problem into a calculation problem for multiple simple cubic units. Cubic units have regular geometric shapes, and their volume calculation is relatively simple, greatly reducing the complexity of calculating the mass of the target cargo stack and improving the calculation efficiency. By reasonably dividing the size of the cubic units, the calculation accuracy can be controlled. Smaller cubic units can more precisely fit the shape of the target 3D model, resulting in a more accurate mass calculation result; while larger cubic units have a faster calculation speed but may sacrifice some accuracy. In practical applications, the size of the cubic units can be flexibly adjusted according to specific requirements to find a balance between calculation efficiency and accuracy. The divided cubic units are independent of each other, facilitating parallel calculation. Using a multi-core processor or a distributed computing system to simultaneously calculate the mass of multiple cubic units can significantly shorten the calculation time and is applicable to the mass calculation scenario of large-scale target cargo stacks. Determining the target volume of each cubic unit can accurately quantify the space occupied by the target cargo stack at different positions. This is crucial for subsequent mass calculation because mass is closely related to volume and density. Accurate volume calculation can avoid mass calculation errors caused by inaccurate volume estimation. Even if the target 3D model has a complex shape, by dividing it into cubic units and determining the volume of each unit, this complexity can be effectively handled. Cubic units can fill various irregular regions of the target 3D model, thereby achieving an accurate calculation of the entire cargo stack volume. The target volume is one of the key parameters for calculating the target mass. Only by accurately determining the volume of each cubic unit can the target mass of each unit be calculated in combination with the density model, and then the total mass of the target cargo stack can be obtained. The density model synthesizes the influence of cargo characteristic data and environmental factor data on the cargo density and can more comprehensively and accurately reflect the density conditions at different positions in the target cargo stack. By determining the predicted density of each cubic unit through the density model, considering the effects of various factors on density, the mass calculation result is more in line with the actual situation. In the port environment, the cargo characteristic data and environmental factor data may change over time. The density model can be updated in real time according to new data, thereby dynamically adjusting the predicted density of each cubic unit. This enables the mass calculation to adapt to different storage conditions and cargo states, improving the practicality and reliability of the model. Accurate density prediction is the key to mass calculation. The density model can provide a more accurate density prediction value by comprehensively considering various factors, thereby improving the calculation accuracy of the target mass. Accurate total mass data of the target cargo stack is of great significance for the operation decision-making of the port. For example, in aspects such as cargo loading and unloading, warehousing cost calculation, and transportation plan arrangement, accurate mass data is required as a basis. The mass data calculated by this method can provide reliable support for these decisions, improving the operation efficiency and economy of the port.

[0042] This embodiment also discloses a 3D yard measurement system based on an unmanned aerial vehicle. Figure 2 It is a schematic diagram of the modules of a 3D yard measurement system based on an unmanned aerial vehicle disclosed in an embodiment of the present application. As Figure 2 shown, the system includes an acquisition module 201, a model module 202, a density module 203, and a calculation module 204, where: The acquisition module 201 is configured to control the unmanned aerial vehicle to fly along a preset flight path to acquire measurement images and point cloud data of each target cargo stack in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and the point cloud data to generate a fused data set; The model module 202 is configured to input the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo stack, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; The density module 203 is configured to obtain the cargo characteristic data and environmental factor data of the target cargo stack, and construct a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data. The cargo characteristic data includes the cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; The calculation module 204 is configured to calculate the mass data of the target cargo stack according to the target three-dimensional model and the density model.

[0043] Optionally, the acquisition module 201 is configured to: Extract the image features of the measurement image, where the image features include edges, corners, and textures, and extract the point cloud features of the point cloud data, where the point cloud features include curvature, normal vector, and local density; Calculate the Euclidean distance between the image features and the point cloud features, and determine the feature point pairs with the Euclidean distance less than a preset threshold as matching pairs. The feature point pairs include one image feature and one point cloud feature; Calculate the registration transformation matrix between the measurement image and the point cloud data according to the matching pairs, obtain the optimal solution of the registration transformation matrix by the least squares method, and align the measurement image and the point cloud data according to the optimal solution; Fuse the aligned measurement image and the point cloud data.

[0044] Optionally, the model module 202 is configured to: Construct a three-dimensional model feature table, and encode the feature elements of the three-dimensional model. The feature elements include geometric features and topological structures, and each feature element has a unique index; When the preliminary three-dimensional model is received, look up the index of the feature elements of the preliminary three-dimensional model in the three-dimensional model feature table, retrieve the corresponding embedding vectors from a preset embedding matrix using the index, and construct an embedding vector sequence based on the embedding vectors; For the embedding vector sequence, calculate the correlation scores between each feature element and other feature elements, construct an attention weight matrix based on the correlation scores, and perform weighted summation on the embedding vector sequence according to the attention weight matrix to obtain the context representation of each feature element; Use graph embedding technology to integrate the context representation into the generative adversarial network, and generate a target three-dimensional model through the generative adversarial network.

[0045] Optionally, the model module 202 is configured to: Generate a sample model through the generator of the generative adversarial network, and input both the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network; Calculate the loss functions of the generator and the discriminator according to the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, update the parameters of the generator and the discriminator through the backpropagation algorithm, and perform a new round of iteration through the updated generator and discriminator; When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate three-dimensional model; According to the curvature magnitude of the surface triangular patches of the intermediate three-dimensional model, remove the redundant triangular patches in the intermediate three-dimensional model, identify the topological structure features in the intermediate three-dimensional model using topological principles to determine the location and type of defects, and adopt a repair algorithm based on topological transformation to repair the detected defects to obtain the target three-dimensional model.

[0046] Optionally, the density module 203 is configured to: Obtain the data association between the cargo characteristic data and the environmental factor data through principal component analysis; Construct first influence models of each factor in the cargo characteristic data on the cargo density respectively, and construct second influence models of each factor in the environmental factor data on the cargo density respectively; Integrate the first influence model and the second influence model according to the data association to construct a density model.

[0047] Optionally, the density module 203 is configured to: Assign weights to each factor in the cargo characteristic data and the environmental factor data according to the results of principal component analysis; Determine the first contribution value of each factor in the cargo characteristic data to the cargo density according to the first influence model, and determine the second contribution value of each factor in the environmental factor data to the cargo density according to the second influence model; Perform weighted summation on the cargo characteristic data and the environmental factor data according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value, so as to construct a density model.

[0048] Optionally, the calculation module 204 is configured to: Divide the target three-dimensional model into multiple cubic units, determine the target volume of each cubic unit, determine the predicted density of each cubic unit according to the density model, and obtain the target mass according to the target volume and the predicted density, and add up the multiple target masses to obtain the total mass of the target cargo stack.

[0049] It should be noted that when the device provided in the above embodiment realizes its functions, only the division of the above functional modules is used for illustration. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process can be found in the method embodiment, which will not be repeated here.

[0050] This embodiment also discloses an electronic device. Refer to Figure 3 , the electronic device may include: at least one processor 301, at least one communication bus 302, a user interface 303, a network interface 304, and at least one memory 305.

[0051] Among them, the communication bus 302 is used to realize the connection and communication between these components.

[0052] Among them, the user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may further include a standard wired interface and a wireless interface.

[0053] Among them, the network interface 304 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0054] Among them, the processor 301 may include one or more processing cores. The processor 301 connects various parts within the entire server using various interfaces and lines. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling the data stored in the memory 305, it performs various functions of the server and processes data. Optionally, the processor 301 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor 301 may integrate a combination of one or several of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc. Among them, the CPU mainly processes the operating system, user interface, application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301 and may be implemented separately by a single chip.

[0055] Among them, the memory 305 may include random access memory (RAM) and may also include read-only memory. Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area can store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store the data involved in the above-mentioned various method embodiments. Optionally, the memory 305 may also be at least one storage device located far from the aforementioned processor 301. As Figure 3 shown, the memory 305, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for a 3D yard measurement method based on an unmanned aerial vehicle.

[0056] In Figure 3In the electronic device shown, the user interface 303 is mainly used to provide an interface for the user to input and obtain the data input by the user; and the processor 301 can be used to call an application program stored in the memory 305 for a 3D yard measurement method based on an unmanned aerial vehicle. When executed by one or more processors 301, the electronic device is caused to execute the method as described in one or more of the above embodiments.

[0057] It should be noted that, for the foregoing method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0058] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0059] In several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0060] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0061] In addition, in each embodiment of this application, the functional units can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0062] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. And the aforementioned memory 305 includes: various media such as USB flash drives, mobile hard disks, magnetic disks, or optical discs that can store program codes.

[0063] The foregoing are only exemplary embodiments of the present disclosure and should not be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure still fall within the scope covered by the present disclosure. Those skilled in the art will readily think of other implementation manners of the present disclosure after considering the disclosure of the specification. This application aims to cover any variations, uses, or adaptive changes of the present disclosure, and these variations, uses, or adaptive changes follow the general principles of the present disclosure and include well-known common knowledge or conventional technical means in the technical field not described in the present disclosure.

Claims

1. A 3D yard measurement method based on drones, characterized in that, Applied to a port management platform, the method includes: Controlling a drone to fly along a preset flight path to collect measurement images and point cloud data of each target cargo stack in the port, registering the measurement images and the point cloud data, and fusing the registered measurement images and the point cloud data to generate a fused data set; Inputting the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo stack, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; Obtaining the cargo characteristic data and environmental factor data of the target cargo stack, and constructing a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data, where the cargo characteristic data includes the cargo source, transportation mode, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; Calculating the mass data of the target cargo stack according to the target three-dimensional model and the density model.

2. The 3D yard measurement method based on an unmanned aerial vehicle according to claim 1, wherein The registering the measurement images and the point cloud data, and fusing the registered measurement images and the point cloud data to generate a fused data set includes: Extracting the image features of the measurement images, where the image features include edges, corners, and textures, and extracting the point cloud features of the point cloud data, where the point cloud features include curvature, normal vector, and local density; Calculating the Euclidean distance between the image features and the point cloud features, and determining the feature point pairs with the Euclidean distance less than a preset threshold as matching pairs, where the feature point pairs include an image feature and a point cloud feature; Calculating a registration transformation matrix between the measurement images and the point cloud data according to the matching pairs, obtaining the optimal solution of the registration transformation matrix by the least squares method, and aligning the measurement images and the point cloud data according to the optimal solution; Fusing the aligned measurement images and the point cloud data.

3. The 3D yard measurement method based on an unmanned aerial vehicle according to claim 1, wherein The inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model includes: Constructing a three-dimensional model feature table and encoding the feature elements of the three-dimensional model, where the feature elements include geometric features and topological structures, and each feature element has a unique index; When receiving the preliminary three-dimensional model, looking up the index of the feature elements of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving the corresponding embedding vectors from a preset embedding matrix using the index, and constructing an embedding vector sequence according to the embedding vectors; For the embedding vector sequence, calculating the correlation scores between each feature element and other feature elements, constructing an attention weight matrix according to the correlation scores, and performing weighted summation on the embedding vector sequence according to the attention weight matrix to obtain the context representation of each feature element; Using graph embedding technology to incorporate the context representation into the generative adversarial network, and generating a target three-dimensional model through the generative adversarial network.

4. The method for 3D yard measurement based on an unmanned aerial vehicle according to claim 3, wherein The generating a target three-dimensional model through the generative adversarial network includes: Generate a sample model through the generator of the generative adversarial network, and input both the sample model and the preliminary 3D model into the discriminator of the generative adversarial network; Calculate the loss functions of the generator and the discriminator according to the discrimination results of the discriminator on the sample model and the preliminary 3D model, update the parameters of the generator and the discriminator through the backpropagation algorithm, and perform a new round of iteration through the updated generator and discriminator; When the generative adversarial network reaches the preset number of iterations or performance convergence, the generator generates an intermediate 3D model; According to the curvature magnitude of the surface triangular patches of the intermediate 3D model, remove the redundant triangular patches in the intermediate 3D model, identify the topological structure features in the intermediate 3D model using topological principles to determine the location and type of defects, and employ a repair algorithm based on topological transformation to repair the detected defects to obtain the target 3D model.

5. The method for 3D yard measurement based on an unmanned aerial vehicle according to claim 1, wherein The constructing the density model of the target cargo stack according to the cargo characteristic data and the environmental factor data includes: Obtain the data association between the cargo characteristic data and the environmental factor data through the principal component analysis method; Construct a first influence model for each factor in the cargo characteristic data on the cargo density respectively, and construct a second influence model for each factor in the environmental factor data on the cargo density respectively; Integrate the first influence model and the second influence model according to the data association to construct a density model.

6. The method for 3D yard measurement based on an unmanned aerial vehicle according to claim 5, wherein The integrating the first influence model and the second influence model according to the data association to construct a density model includes: According to the results of the principal component analysis, assign weights to each factor in the cargo characteristic data and the environmental factor data; Determine the first contribution value of each factor in the cargo characteristic data to the cargo density according to the first influence model, and determine the second contribution value of each factor in the environmental factor data to the cargo density according to the second influence model; Perform weighted summation on the cargo characteristic data and the environmental factor data according to the weights, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value, so as to construct a density model.

7. The method for 3D yard measurement based on an unmanned aerial vehicle according to claim 1, wherein The calculating the mass data of the target cargo stack according to the target 3D model and the density model includes: Divide the target 3D model into multiple cubic units, determine the target volume of each cubic unit, determine the predicted density of each cubic unit according to the density model, and obtain the target mass according to the target volume and the predicted density, and add up the multiple target masses to obtain the total mass of the target cargo stack.

8. A drone-based 3D yard measurement system, characterized in that, Includes a collection module, a model module, a density module, and a calculation module, where: The collection module is configured to control the drone to fly according to a preset flight path to collect measurement images and point cloud data of each target cargo stack in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and the point cloud data to generate a fused data set; A model module, configured to input the dataset into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo stack, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; A density module, configured to obtain the cargo characteristic data and environmental factor data of the target cargo stack, and construct a density model of the target cargo stack according to the cargo characteristic data and the environmental factor data, where the cargo characteristic data includes the cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; A calculation module, configured to calculate the mass data of the target cargo stack according to the target three-dimensional model and the density model.

9. An electronic device, characterized in that, It includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions. The user interface and the network interface are both used to communicate with other devices. The processor is used to execute the instructions stored in the memory, so that the electronic device executes the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1-7 is executed.

Images