A 3D yard measurement method, system, equipment and medium based on drone

Abstract

A drone-based 3D yard measurement method, system, equipment, and medium relate to the field of yard measurement. In this method, a drone is controlled to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in a port. The measurement images and point cloud data are registered, and the registered measurement images and point cloud data are fused to generate a fused dataset. The dataset is input into a preset deep convolutional neural network to generate a preliminary 3D model of the target cargo pile, and the preliminary 3D model is input into a generative adversarial network to generate a target 3D model. Cargo characteristic data and environmental factor data of the target cargo pile are obtained, and a density model of the target cargo pile is constructed based on the cargo characteristic data and environmental factor data. The mass data of the target cargo pile is calculated based on the target 3D model and density model. Implementing the technical solution provided by this application can more accurately and efficiently complete port yard cargo measurement.

Description

Technical Field

[0001] The present application relates to the technical field of storage yard measurement, and in particular to a 3D storage yard measurement method, system, equipment and medium based on drones. Background Art

[0002] In the field of port management, cargo throughput is increasing, and cargo stockpile management has become a crucial component of port operations. Accurate cargo stockpile measurement is crucial for rationally arranging storage space, improving cargo allocation efficiency, and ensuring safe and stable port operations. Accurately understanding the actual cargo stockpile situation helps optimize port resource utilization and reduce space waste. It also provides a scientific basis for cargo loading, unloading, and transportation, thereby improving the operational efficiency and economic benefits of the entire port.

[0003] Traditional port cargo pile measurements rely primarily on manual field surveys. These typically involve workers carrying simple measuring tools, such as tape measures and rods, to the cargo piles themselves. This method requires close contact with the cargo piles, is cumbersome, and is susceptible to factors such as the irregular shape of the cargo piles and complex on-site environments.

[0004] However, these traditional measurement methods have significant drawbacks. Manual field surveys and total station surveys are not only labor-intensive and time-consuming, but also suffer from low accuracy, making them inadequate for the large-scale, high-efficiency operations required by modern ports. This accuracy is particularly degraded when dealing with complex and widely distributed cargo piles. Furthermore, these methods fail to fully account for the impact of cargo characteristics and environmental factors on cargo density and condition, resulting in measurement results that fail to truly reflect the actual cargo pile conditions, impacting port management decisions and operational efficiency. Summary of the Invention

[0005] The present application provides a drone-based 3D yard measurement method, system, equipment, and medium, which can complete port yard cargo measurement more accurately and efficiently by comprehensively considering multiple factors.

[0006] In a first aspect of the present application, a 3D yard measurement method based on a drone is provided, which is applied to a port management platform. The method comprises:

[0007] controlling the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, registering the measurement images and the point cloud data, and fusing the registered measurement images and point cloud data to generate a fused data set;

[0008] Inputting the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model;

[0009] Obtaining cargo characteristic data and environmental factor data of the target cargo pile, and constructing a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity;

[0010] The mass data of the target cargo pile is calculated according to the target three-dimensional model and the density model.

[0011] Optionally, registering the measurement image and the point cloud data, and fusing the registered measurement image and the point cloud data to generate a fused data set includes:

[0012] Extracting image features of the measurement image, wherein the image features include edges, corners, and textures; and extracting point cloud features of the point cloud data, wherein the point cloud features include curvature, normal vector, and local density;

[0013] Calculating the Euclidean distance between the image feature and the point cloud feature, and determining a feature point pair whose Euclidean distance is less than a preset threshold as a matching pair, wherein the feature point pair includes an image feature and a point cloud feature;

[0014] calculating a registration transformation matrix between the measurement image and the point cloud data based on the matching pairs, obtaining an optimal solution of the registration transformation matrix by a least squares method, and aligning the measurement image and the point cloud data based on the optimal solution;

[0015] The aligned measurement image and the point cloud data are fused.

[0016] Optionally, inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model includes:

[0017] Constructing a 3D model feature table to encode feature elements of the 3D model, wherein the feature elements include geometric features and topological structures, and each feature element has a unique index;

[0018] When the preliminary three-dimensional model is received, searching for an index of a characteristic element of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving a corresponding embedding vector from a preset embedding matrix using the index, and constructing an embedding vector sequence based on the embedding vector;

[0019] For the embedding vector sequence, calculating a correlation score between each feature element and other feature elements, constructing an attention weight matrix based on the correlation score, and performing a weighted summation on the embedding vector sequence according to the attention weight matrix to obtain a contextual representation of each feature element;

[0020] The context representation is integrated into the generative adversarial network using graph embedding technology, and a target three-dimensional model is generated through the generative adversarial network.

[0021] Optionally, generating a target three-dimensional model by using the generative adversarial network includes:

[0022] Generate a sample model through the generator of the generative adversarial network, and input the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network;

[0023] Calculating the loss functions of the generator and the discriminator based on the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, updating the parameters of the generator and the discriminator through a back-propagation algorithm, and performing a new round of iteration using the updated generator and the discriminator;

[0024] When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate 3D model;

[0025] According to the curvature of the surface triangular facets of the intermediate three-dimensional model, redundant triangular facets in the intermediate three-dimensional model are removed, and the topological structural features in the intermediate three-dimensional model are identified using topological principles to determine the location and type of defects. The detected defects are repaired using a repair algorithm based on topological transformation to obtain the target three-dimensional model.

[0026] Optionally, constructing the density model of the target cargo pile according to the cargo characteristic data and the environmental factor data includes:

[0027] Obtaining data association between the cargo characteristic data and the environmental factor data through principal component analysis;

[0028] Constructing a first influence model of each factor in the cargo characteristic data on cargo density, and constructing a second influence model of each factor in the environmental factor data on cargo density;

[0029] The first influence model and the second influence model are integrated according to the data association to construct a density model.

[0030] Optionally, the integrating the first influence model and the second influence model according to the data association to construct a density model includes:

[0031] Assigning weights to the various factors in the cargo characteristic data and the environmental factor data based on the results of the principal component analysis;

[0032] Determining a first contribution value of each factor in the cargo characteristic data to cargo density based on the first influencing model, and determining a second contribution value of each factor in the environmental factor data to cargo density based on the second influencing model;

[0033] The cargo characteristic data and the environmental factor data are weightedly summed according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value to construct a density model.

[0034] Optionally, calculating the mass data of the target cargo pile according to the target three-dimensional model and the density model includes:

[0035] The target three-dimensional model is divided into a plurality of cubic units, a target volume of each cubic unit is determined, a predicted density of each cubic unit is determined according to the density model, a target mass is obtained according to the target volume and the predicted density, and a plurality of target masses are added together to obtain the total mass of the target cargo pile.

[0036] In a second aspect of the present application, a 3D storage yard measurement system based on a drone is provided, comprising an acquisition module, a model module, a density module, and a calculation module, wherein:

[0037] an acquisition module configured to control the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and point cloud data to generate a fused data set;

[0038] a model module configured to input the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model;

[0039] a density module configured to obtain cargo characteristic data and environmental factor data of the target cargo pile, and construct a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity;

[0040] A calculation module is configured to calculate mass data of the target cargo pile based on the target three-dimensional model and the density model.

[0041] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device performs any of the methods described above.

[0042] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions. When the instructions are executed, any one of the methods described above is executed.

[0043] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0044] 1. By using drones to fly along pre-set paths, full coverage and highly efficient data collection is achieved, avoiding the limitations and subjectivity of manual measurement while reducing labor costs and safety risks. Images that provide texture and color information are fused with point cloud registration that provides precise geometric information to generate a more complete 3D dataset, addressing the shortcomings of a single data source and improving model accuracy.

[0045] 2. Utilize a deep convolutional neural network to extract features from the fused dataset and quickly generate a preliminary 3D model that preserves the cargo pile's geometry and surface details. Generative adversarial networks refine the preliminary model to generate a higher-resolution, more realistic 3D model of the target, reducing noise and distortion and improving model quality.

[0046] 3. Combining cargo characteristics (such as source, transportation method, storage time) and environmental factors (such as temperature and humidity), a dynamic density model is constructed to reflect density changes in cargo piles under different conditions (such as grain expansion due to moisture and metal oxidation). Based on the target 3D model and density model, the mass of the cargo pile is calculated, avoiding the limitations of traditional weighing methods (such as the inability to directly measure large cargo piles or hazardous materials).

[0047] 4. Real-time, accurate cargo pile quality data supports ports in dynamically adjusting inventory strategies, reducing the risk of overloading or underloading. Density models are used to monitor cargo pile stability (e.g., landslide risk due to excessive humidity), and three-dimensional models are used to analyze structural safety. Accurate quality data helps ports rationally allocate resources such as loading and unloading equipment and transport vehicles, improving operational efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a flow chart of the 3D storage yard measurement method based on drones disclosed in an embodiment of the present application;

[0049] Figure 2Schematic diagram of a module of a drone-based 3D yard measurement system disclosed in an embodiment of the present application;

[0050] Figure 3 This is a structural diagram of an electronic device disclosed in an embodiment of the present application.

[0051] Explanation of the accompanying drawings: 201, acquisition module; 202, model module; 203, density module; 204, calculation module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0053] In the description of the embodiments of this application, words such as "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "for example" or "for instance" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "for example" or "for instance" is intended to present the relevant concepts in a concrete manner.

[0054] In the description of the embodiments of the present application, the term "multiple" means two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0055] This embodiment discloses a 3D yard measurement method based on a drone. Figure 1 This is a flow chart of a 3D yard measurement method based on drones disclosed in the embodiment of this application, which is applied to the port management platform, such as Figure 1 As shown, the method includes the following steps:

[0056] S101, controlling the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, registering the measurement images and the point cloud data, and fusing the registered measurement images and point cloud data to generate a fused data set;

[0057] S102: Inputting the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model;

[0058] S103. Obtain cargo characteristic data and environmental factor data of the target cargo pile, and construct a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity;

[0059] S104: Calculate mass data of the target cargo pile based on the target three-dimensional model and the density model.

[0060] The drone is controlled to fly along a pre-set flight path within the port area. During flight, it uses its onboard sensors (such as cameras and lidar) to collect measurement images and point cloud data of each target cargo pile within the port. Measurement images typically provide visual information about the cargo pile's appearance and color, while point cloud data accurately describes the three-dimensional geometry and spatial position of the cargo pile's surface. Due to differences in acquisition equipment, viewing angle, and time, the collected measurement images and point cloud data may have spatial inconsistencies. Therefore, these data require registration—spatial alignment of the measurement images and point cloud data to ensure accurate correspondence for subsequent fusion processing. The registered measurement images and point cloud data are then fused. The resulting fused dataset combines the advantages of both image and point cloud data, containing both rich visual information and precise three-dimensional geometric information, enabling a more comprehensive and accurate description of the target cargo pile's characteristics. This fused dataset is then fed into a pre-defined deep convolutional neural network. By learning from large amounts of data, deep convolutional neural networks can extract characteristic information about cargo piles from input datasets and, based on this information, generate a preliminary 3D model of the target cargo pile. This preliminary 3D model provides a preliminary estimate of the target cargo pile's shape and structure. This preliminary 3D model is then fed into a generative adversarial network (GAN). A GAN consists of a generator and a discriminator. The generator attempts to generate realistic data, while the discriminator attempts to distinguish generated data from real data. Through adversarial training between the generator and the discriminator, the generator continuously optimizes the generated 3D model, making it more realistic and accurate, ultimately generating the target 3D model. Cargo characteristic data and environmental factor data are obtained for the target cargo pile. Cargo characteristic data includes cargo source, transportation method, storage time, and stacking method. Cargo from different sources may vary in material and density. Bumps and squeezing during transportation can affect the stacking state and density distribution of cargo. Longer storage time can lead to changes in density due to moisture, deterioration, and other factors. Different stacking methods can affect the porosity and density distribution within the cargo pile. Environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity. These environmental factors may affect the physical properties of the cargo and, in turn, the density of the cargo pile. A density model for the target cargo pile is constructed based on the acquired cargo characteristic data and environmental factor data. This model can be a mathematical formula, algorithm, or machine learning-based model that describes the relationship between the cargo characteristic data, environmental factor data, and cargo pile density. Using this density model, the density distribution of the target cargo pile can be calculated based on the given cargo characteristic and environmental factor data. Based on the generated target 3D model and the constructed density model, the mass data of the target cargo pile is calculated. The target 3D model provides volume information for the cargo pile, while the density model provides density information.By multiplying the volume of the cargo pile by its corresponding density, and taking into account density differences within different parts of the cargo pile (if the density model is distributed), the mass data of the target cargo pile can be calculated. This mass data provides an important reference for port cargo management, transportation scheduling, and other aspects.

[0061] Optionally, registering the measurement image and the point cloud data, and fusing the registered measurement image and the point cloud data to generate a fused data set includes:

[0062] Extracting image features of the measurement image, wherein the image features include edges, corners, and textures; and extracting point cloud features of the point cloud data, wherein the point cloud features include curvature, normal vector, and local density;

[0063] Calculating the Euclidean distance between the image feature and the point cloud feature, and determining a feature point pair whose Euclidean distance is less than a preset threshold as a matching pair, wherein the feature point pair includes an image feature and a point cloud feature;

[0064] calculating a registration transformation matrix between the measurement image and the point cloud data based on the matching pairs, obtaining an optimal solution of the registration transformation matrix by a least squares method, and aligning the measurement image and the point cloud data based on the optimal solution;

[0065] The aligned measurement image and the point cloud data are fused.

[0066] Image features are extracted from the measured image. These features primarily include edges, corners, and texture. Edges are regions in an image where grayscale changes dramatically. They can reflect the outline of an object. For example, in a measured image of a cargo pile, edges clearly outline the pile's boundaries, facilitating subsequent determination of the pile's position and shape within the image. Corners are points in the image with significant local curvature, typically located at the intersection of two edges. Corners are stable and distinguishable, providing important reference points for image registration. For example, corners may exist at certain protruding parts or at the edges of a cargo pile. Texture reflects the spatial distribution of pixel grayscale within the image. Different object surfaces exhibit distinct texture characteristics. By extracting texture features, different parts of a cargo pile or different types of cargo can be further distinguished. Point cloud features are extracted from point cloud data, including curvature, normal vector, and local density. Curvature describes the degree of curvature of a point cloud surface at a specific point. In point cloud data of a cargo pile, curvature can reflect surface fluctuations. For example, convex and concave areas of the cargo pile have different curvature characteristics. Normal vector: A vector perpendicular to the tangent plane of a point cloud surface at a specific point. It indicates the surface orientation at that point. Normal vector information is crucial for determining the surface orientation and spatial posture of a cargo pile. Local density: This reflects the density of points within a certain range around a point in the point cloud. Local density may vary in different parts of the cargo pile. For example, areas with densely packed cargo have higher local density, while areas with gaps have lower local density. The Euclidean distance between the extracted image features and the point cloud features is calculated. Euclidean distance is a commonly used distance metric used to measure the similarity between two feature points. For each image feature, the Euclidean distance is calculated between it and all point cloud features. Feature point pairs with a Euclidean distance less than a preset threshold are identified as matching pairs. The preset threshold is set based on actual conditions and is used to filter out feature point pairs that may have a corresponding relationship. A matching pair consists of an image feature and a point cloud feature. These matching pairs provide the basis for the subsequent calculation of the registration transformation matrix. Based on the identified matching pairs, the registration transformation matrix between the measured image and the point cloud data is calculated. The registration transformation matrix is ​​a mathematical model that describes how points in the measured image are mapped to corresponding points in the point cloud data, and vice versa. The optimal solution for the registration transformation matrix is ​​obtained using the least squares method. The least squares method is a commonly used optimization method that finds optimal parameter values ​​by minimizing the sum of squared errors. Here, the optimal solution for the registration transformation matrix is ​​found by minimizing the error between matching pairs, minimizing the difference between the registered measured image and point cloud data. The measured image and point cloud data are aligned based on the obtained optimal solution. After alignment, the measured image and point cloud data are spatially aligned, preparing for subsequent data fusion. The aligned measured image and point cloud data are then fused.The fused dataset combines the strengths of both image and point cloud data, encompassing both the visual information of the image (such as color and texture) and the precise 3D geometric information of the point cloud. This fused dataset can more comprehensively and accurately describe the characteristics of the target cargo pile, providing a more reliable data foundation for subsequent tasks such as 3D model generation and mass calculation.

[0067] Extracting image features such as edges, corners, and texture from the measured image, as well as point cloud features such as curvature, normals, and local density from the point cloud data, comprehensively describes the geometric and textural information of the measured image and point cloud data from different dimensions. These different types of features complement each other, helping to more accurately identify correspondences between the measured image and point cloud data during the subsequent registration process. The Euclidean distance between image and point cloud features is calculated, and feature point pairs with a distance less than a preset threshold are identified as matching pairs. This method quickly identifies potential matching feature point pairs, reducing subsequent computational effort. Using Euclidean distance for feature matching is robust and adaptable to data variations in various scenarios. Even if the measured image and point cloud data contain noise, occlusion, or deformation, matching pairs can still be identified as such as long as the Euclidean distance between feature points meets the threshold, ensuring reliable matching. The registration transformation matrix calculation method is adaptable to scenarios of varying complexity. Whether it's a simple, regular pile or a complex, irregularly shaped pile, as long as sufficient matching pairs are extracted, this method can calculate an accurate registration transformation matrix, effectively aligning the measured image and point cloud data. The aligned measured images and point cloud data are fused, combining the strengths of both. The resulting dataset incorporates both the rich visual information of the images and the precise 3D geometric information of the point cloud data, enabling a more comprehensive and accurate description of the target cargo pile's characteristics. This fused dataset provides a higher-quality data foundation for subsequent tasks such as 3D model generation and mass calculation.

[0068] Optionally, inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model includes:

[0069] Constructing a 3D model feature table to encode feature elements of the 3D model, wherein the feature elements include geometric features and topological structures, and each feature element has a unique index;

[0070] When the preliminary three-dimensional model is received, searching for an index of a characteristic element of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving a corresponding embedding vector from a preset embedding matrix using the index, and constructing an embedding vector sequence based on the embedding vector;

[0071] For the embedding vector sequence, calculating a correlation score between each feature element and other feature elements, constructing an attention weight matrix based on the correlation score, and performing a weighted summation on the embedding vector sequence according to the attention weight matrix to obtain a contextual representation of each feature element;

[0072] The context representation is integrated into the generative adversarial network using graph embedding technology, and a target three-dimensional model is generated through the generative adversarial network.

[0073] To effectively manage and process 3D model features, a 3D model feature table is first constructed. This feature table can be thought of as a database that stores various feature information about the 3D model. Feature elements of the 3D model are identified, primarily including geometric features (such as the model's shape, size, and curvature, which describe the model's specific form in 3D space) and topological structures (such as the model's connectivity and number of holes, which reflect the logical relationships between the model's components). Each feature element is encoded and assigned a unique index. This ensures that these features can be quickly and accurately located and referenced in subsequent processing. For example, when processing a complex 3D cargo pile model, different geometric features (such as the pile's top shape and side profile) and topological structures (such as the presence of internal holes) can be identified by unique indexes. Upon receiving a preliminary 3D model, its feature elements need to be analyzed. By finding their indexes in the 3D model feature table, the locations of these feature elements within the table can be determined. The found indexes are then used to retrieve the corresponding embedding vectors from the pre-set embedding matrix. The embedding matrix is ​​a pre-trained matrix that maps each feature element to a high-dimensional embedding vector. These embedding vectors contain both semantic and geometric information about the feature element. Based on the retrieved embedding vectors, a sequence of embedding vectors is constructed. This sequence follows the order of the feature elements in the preliminary 3D model, providing a foundation for subsequent feature processing. For example, for a cargo pile model, the embedding vectors corresponding to the geometric features and topological structure of different parts are arranged in a sequence based on the order in which the cargo pile was constructed. For this constructed embedding vector sequence, a correlation score is calculated between each feature element and the other feature elements. The correlation score reflects the degree of association between feature elements. For example, in a cargo pile model, the top shape of the pile may have a certain geometric correlation with the side profile, resulting in a high correlation score. Common similarity metrics such as cosine similarity and dot product can be used to calculate the correlation score. Based on the calculated correlation scores, an attention weight matrix is ​​constructed. Each element in the attention weight matrix represents the attention weight between the corresponding feature elements. The larger the weight, the more important the association between the two feature elements. The sequence of embedding vectors is then weighted and summed according to the attention weight matrix to obtain a contextual representation of each feature element. Contextual representations comprehensively consider both the information of a feature element itself and that of other related features, enabling a more comprehensive description of its role and significance within the model. For example, through weighted summation, the contextual representation of a feature element in a cargo pile model not only incorporates its own geometric and topological information but also incorporates information from other related features, making the representation of that feature element richer and more accurate. The resulting contextual representation is then incorporated into a generative adversarial network using graph embedding techniques.Graph embedding technology represents the characteristic elements of a 3D model and their relationships as a graph structure. The nodes (characteristic elements) and edges (relationships between characteristic elements) in the graph structure are mapped into a low-dimensional space, resulting in graph embedding vectors. These graph embedding vectors contain the overall structure and feature information of the model, better representing the complex relationships within the 3D model. Inputting these graph embedding vectors into a generative adversarial network (GAN) provides the generator with richer model information, helping it produce more realistic 3D models. A GAN consists of a generator and a discriminator. The generator generates a 3D model based on the input graph embedding vector, while the discriminator attempts to distinguish the generated 3D model from the real 3D model. Through adversarial training between the generator and discriminator, the generator continuously optimizes the generated 3D model, making it more realistic and accurate. Ultimately, the target 3D model generated by the GAN better reflects the features of the preliminary 3D model, with improved detail and overall quality. For example, in a port cargo stack scenario, the generated target 3D model can more accurately represent the shape, texture, and stacking arrangement of the cargo, providing a more reliable basis for cargo management at the port.

[0074] A 3D model feature table is constructed and feature elements such as geometric features and topological structure are encoded, providing a standardized representation for 3D model features. This allows features from preliminary 3D models from different sources and formats to be stored and processed in a unified format, facilitating subsequent feature retrieval and manipulation. Each feature element is assigned a unique index. Upon receiving the preliminary 3D model, the corresponding feature element can be quickly located by searching the index. This efficient retrieval mechanism significantly reduces feature processing time and improves the efficiency of the entire 3D model generation process. The corresponding embedding vector is retrieved from a preset embedding matrix using the index, mapping the feature elements from discrete symbolic representation to a continuous vector space. This semantic mapping captures the underlying semantic relationships between feature elements, enabling subsequent computations to better understand the meaning of the features. An embedding vector sequence is constructed based on the retrieved embedding vectors, providing an ordered data structure for subsequent feature processing. This serialized representation facilitates feature analysis and manipulation using various sequence processing algorithms, such as the attention mechanism. Computing a correlation score between each feature element and other feature elements captures the inherent connections between them. An attention weight matrix is ​​constructed based on the relevance scores, and a weighted summation of the embedding vector sequence is performed based on this matrix to obtain a contextual representation of each feature element. This dynamic feature weighting method can highlight important feature elements and suppress unimportant feature elements, allowing the model to focus more on features that have a key impact on 3D model generation, thereby improving the accuracy of the generated target 3D model. Using graph embedding technology to integrate contextual representation into the generative adversarial network can better utilize the structural information of the 3D model. Graph embedding technology can represent the topological structure of the 3D model as a graph structure and embed feature information into the nodes of the graph, allowing the generative adversarial network to learn the structural features of the 3D model and the relationships between features. By combining graph embedding technology with the generative adversarial network, the generative adversarial network can more accurately learn the distribution characteristics of the 3D model, thereby generating a more realistic and accurate target 3D model.

[0075] Optionally, generating a target three-dimensional model by using the generative adversarial network includes:

[0076] Generate a sample model through the generator of the generative adversarial network, and input the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network;

[0077] Calculating the loss functions of the generator and the discriminator based on the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, updating the parameters of the generator and the discriminator through a back-propagation algorithm, and performing a new round of iteration using the updated generator and the discriminator;

[0078] When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate 3D model;

[0079] According to the curvature of the surface triangular facets of the intermediate three-dimensional model, redundant triangular facets in the intermediate three-dimensional model are removed, and the topological structural features in the intermediate three-dimensional model are identified using topological principles to determine the location and type of defects. The detected defects are repaired using a repair algorithm based on topological transformation to obtain the target three-dimensional model.

[0080] The generator in a generative adversarial network is a neural network model. Its role is to generate sample models similar to real data (here, 3D models) based on input random noise (or other latent variables). Through continuous learning and optimization, the generator attempts to generate increasingly realistic 3D models to deceive the discriminator. The discriminator, also a neural network model, is tasked with distinguishing whether the input model is a sample model generated by the generator or a true preliminary 3D model. The discriminator analyzes and judges the input model, outputting a probability value indicating the likelihood that the model is the true model. By inputting both the sample model and the preliminary 3D model into the discriminator, the discriminator generates its respective judgments. Based on the discriminator's judgments on the sample model and the preliminary 3D model, the loss function for the generator and discriminator is calculated. The loss function measures the performance of the generator and discriminator. The generator's goal is to generate sample models as realistic as possible, making them difficult for the discriminator to distinguish. The discriminator's goal, on the other hand, is to accurately determine the source of the input model. The loss function is calculated based on the difference between the discriminator's judgment and the true label (sample model is fake, preliminary 3D model is true). The backpropagation algorithm updates the parameters of the generator and discriminator based on the calculated loss function value. Backpropagation is an optimization algorithm used to train neural networks. It adjusts parameter values ​​based on the gradient of the loss function with respect to network parameters to reduce the loss function, thereby improving the performance of the generator and discriminator. After the parameter updates, the generator and discriminator undergo a new round of iterations, with the generator continuing to generate new sample models and the discriminator continuing to perform discrimination. The iteration process terminates when the generative adversarial network reaches a preset number of iterations or when performance converges. The preset number of iterations is determined based on actual needs and computing resources. Performance convergence generally means that the loss function values ​​of the generator and discriminator no longer change significantly, or the discriminator's accuracy reaches a certain threshold. After the iteration terminates, the generator generates an intermediate 3D model. This intermediate 3D model is a relatively realistic 3D model generated by the generator after multiple iterations of optimization, but it may still contain some defects or areas that require further optimization. Redundant triangles in the intermediate 3D model are removed based on the curvature of the surface triangles. Curvature reflects the degree of curvature of the surface on which the triangular facets are located. Triangular facets with less curvature may contribute less to the shape and structure of the model and can be considered redundant. Removing redundant facets can reduce the amount of data in the model and improve the storage and computational efficiency of the model while maintaining the model's primary shape characteristics. Topological structural features in the intermediate 3D model are identified using topological principles. Topology focuses on the invariant properties of an object under continuous deformation. By analyzing the topological structure of the model, the location and type of possible defects in the model can be determined. For example, holes and cracks in the topological structure can be identified as defects. Detected defects are repaired using a repair algorithm based on topological transformation.Topological transformations can alter a model's topological structure while preserving its overall shape and features. Repair algorithms based on topological transformations can perform corresponding topological operations on the model, such as filling holes and repairing cracks, based on the type and location of defects, to produce a target 3D model. This target 3D model is a high-quality, optimized and repaired 3D model that more accurately reflects the actual shape and structure of the port's cargo stacks.

[0081] The generator generates sample models and inputs them into the discriminator along with a preliminary 3D model. The discriminator distinguishes between the sample models and the preliminary 3D model, providing feedback to the generator. Based on the discriminator's feedback, the generator continuously adjusts its parameters, making the generated sample models increasingly similar to the true 3D model (the preliminary 3D model can be considered a reference true model), thereby improving the realism and accuracy of the generated target 3D model. By calculating the loss function for the generator and discriminator and updating the parameters using the backpropagation algorithm, the performance of both can be continuously optimized. The design of the loss function ensures that the sample models generated by the generator more closely match the characteristics of the true 3D model in terms of geometry and texture, thereby improving the overall quality of the target 3D model. Through repeated rounds of iterative training, the generative adversarial network gradually approaches the optimal solution. Each iteration improves the performance of the generator and discriminator, and the generated intermediate 3D models increasingly resemble the ideal target 3D model. The generative adversarial network generates intermediate 3D models when it reaches a preset number of iterations or when its performance converges. This control mechanism prevents overtraining or undertraining. The preset number of iterations can be adjusted based on actual conditions, and the performance convergence judgment criteria can also be set according to specific needs, ensuring the effectiveness and controllability of the training process. Removing redundant triangular facets based on the curvature of the intermediate 3D model surface can simplify the model structure, reduce the model's storage space, and reduce computational complexity. The curvature reflects the degree of curvature of the facet. Removing redundant facets with small curvature that have little impact on the overall model shape can improve model efficiency while ensuring model accuracy. Using topological principles to identify topological structural features in the intermediate 3D model can accurately determine the location and type of defects. Topological principles focus on structural features such as connectivity and holes in the model. By analyzing these features, anomalies in the model, such as holes and cracks, can be detected. Using a repair algorithm based on topological transformation to repair detected defects, the repaired model maintains its original topological structure and geometry. Topological transformation can repair local defects without changing the overall topological properties of the model, thereby obtaining a high-quality target 3D model.

[0082] Optionally, constructing the density model of the target cargo pile according to the cargo characteristic data and the environmental factor data includes:

[0083] Obtaining data association between the cargo characteristic data and the environmental factor data through principal component analysis;

[0084] Constructing a first influence model of each factor in the cargo characteristic data on cargo density, and constructing a second influence model of each factor in the environmental factor data on cargo density;

[0085] The first influence model and the second influence model are integrated according to the data association to construct a density model.

[0086] Principal component analysis (PCA) is a commonly used data dimensionality reduction and feature extraction method. Its core concept is to transform raw data into a set of new orthogonal variables, known as principal components, through a linear transformation. These principal components are linear combinations of the original variables and are arranged in descending order of variance. The first principal component has the largest variance and best reflects the variation in the original data, while subsequent principal components decrease in order of variance. In port cargo storage scenarios, complex correlations may exist between cargo characteristic data (such as cargo source, transportation method, storage time, and stacking method) and environmental factor data (such as temperature, humidity, wind speed, air pressure, and light intensity). PCA can process this multidimensional data to identify the main trends and potential correlations within the data. For example, certain cargo sources may correspond to specific transportation methods, and environmental factors may affect them to varying degrees under different storage times. PCA can reveal these hidden correlation patterns. By extracting principal components, the original high-dimensional data can be transformed into a low-dimensional principal component space, reducing the data's dimensionality and complexity. This not only reduces the computational burden of subsequent model construction but also avoids overfitting caused by high data dimensionality, improving the model's generalization. The various factors in cargo characteristic data (cargo source, transportation method, storage time, and stacking method) influence cargo density through different mechanisms. For example, cargo from different sources may have varying densities due to factors such as raw materials and production processes. Bumps and squeezing during transportation can alter the stacking state of cargo, thereby affecting density distribution. Longer storage time can lead to changes in density due to moisture and deterioration. Different stacking methods can affect the void ratio and density distribution within the cargo pile. For each factor, a primary influence model on cargo density can be constructed. These models can utilize machine learning methods such as regression analysis, decision trees, and neural networks. For example, for the cargo source factor, density data for cargo from different sources can be collected and regression analysis can be used to establish a quantitative relationship between cargo source and density. For the storage time factor, time series analysis can be used to analyze density trends over storage time. Environmental factors (temperature, humidity, wind speed, air pressure, and light intensity) also influence cargo density. For example, rising temperatures can cause cargo to expand and decrease its density; increased humidity can cause cargo to absorb water, increasing its density; and factors such as wind speed, air pressure, and light intensity can also indirectly affect density by affecting the cargo's physical properties or chemical reactions. Similarly, a secondary impact model on cargo density is constructed for each environmental factor. These models can be tailored to the specific relationship between the environmental factor and density. For example, if the relationship between temperature and density exhibits a linear trend, a linear regression model can be used; if the relationship is more complex, nonlinear regression or neural network models can be considered. The data associations obtained through principal component analysis reflect the inherent connection between cargo characteristic data and environmental factor data.When building the density model, the first and second impact models are integrated based on these data associations, allowing for comprehensive consideration of the impact of various factors on cargo density. For example, if data associations indicate an interaction between cargo source and temperature, the impact of this interaction on density needs to be considered in the integrated model. The integrated density model can more comprehensively and accurately describe the relationship between cargo density and cargo characteristic data and environmental factor data. Compared to a single impact model, the integrated model can better capture the complexity and nonlinear relationships of the data, thereby improving the accuracy of density prediction. For example, in practical applications, the integrated model can more accurately calculate the density distribution of a target cargo pile based on given cargo characteristic data and environmental factor data, providing a reliable foundation for subsequent tasks such as mass calculation.

[0087] Cargo characteristic data (such as cargo origin, transportation method, storage time, and stacking method) and environmental factor data (such as temperature, humidity, wind speed, air pressure, and light intensity) typically contain multiple dimensions, potentially with complex correlations between them. Principal component analysis (PCA) can transform this multidimensional data into a small number of principal components, which retain the key information in the original data while reducing its dimensionality. This makes subsequent analysis of data associations simpler and more efficient, reducing computational complexity. Separately constructing a primary impact model for each factor in the cargo characteristic data on cargo density and a secondary impact model for each factor in the environmental factor data can accurately quantify the impact of each factor on cargo density. Separately modeling cargo characteristic data and environmental factor data allows for more detailed analysis of different data types. Cargo characteristic data and environmental factor data have distinct characteristics and patterns, so separate modeling better captures the unique characteristics of each data type, improving model accuracy and adaptability. Constructing separate impact models facilitates subsequent model adjustments and optimization. If a factor's impact on cargo density is found to be inconsistent with expectations, the impact model corresponding to that factor can be adjusted independently without affecting the impact models of other factors, improving model maintainability. By integrating the first and second impact models based on data association, the resulting density model comprehensively considers the impact of cargo characteristic data and environmental factor data on cargo density. This integrated model more comprehensively reflects actual conditions, avoids the limitations of single-factor models, and improves the accuracy and reliability of the density model.

[0088] Optionally, the integrating the first influence model and the second influence model according to the data association to construct a density model includes:

[0089] Assigning weights to the various factors in the cargo characteristic data and the environmental factor data based on the results of the principal component analysis;

[0090] Determining a first contribution value of each factor in the cargo characteristic data to cargo density based on the first influencing model, and determining a second contribution value of each factor in the environmental factor data to cargo density based on the second influencing model;

[0091] The cargo characteristic data and the environmental factor data are weightedly summed according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value to construct a density model.

[0092] Principal component analysis (PCA) is a dimensionality reduction technique that transforms multiple correlated variables into a small number of uncorrelated principal components by performing a linear transformation on the original data. During this process, each principal component contains a portion of the original data, and the amount of information contained (the variance contribution) varies from principal component to principal component. Cargo characteristic data (such as cargo type, packaging method, storage duration) and environmental factor data (such as temperature, humidity, and air pressure) each have varying degrees of influence on cargo density. PCA calculates the loading of each factor in each principal component (i.e., the correlation coefficient between the factor and the principal component) and the variance contribution of each principal component. Based on this information, a weight is assigned to each factor. Generally speaking, factors with larger loadings and higher variance contributions are assigned larger weights, as these factors have a more significant impact on cargo density. For example, if temperature has a large loading in a principal component and a high variance contribution, then the temperature factor will be assigned a higher weight, indicating a greater impact on cargo density. A first-impact model is constructed for each factor in the cargo characteristic data to quantify its impact on cargo density. For each factor in the cargo characteristic data, the first impact model calculates the first contribution value of that factor to cargo density. This contribution value reflects the specific impact of that factor on cargo density, considering only cargo characteristic factors. For example, for the cargo type factor, the first impact model might calculate the contribution of each cargo type to the overall cargo density based on the physical properties of each cargo type (such as density and porosity). Similarly, a second impact model is constructed for each factor in the environmental factor data to quantify the impact of each environmental factor on cargo density. For each factor in the environmental factor data, the second impact model calculates the second contribution value of that factor to cargo density. This contribution value reflects the specific impact of that factor on cargo density, considering only environmental factors. For example, for the humidity factor, the second impact model might calculate the contribution value of humidity to cargo density based on the impact of humidity changes on cargo water absorption and expansion. After determining the weights of each factor, the first and second contribution values, a weighted summation method is used to calculate the overall density prediction. For each factor in the cargo characteristic data, its first contribution value is multiplied by its corresponding weight. For each factor in the environmental factor data, its second contribution value is multiplied by its corresponding weight. The weighted contributions of all factors are then added together to produce a comprehensive density prediction. This prediction comprehensively considers the impact of both cargo characteristic and environmental factors on cargo density. A density model is constructed using the comprehensive density prediction as output and the cargo characteristic and environmental factor data as input.The model can be represented as a mathematical function or algorithm. When new cargo characteristic data and environmental factor data are input, the model can quickly calculate the corresponding cargo density prediction value based on pre-calculated weights and influence models. For example, in port cargo management, when new cargo arrives, the characteristic data and current environmental factor data are input, and the density model can predict the density of the cargo pile, providing a basis for cargo quality calculation and management.

[0093] Assigning weights to each factor in cargo characteristic data and environmental factor data based on principal component analysis results quantifies the relative importance of each factor's impact on cargo density. Principal component analysis reveals the inherent connections and structure within the data, and the weights determined through this analysis reflect the role and significance of each factor within the overall data. The impact of each factor on cargo density may vary across ports, cargo types, and storage conditions. Reasonable weight assignment makes the density model more realistic, enhancing its relevance and applicability. Weight assignment provides a clear framework for subsequent model construction. By clearly assigning weights to each factor, the model integrates cargo characteristic data and environmental factor data more systematically, avoiding the chaotic interaction of factors within the model and improving model stability and interpretability. Determining the primary contribution of each factor in the cargo characteristic data to cargo density based on the primary impact model and the secondary contribution of each factor in the environmental factor data based on the secondary impact model allows for a precise assessment of the specific impact of each factor on cargo density. This contribution value, based on the calculation results of each impact model, is highly accurate and reliable. Determining the contribution value provides a clear basis for model debugging and optimization. If the contribution value of a factor is found to be inconsistent with expectations, the impact model corresponding to that factor can be adjusted, or the weight distribution of that factor can be re-examined, thereby gradually optimizing the model's performance. Clarifying the contribution value of each factor makes the density model more interpretable. In practical applications, managers can understand which factors have a greater impact on cargo density based on the contribution value of each factor, and thus take targeted measures to control cargo density and improve the management efficiency of port cargo piles. A weighted summation of cargo characteristic data and environmental factor data based on the weight, first contribution value, and second contribution value is performed to obtain a comprehensive density prediction value, which can comprehensively consider the impact of multiple factors on cargo density. This comprehensive prediction method avoids the limitations of single-factor models, allowing the density model to more comprehensively and accurately reflect the actual situation.

[0094] Optionally, calculating the mass data of the target cargo pile according to the target three-dimensional model and the density model includes:

[0095] The target three-dimensional model is divided into a plurality of cubic units, a target volume of each cubic unit is determined, a predicted density of each cubic unit is determined according to the density model, a target mass is obtained according to the target volume and the predicted density, and a plurality of target masses are added together to obtain the total mass of the target cargo pile.

[0096] The target 3D model is an accurate representation of the geometric shape of the target cargo pile in 3D space. Since cargo piles are often irregular in shape, directly calculating their mass is complex. By dividing the pile into multiple cubic units, the complex geometric shape problem can be transformed into a problem of processing simple geometric bodies (cubes), greatly simplifying the calculation process. In the fields of computer graphics and 3D modeling, there are various algorithms for meshing 3D models, such as the octree algorithm. The octree algorithm recursively divides the 3D space into eight subspaces of equal volume until the geometric shape within each subspace is simple enough (e.g., approximately a cube), thereby completing the division of the target 3D model. This division method facilitates the subsequent independent mass calculation of each unit and allows the size and number of cubic units to be flexibly adjusted according to the complexity of the cargo pile and the required calculation accuracy. The volume of the cubic unit is one of the important parameters for its mass calculation. After the cubic units are divided, their volume can be calculated using a simple formula based on the geometric properties of the cube. For a cubic unit with a side length of a, its volume V = a 3In practical applications, while the divided cubic units may not be strictly standard cubes, their volumes can be determined through approximate calculations or numerical integration. For example, for some irregular approximate cubic units, they can be divided into multiple small standard cubes or their volumes can be estimated using other numerical methods. Accurately determining the volume of each cubic unit is a key step in ensuring the accuracy of mass calculations. The density model describes the relationship between cargo density and cargo characteristic data and environmental factor data. Each cubic unit can be considered a local area in the cargo pile. The cargo density in this area may be affected by various factors, such as cargo type, storage time, and ambient temperature and humidity. The density model can be used to predict the cargo density of each cubic unit based on the relevant data corresponding to that cubic unit. First, the location of each cubic unit in the target three-dimensional model and the corresponding cargo characteristic data and environmental factor data must be determined. For example, spatial interpolation can be used to estimate the environmental factor data at that cubic unit based on data from surrounding known points. This data is then input into the density model to obtain the predicted density ρ for each cubic unit. The accuracy of the predicted density directly affects the accuracy of the final mass calculation, so the construction and parameterization of the density model are crucial. According to the mass calculation formula m=ρV in physics (where m is mass, ρ is density, and V is volume), for each cubic unit, if its target volume V and predicted density ρ are known, the target mass m of the unit can be calculated. Assume that the volume of a cubic unit V=1m 3 , predicted density ρ = 800 kg / m 3 , then the target mass of this unit is m = ρV = 800 × 1 = 800 kg. This step combines geometric and density information to determine the mass of each local area, laying the foundation for the subsequent calculation of the total mass of the cargo pile. The target cargo pile is composed of multiple cubic units, each with its own target mass. By summing the target masses of all cubic units, we can obtain the total mass of the target cargo pile.

[0097] Dividing the target 3D model into multiple cubic units transforms the previously complex 3D model calculation problem into a computational problem involving multiple simple cubic units. Cubic units have regular geometric shapes, making volume calculation relatively simple. This significantly reduces the complexity of calculating the target cargo pile's mass and improves computational efficiency. By appropriately sizing the cubic units, computational accuracy can be controlled. Smaller cubic units more precisely fit the target 3D model's shape, resulting in more accurate mass calculations. Larger cubic units, while faster, may sacrifice accuracy. In practical applications, the cubic unit size can be flexibly adjusted to meet specific requirements, achieving a balance between computational efficiency and accuracy. The divided cubic units are independent of each other, facilitating parallel computation. Utilizing multi-core processors or distributed computing systems, simultaneous mass calculations for multiple cubic units can significantly reduce computation time, making it suitable for mass calculations of large-scale target cargo piles. Determining the target volume for each cubic unit accurately quantifies the space occupied by the target cargo pile at different locations. This is crucial for subsequent mass calculations, as mass is closely related to volume and density. Accurate volume calculations can avoid mass calculation errors caused by inaccurate volume estimates. Even if the target 3D model has a complex shape, this complexity can be effectively handled by dividing it into cubic units and determining the volume of each unit. Cubic units can fill various irregular areas of the target 3D model, enabling accurate calculation of the volume of the entire cargo pile. The target volume is a key parameter for calculating the target mass. Only by accurately determining the volume of each cubic unit can the target mass for each unit be calculated in conjunction with the density model, ultimately yielding the total mass of the target cargo pile. The density model integrates the impact of cargo characteristic data and environmental factors on cargo density, providing a more comprehensive and accurate reflection of density conditions at different locations within the target cargo pile. Determining the predicted density of each cubic unit using the density model takes into account the impact of multiple factors on density, ensuring that the mass calculation results are more realistic. In port environments, cargo characteristic data and environmental factor data may change over time. The density model can be updated in real time based on new data, dynamically adjusting the predicted density of each cubic unit. This allows the mass calculation to adapt to varying storage conditions and cargo conditions, improving the model's practicality and reliability. Accurate density prediction is crucial for mass calculation. By comprehensively considering various factors, the density model provides more precise density predictions, thereby improving the accuracy of target mass calculations. Accurate data on the target cargo pile's total mass is crucial for port operational decision-making. For example, accurate mass data is essential for cargo loading and unloading, calculating storage costs, and scheduling transportation plans. The mass data calculated using this method can provide reliable support for these decisions, improving port operational efficiency and economic efficiency.

[0098] This embodiment also discloses a 3D yard measurement system based on drones. Figure 2 This is a module diagram of a 3D yard measurement system based on a drone disclosed in an embodiment of the present application, such as Figure 2 As shown, the system includes an acquisition module 201, a model module 202, a density module 203, and a calculation module 204, wherein:

[0099] an acquisition module 201 configured to control the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and point cloud data to generate a fused data set;

[0100] Model module 202 is configured to input the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model;

[0101] Density module 203 is configured to obtain cargo characteristic data and environmental factor data of the target cargo pile, and construct a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity;

[0102] The calculation module 204 is configured to calculate mass data of the target cargo pile according to the target three-dimensional model and the density model.

[0103] Optionally, the acquisition module 201 is configured to:

[0104] Extracting image features of the measurement image, wherein the image features include edges, corners, and textures; and extracting point cloud features of the point cloud data, wherein the point cloud features include curvature, normal vector, and local density;

[0105] Calculating the Euclidean distance between the image feature and the point cloud feature, and determining a feature point pair whose Euclidean distance is less than a preset threshold as a matching pair, wherein the feature point pair includes an image feature and a point cloud feature;

[0106] calculating a registration transformation matrix between the measurement image and the point cloud data based on the matching pairs, obtaining an optimal solution of the registration transformation matrix by a least squares method, and aligning the measurement image and the point cloud data based on the optimal solution;

[0107] The aligned measurement image and the point cloud data are fused.

[0108] Optionally, the model module 202 is configured to:

[0109] Constructing a 3D model feature table to encode feature elements of the 3D model, wherein the feature elements include geometric features and topological structures, and each feature element has a unique index;

[0110] When the preliminary three-dimensional model is received, searching for an index of a characteristic element of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving a corresponding embedding vector from a preset embedding matrix using the index, and constructing an embedding vector sequence based on the embedding vector;

[0111] For the embedding vector sequence, calculating a correlation score between each feature element and other feature elements, constructing an attention weight matrix based on the correlation score, and performing a weighted summation on the embedding vector sequence according to the attention weight matrix to obtain a contextual representation of each feature element;

[0112] The context representation is integrated into the generative adversarial network using graph embedding technology, and a target three-dimensional model is generated through the generative adversarial network.

[0113] Optionally, the model module 202 is configured to:

[0114] Generate a sample model through the generator of the generative adversarial network, and input the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network;

[0115] Calculating the loss functions of the generator and the discriminator based on the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, updating the parameters of the generator and the discriminator through a back-propagation algorithm, and performing a new round of iteration using the updated generator and the discriminator;

[0116] When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate 3D model;

[0117] According to the curvature of the surface triangular facets of the intermediate three-dimensional model, redundant triangular facets in the intermediate three-dimensional model are removed, and the topological structural features in the intermediate three-dimensional model are identified using topological principles to determine the location and type of defects. The detected defects are repaired using a repair algorithm based on topological transformation to obtain the target three-dimensional model.

[0118] Optionally, the density module 203 is configured to:

[0119] Obtaining data association between the cargo characteristic data and the environmental factor data through principal component analysis;

[0120] Constructing a first influence model of each factor in the cargo characteristic data on cargo density, and constructing a second influence model of each factor in the environmental factor data on cargo density;

[0121] The first influence model and the second influence model are integrated according to the data association to construct a density model.

[0122] Optionally, the density module 203 is configured to:

[0123] Assigning weights to the various factors in the cargo characteristic data and the environmental factor data based on the results of the principal component analysis;

[0124] Determining a first contribution value of each factor in the cargo characteristic data to cargo density based on the first influencing model, and determining a second contribution value of each factor in the environmental factor data to cargo density based on the second influencing model;

[0125] The cargo characteristic data and the environmental factor data are weightedly summed according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value to construct a density model.

[0126] Optionally, the calculation module 204 is configured to:

[0127] The target three-dimensional model is divided into a plurality of cubic units, a target volume of each cubic unit is determined, a predicted density of each cubic unit is determined according to the density model, a target mass is obtained according to the target volume and the predicted density, and a plurality of target masses are added together to obtain the total mass of the target cargo pile.

[0128] It should be noted that the above embodiments provide devices that implement their functions using only the division of the above functional modules as examples. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0129] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0130] The communication bus 302 is used to implement the connection and communication between these components.

[0131] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0132] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0133] The processor 301 may include one or more processing cores. Using various interfaces and circuits, the processor 301 connects to various components within the server. It executes instructions, programs, code sets, or instruction sets stored in the memory 305, as well as accesses data stored in the memory 305, to perform various server functions and process data. Optionally, the processor 301 may be implemented using at least one of the following hardware forms: a digital signal processing (DSP), a field-programmable gate array (FPGA), or a programmable logic array (PLA). The processor 301 may integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU primarily processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing content displayed on the display screen; and the modem handles wireless communications. It is understood that the modem may not be integrated into the processor 301 but implemented as a separate chip.

[0134] Among them, the memory 305 may include a random access memory (RAM) or a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may also be optionally at least one storage device located away from the aforementioned processor 301. As Figure 3As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program for a 3D yard measurement method based on a drone.

[0135] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call an application program stored in the memory 305 for a drone-based 3D yard measurement method. When executed by one or more processors 301, the electronic device executes one or more methods in the above embodiments.

[0136] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for this application.

[0137] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic, such as the division of units, which is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0139] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0140] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0141] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory 305 and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of this application. The aforementioned memory 305 includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a magnetic disk, or an optical disk.

[0142] The above is merely an exemplary embodiment of the present disclosure and is not intended to limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not described in the present disclosure.

Claims

1. A 3D yard measurement method based on drone, characterized in that: Applied to a port management platform, the method includes: controlling the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, registering the measurement images and the point cloud data, and fusing the registered measurement images and point cloud data to generate a fused data set; Inputting the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; Obtaining cargo characteristic data and environmental factor data of the target cargo pile, and constructing a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; Calculate the mass data of the target cargo pile according to the target three-dimensional model and the density model, Inputting the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model includes: Constructing a 3D model feature table to encode feature elements of the 3D model, wherein the feature elements include geometric features and topological structures, and each feature element has a unique index; When the preliminary three-dimensional model is received, searching for an index of a characteristic element of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving a corresponding embedding vector from a preset embedding matrix using the index, and constructing an embedding vector sequence based on the embedding vector; For the embedding vector sequence, calculating a correlation score between each feature element and other feature elements, constructing an attention weight matrix based on the correlation score, and performing a weighted summation on the embedding vector sequence according to the attention weight matrix to obtain a contextual representation of each feature element; Using graph embedding technology to integrate the context representation into the generative adversarial network, and generating a target three-dimensional model through the generative adversarial network, Generating a target three-dimensional model by using the generative adversarial network includes: Generate a sample model through the generator of the generative adversarial network, and input the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network; Calculating the loss functions of the generator and the discriminator based on the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, updating the parameters of the generator and the discriminator through a back-propagation algorithm, and performing a new round of iteration using the updated generator and the discriminator; When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate 3D model; According to the curvature of the surface triangular facets of the intermediate three-dimensional model, redundant triangular facets in the intermediate three-dimensional model are removed, and the topological structural features in the intermediate three-dimensional model are identified using topological principles to determine the location and type of defects. The detected defects are repaired using a repair algorithm based on topological transformation to obtain the target three-dimensional model.

2. The 3D storage yard measurement method based on drone according to claim 1 is characterized in that: The registering the measurement image and the point cloud data, and fusing the registered measurement image and the point cloud data to generate a fused data set includes: Extracting image features of the measurement image, wherein the image features include edges, corners, and textures; and extracting point cloud features of the point cloud data, wherein the point cloud features include curvature, normal vector, and local density; Calculating the Euclidean distance between the image feature and the point cloud feature, and determining a feature point pair whose Euclidean distance is less than a preset threshold as a matching pair, wherein the feature point pair includes an image feature and a point cloud feature; calculating a registration transformation matrix between the measurement image and the point cloud data based on the matching pairs, obtaining an optimal solution of the registration transformation matrix by a least squares method, and aligning the measurement image and the point cloud data based on the optimal solution; The aligned measurement image and the point cloud data are fused.

3. The 3D storage yard measurement method based on drone according to claim 1 is characterized in that: The step of constructing the density model of the target cargo pile according to the cargo characteristic data and the environmental factor data includes: Obtaining data association between the cargo characteristic data and the environmental factor data through principal component analysis; Constructing a first influence model of each factor in the cargo characteristic data on cargo density, and constructing a second influence model of each factor in the environmental factor data on cargo density; The first influence model and the second influence model are integrated according to the data association to construct a density model.

4. The 3D storage yard measurement method based on drone according to claim 3 is characterized in that: The integrating the first influence model and the second influence model according to the data association to construct a density model includes: Assigning weights to the various factors in the cargo characteristic data and the environmental factor data based on the results of the principal component analysis; Determining a first contribution value of each factor in the cargo characteristic data to cargo density based on the first influencing model, and determining a second contribution value of each factor in the environmental factor data to cargo density based on the second influencing model; The cargo characteristic data and the environmental factor data are weightedly summed according to the weight, the first contribution value, and the second contribution value to obtain a comprehensive density prediction value to construct a density model.

5. The 3D storage yard measurement method based on drone according to claim 1, characterized in that: Calculating the mass data of the target cargo pile according to the target three-dimensional model and the density model includes: The target three-dimensional model is divided into a plurality of cubic units, a target volume of each cubic unit is determined, a predicted density of each cubic unit is determined according to the density model, a target mass is obtained according to the target volume and the predicted density, and a plurality of target masses are added together to obtain the total mass of the target cargo pile.

6. A 3D yard measurement system based on drone, characterized in that: It includes acquisition module, model module, density module and calculation module, among which: an acquisition module configured to control the UAV to fly along a preset flight path to collect measurement images and point cloud data of each target cargo pile in the port, register the measurement images and the point cloud data, and fuse the registered measurement images and point cloud data to generate a fused data set; a model module configured to input the data set into a preset deep convolutional neural network to generate a preliminary three-dimensional model of the target cargo pile, and input the preliminary three-dimensional model into a generative adversarial network to generate a target three-dimensional model; a density module configured to obtain cargo characteristic data and environmental factor data of the target cargo pile, and construct a density model of the target cargo pile based on the cargo characteristic data and the environmental factor data, wherein the cargo characteristic data includes cargo source, transportation method, storage time, and stacking method, and the environmental factor data includes temperature, humidity, wind speed, air pressure, and light intensity; a calculation module configured to calculate mass data of the target cargo pile based on the target three-dimensional model and the density model, The model module is configured to: Constructing a 3D model feature table to encode feature elements of the 3D model, wherein the feature elements include geometric features and topological structures, and each feature element has a unique index; When the preliminary three-dimensional model is received, searching for an index of a characteristic element of the preliminary three-dimensional model in the three-dimensional model feature table, retrieving a corresponding embedding vector from a preset embedding matrix using the index, and constructing an embedding vector sequence based on the embedding vector; For the embedding vector sequence, calculating a correlation score between each feature element and other feature elements, constructing an attention weight matrix based on the correlation score, and performing a weighted summation on the embedding vector sequence according to the attention weight matrix to obtain a contextual representation of each feature element; Using graph embedding technology to integrate the context representation into the generative adversarial network, and generating a target three-dimensional model through the generative adversarial network, The model module is further configured to: Generate a sample model through the generator of the generative adversarial network, and input the sample model and the preliminary three-dimensional model into the discriminator of the generative adversarial network; Calculating the loss functions of the generator and the discriminator based on the discrimination results of the discriminator on the sample model and the preliminary three-dimensional model, updating the parameters of the generator and the discriminator through a back-propagation algorithm, and performing a new round of iteration using the updated generator and the discriminator; When the generative adversarial network reaches a preset number of iterations or performance convergence, the generator generates an intermediate 3D model; According to the curvature of the surface triangular facets of the intermediate three-dimensional model, redundant triangular facets in the intermediate three-dimensional model are removed, and the topological structural features in the intermediate three-dimensional model are identified using topological principles to determine the location and type of defects. The detected defects are repaired using a repair algorithm based on topological transformation to obtain the target three-dimensional model.

7. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 5 is executed.

Images