Carriage cargo volume measuring and calculating method and device integrating point cloud semantic segmentation and grid reconstruction

Abstract

The invention discloses a carriage cargo volume measuring and calculating method and device fusing point cloud semantic segmentation and grid reconstruction. The method comprises the steps that S01, multi-frame original point cloud data, collected by a laser radar, of a measured vehicle are acquired and preprocessed; s02, performing three-dimensional reconstruction on the preprocessed point cloud data to obtain a three-dimensional point cloud model of the tested vehicle; s03, inputting the three-dimensional point cloud model into an improved semantic segmentation network for point cloud semantic segmentation, segmenting cargo surface point cloud and carriage structure point cloud, and calculating attention weights of local neighborhoods by the improved semantic segmentation network by using a spatial distance and a feature distance; and S04, performing three-dimensional grid reconstruction according to the cargo surface point cloud and the carriage structure point cloud obtained by segmentation, and calculating the volume of the cargo in the carriage based on a projection infinitesimal integral method. The method has the advantages of being simple in implementation method, low in cost, high in measurement and calculation efficiency and precision, high in anti-interference performance and adaptability and the like.

Description

Technical Field

[0001] This invention relates to the field of logistics loading and measurement technology, and in particular to a method and device for calculating the volume of cargo in a wagon by integrating point cloud semantic segmentation and grid reconstruction. Background Technology

[0002] In large-scale modern mines, coal washing plants, and logistics parks, the consumption of bulk materials such as sand, gravel, coal, and mineral powder is enormous. Therefore, the loading, measurement, and settlement of bulk materials are crucial links in the production process. Traditional bulk material loading and measurement mainly rely on manual operation and weighbridges. However, manual loading and verification methods heavily depend on operator experience, are labor-intensive and subjective, and are prone to uneven loading, off-center loading, or even spillage accidents due to fatigue or negligence. Furthermore, repeated reversing and leveling operations lead to low loading efficiency, causing vehicle queues and road congestion. Although weighbridges (truck scales) can provide accurate weight data, in many specific scenarios, "volume" data is more critical than "weight" data. For example, for materials with large density variations (such as sand and gravel with different moisture contents), weight alone cannot reflect the actual volume. In addition, weighbridges cannot detect the spatial distribution of goods, making it difficult to identify safety hazards caused by uneven loading. Therefore, achieving non-contact, automated, and accurate measurement of cargo volume has become an urgent need in the field of bulk material transportation.

[0003] Currently, 3D spatial information detection technologies are mainly divided into two categories: vision-based image processing technologies and laser scanning-based point cloud processing technologies. Vision-based image processing technologies typically employ image measurement schemes based on binocular vision or multi-view geometry. While these are relatively inexpensive, they are extremely sensitive to ambient lighting. Environments such as mines and ports often operate around the clock, and strong outdoor light, shadows, low nighttime illumination, and pervasive dust can severely interfere with the imaging quality of visual sensors, making it difficult to guarantee the accuracy of 3D reconstruction and failing to meet the requirements of high-precision industrial-grade measurements. In contrast, 3D LiDAR, as an active detection technology, possesses unique advantages such as millimeter-level resolution, strong anti-interference capabilities, and no lighting limitations.

[0004] However, existing solutions for volume measurement using lidar have the following problems: 1. Static measurement is inefficient. Currently, high-precision systems require vehicles to come to a complete stop before scanning, which is a "static measurement" method. This method is inconvenient to implement and inefficient, especially at checkpoints with high traffic volume. Stopping for scanning will severely reduce traffic efficiency and cannot meet the needs of high-frequency logistics turnover.

[0005] 2. The volume calculation model is coarse. Due to the uneven and irregular characteristics of the cargo surface, existing measurement methods often use simple regular geometric approximations or low-resolution voxel accumulation methods, which are difficult to accurately describe the minute undulations of the cargo surface, resulting in a large deviation in the final volume measurement.

[0006] 3. Difficulty in point cloud segmentation and extraction. Accurately calculating cargo volume requires precisely separating the "cargo point cloud" from the background. Existing technologies typically employ traditional geometric rules (such as defining a fixed rectangular region of interest (ROI)) or simple plane fitting to extract the cargo compartment. However, in real-world scenarios, there are numerous vehicle types, complex structures at the connection between the cab and the cargo compartment, and frequent interference from elements such as cab guardrails, fuel tanks, or tires. Traditional algorithms struggle to handle these complex structures, easily misidentifying vehicle components as cargo or cutting off edge cargo, leading to further computational errors. Summary of the Invention

[0007] The technical problem to be solved by the present invention is: in view of the above-mentioned problems existing in the prior art, the present invention provides a method and equipment for calculating the volume of cargo in a carriage by fusing point cloud semantic segmentation and mesh reconstruction, which is simple to implement, low in cost, efficient and accurate in calculation, and has strong anti-interference and adaptability.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows: A method for calculating the volume of cargo in a wagon by integrating point cloud semantic segmentation and mesh reconstruction, comprising the following steps: Step S01. Point cloud data acquisition: Acquire multiple frames of raw point cloud data of the vehicle under test collected by the lidar and preprocess them to obtain preprocessed point cloud data; Step S02. 3D Reconstruction: Perform 3D reconstruction on the preprocessed point cloud data to obtain a 3D point cloud model of the vehicle under test; Step S03. Point cloud semantic segmentation: The three-dimensional point cloud model is input into the improved semantic segmentation network for point cloud semantic segmentation to segment the cargo surface point cloud and the carriage structure point cloud. The improved semantic segmentation network is based on the semantic segmentation network. In the local feature aggregation stage, spatial distance and feature distance are used to calculate the attention weight of the local neighborhood. Step S04. Volume Calculation: Based on the point cloud of the cargo surface obtained by segmentation and the point cloud of the carriage structure, a three-dimensional mesh is reconstructed, and the volume of the cargo inside the carriage is calculated based on the projection infinitesimal integral method.

[0009] Further, in step S01, the preprocessing includes outlier filtering and / or ground point removal. The outlier filtering involves calculating the average distance from all points to their nearest neighbors and filtering out outliers based on the calculated average distance. The ground point removal involves calculating the distance from all points to the ground plane based on the ground plane model and filtering out ground points based on the calculated distance to the ground plane.

[0010] Furthermore, in step S02, the process of 3D reconstruction of the acquired raw point cloud data also includes motion compensation to correct pose deviations, including: Step S21. Coarse registration: Calculate and match the fast point feature histogram features of two adjacent frames of point cloud data to obtain an initial transformation matrix; Step S22. Fine registration: Based on the initial transformation matrix, solve for the optimal transformation matrix with the goal of minimizing the distance between two adjacent frames of point cloud data; Step S23. Coordinate transformation: Use the optimal transformation matrix to transform the point cloud data of all frames to the same coordinate system to obtain the reconstructed three-dimensional point cloud model of the vehicle under test.

[0011] Furthermore, in the improved semantic segmentation network, the spatial distance is calculated based on the spatial Euclidean distance between points. The characteristic distance is calculated based on the differences in reflection intensity and geometric features. Using spatial distance Distance from features Calculate the attention weights for the local neighborhood: , in, This represents the attention weight of the k-th point relative to the center point within the local neighborhood. This represents a multilayer perceptron. This represents the Softmax function. This indicates feature splicing.

[0012] Furthermore, the input feature vector of the improved semantic segmentation network includes at least the three-dimensional coordinates of each point in the point cloud, the lidar reflection intensity, and the covariance matrix feature values ​​calculated based on the neighborhood of each point. The covariance matrix feature values ​​include flatness features and / or verticality features.

[0013] Furthermore, the improved semantic segmentation network is trained using a hybrid loss function consisting of a weighted sum of Focal Loss and Dice Loss. The calculation expression for the hybrid loss function is as follows: ,in, Indicates Dice Loss, Indicates Focal Loss, express The weighting coefficients.

[0014] Furthermore, step S04 specifically includes: Step S41. Determining the reference plane: Based on the three-dimensional point cloud model of the empty vehicle, the reference plane at the bottom of the vehicle compartment is obtained by fitting. Step S42. Cargo surface meshing: Project the segmented cargo surface point cloud onto a horizontal plane to construct a two-dimensional triangular mesh, and map the constructed triangular mesh back into three-dimensional space to form a triangular mesh model of the cargo upper surface; Step S43. Volume Calculation: Project each triangular facet in the triangular mesh model on the upper surface of the cargo onto the reference plane at the bottom of the carriage to form a triangular prism micro-element. Combine the volumes of all the triangular prism micro-elements to obtain the total volume of the cargo inside the carriage.

[0015] Furthermore, in step S43, the total volume of goods inside the carriage... The calculation expression is:

[0016]

[0017]

[0018] in, For the volume of a single triangular prism, M This refers to the number of triangular facets in the grid on the upper surface of the cargo. For triangular facets in Projected area on a plane This is the average height of the three vertices of the triangular facet from the reference plane.

[0019] A computer device includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method described above.

[0020] A computer-readable storage medium storing a computer program that, when executed, implements the method described above.

[0021] Compared with the prior art, the advantages of the present invention are as follows: 1. This invention utilizes a semantic segmentation network based on deep learning to perform semantic segmentation on the point cloud of the tested vehicle, achieving pixel-level automated classification of the cargo plane and cargo plane. Compared with the traditional region of interest extraction method based on fixed geometric bounding boxes, it can adapt to different vehicle models and accurately remove interference information such as the cab, tires, and guardrails in complex environments. At the same time, by combining mesh reconstruction and projection infinitesimal integration methods, and using fine mesh for volume integration, it can perform fine modeling and calculation of bulk cargo with uneven surfaces. It can achieve high-precision and automated volume measurement while the vehicle is in motion, improve the level of intelligence in bulk material transportation, and meet the needs of high-throughput operations in mines, ports, etc.

[0022] 2. This invention employs an improved semantic segmentation network and introduces a dual-distance attention pooling mechanism. During the local feature aggregation stage of the network, the attention weights of the local neighborhood are calculated based on the spatial distance and feature distance between points. This enables "homogeneous feature aggregation and heterogeneous feature suppression," generating sharp and clear segmentation boundaries. It effectively solves the problem of misjudging the "adhesion" between the inner wall of the carriage and the contact surface of the goods, avoiding misjudging the inner wall of the carriage as goods, ensuring the accuracy of the volume calculation boundary, and thus ensuring the accuracy of the measurement.

[0023] 3. In the process of 3D reconstruction of the acquired original point cloud data, the present invention also includes a motion compensation strategy of "coarse registration + fine registration", which can effectively correct the point cloud motion distortion caused by the vehicle during non-uniform speed driving and build a high-fidelity 3D point cloud model of the vehicle without stopping.

[0024] 4. This invention further constructs a "physical-geometric" multimodal input feature by fusing the reflection intensity, flatness, verticality and other geometric covariance features of the lidar, which is used as the input feature of the semantic segmentation network. This can effectively distinguish objects with similar geometric shapes but different materials, significantly improve the model's adaptability to different goods, and solve the problem that traditional networks are insufficient to perceive material differences.

[0025] 5. This invention further adopts Focal Loss training and Dice Loss joint training, which can solve the problem of missed detection caused by sparse point cloud at the edge of the carriage and the problem of extremely high accuracy of volume calculation at the boundary, making the calculated volume model more closed and realistic, thereby further improving the accuracy of volume measurement. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the implementation process of the method for calculating the volume of cargo in a wagon that integrates point cloud semantic segmentation and mesh reconstruction in this embodiment.

[0027] Figure 2 This is a schematic diagram illustrating the principle of segmentation and noise reduction in a specific application embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0029] This invention utilizes a semantic segmentation network based on deep learning to perform semantic segmentation on the point cloud of the tested vehicle, achieving pixel-level automated classification of the cargo plane and cargo plane. Compared with the traditional region of interest extraction method based on fixed geometric bounding boxes, it can adapt to different vehicle models and accurately remove interfering information such as the cab, tires, and guardrails in complex environments. At the same time, by combining mesh reconstruction and projection infinitesimal integration methods, and using fine mesh for volume integration, it can perform fine modeling and calculation of bulk cargo with uneven surfaces. It can achieve high-precision and automated volume measurement while the vehicle is in motion, improve the level of intelligence in bulk material transportation, and meet the needs of high-throughput operations in mines, ports, and other places.

[0030] Meanwhile, this invention employs an improved semantic segmentation network and introduces a dual-distance attention pooling mechanism. In the Local Feature Aggregation (LFA) stage of the network, the attention weight of the local neighborhood is calculated based on the spatial distance and feature distance between points. This allows the network to automatically assign a very low aggregation weight to the neighboring point when the center point is located on the carriage wall and the neighboring point is located on the cargo, even when the spatial distance is small and the feature distance is large. This achieves "homogeneous feature aggregation and heterogeneous feature suppression," thereby generating sharp and clear segmentation boundaries. This effectively solves the problem of misjudging the "adhesion" between the carriage wall and the cargo contact surface, avoids misjudging the carriage wall as cargo, ensures the accuracy of the volume calculation boundary, and thus ensures the accuracy of the measurement.

[0031] like Figure 1 As shown, the steps of the method for calculating the volume of cargo in a wagon that integrates point cloud semantic segmentation and mesh reconstruction in this embodiment include: Step S01. Point cloud data acquisition: Acquire multiple frames of raw point cloud data of the vehicle under test collected by the lidar and perform preprocessing to obtain preprocessed point cloud data.

[0032] In this embodiment, a multi-line lidar (such as a 96-line lidar) mounted on a gantry can be used to collect dynamic data. When a vehicle passes through the detection area, multiple frames of raw point cloud data are continuously collected. After acquiring multiple frames of raw point cloud data of the vehicle under test collected by the lidar, the collected point cloud is preprocessed. The preprocessing includes outlier filtering and ground point removal. Outlier filtering involves calculating the average distance from all points to their nearest neighbors and filtering out outliers based on the calculated average distance. Ground point removal involves calculating the distance from all points to the ground plane based on the ground plane model and filtering out ground points based on the calculated distance to the ground plane.

[0033] Specifically, the outlier filtering process employs a statistical filtering algorithm, applying it to any point in the point cloud. Calculate its to The average distance between the nearest neighbors Assume the distance follows a Gaussian distribution. ,like If so, it is considered an outlier and removed. (The threshold is the multiple of the standard deviation), which can remove dust and noise from the air.

[0034] Specifically, during the ground point removal (RANSAC) process, the Random Sample Consensus (RANSAC) algorithm can be used to fit the ground plane model. For example, the ground plane equation can be set as: (1) Where A, B, and C are the plane normal vector parameters, and D is a constant. The parameters are fitted iteratively using a random subset, and the distance from all points to the plane is calculated. : (2) Set threshold ,like If the data is not found in the data cloud, it will be classified as a ground point and removed, in order to retain the point cloud of the vehicle body and cargo. .

[0035] Step S02. 3D Reconstruction: Perform 3D reconstruction on the preprocessed point cloud data to obtain a 3D point cloud model of the vehicle under test.

[0036] While dynamic scanning via a gantry during vehicle movement can improve efficiency, the laser frames acquired at different times during vehicle movement will exhibit pose deviations. Furthermore, vehicle speed changes (non-uniform speed) and road surface disturbances often occur as the vehicle passes through the survey area, leading to stretching, compression, or distortion of the acquired point cloud data. This results in severe distortion of the point cloud obtained through dynamic scanning, directly causing inaccurate reconstruction of the vehicle body model and directly affecting volume calculation results. This embodiment incorporates motion compensation during the 3D reconstruction of the acquired raw point cloud data to correct pose deviations. This effectively corrects the motion distortion of the point cloud caused by the vehicle's non-uniform speed movement, enabling the construction of a high-fidelity 3D vehicle point cloud model without stopping the vehicle.

[0037] As an optional implementation method, motion compensation based on feature matching and generalized iterative nearest point (GICP) can be used to achieve 3D reconstruction. The specific steps include: Step S21. Coarse Registration (FPFH): Calculate the fast point feature histogram features of two adjacent frames of point cloud data and match them to obtain the initial transformation matrix.

[0038] Specifically, for two adjacent point cloud frames and Calculate the Fast Point Feature Histogram (FPFH) features and compute the initial transformation matrix through feature matching. Step S22. Fine Registration (GICP): Based on the initial transformation matrix, solve for the optimal transformation matrix with the goal of minimizing the distance between point clouds in two adjacent frames.

[0039] Specifically, based on the initial transformation matrix, the Generalized Iterative Closest Point (GICP) algorithm can be used for fine registration. GICP extends the point-to-point distance minimization of standard ICP to a probabilistic model, assuming that the point cloud follows a Gaussian distribution. and ,in , Let be the covariance matrix. The optimization objective is to minimize the Mahalanobis distance error function between two frames of point clouds. : (3) in, The transformation matrix to be solved (including rotations) Peaceful relocation ), This is the residual vector for the corresponding point pair. Step S23: Coordinate Transformation: Use the optimal transformation matrix to transform the point cloud data of all frames to the same coordinate system to obtain the reconstructed 3D point cloud model of the tested vehicle.

[0040] Specifically, the optimal solution is obtained through iteration. All frame point clouds can be transformed to the same coordinate system to obtain the reconstructed complete vehicle point cloud. .

[0041] This embodiment employs the above-mentioned "coarse registration + fine registration" strategy for motion compensation, and then stitches together multiple consecutive point clouds into a complete 3D vehicle model. This can accurately correct the point cloud motion distortion caused by the vehicle during non-uniform speed driving, thereby ensuring the construction of a high-fidelity 3D vehicle point cloud model without stopping.

[0042] Step S03. Point cloud semantic segmentation: Input the 3D point cloud model into the improved semantic segmentation network for point cloud semantic segmentation, segmenting the cargo surface point cloud and the carriage structure point cloud. The improved semantic segmentation network is based on semantic segmentation network. In the local feature aggregation stage, spatial distance and feature distance are used to calculate the attention weight of the local neighborhood.

[0043] In this embodiment, a deep learning network is used to achieve accurate semantic segmentation of the carriage and the cargo, so as to accurately divide the reconstructed point cloud into "cargo" and "carriage".

[0044] In this embodiment, an improved semantic segmentation network can be formed by improving RandLA-Net to address the characteristics of cargo volume measurement. First, to address the shortcomings of traditional networks in material differentiation, the input features of RandLA-Net are improved. Specifically, the input feature vector of the improved semantic segmentation network includes at least the three-dimensional coordinates of each point in the point cloud. The input features include LiDAR reflection intensity and covariance matrix eigenvalues ​​calculated from the neighborhood of each point. These covariance matrix eigenvalues ​​include flatness and verticality features, forming a "physical-geometric" multimodal input feature set. Compared to traditional methods that simply use position coordinates as input features, this embodiment effectively distinguishes objects with similar geometric shapes but different materials (such as flat sand and a truck bed) by fusing LiDAR reflection intensity with geometric covariance features like flatness and verticality. This significantly improves the model's adaptability to different goods and solves the problem that traditional networks are insufficient to perceive material differences.

[0045] Specifically, for each point in the point cloud Search for it In the neighborhood, calculate the eigenvalues ​​of the covariance matrix and construct the following feature channels: (1) Reflection intensity ( By utilizing the physical echo differences between different materials such as metal car bodies (high reflectivity) and coal / sand (low reflectivity / diffuse reflectivity), material sensing capabilities can be achieved, enabling the precise separation of cargo (low reflectivity / diffuse reflectivity) from complex vehicle bodies (high reflectivity).

[0046] (2) Smoothness ): It can be used to identify smooth artificial structures such as the side panels of a carriage.

[0047] (3) Verticality ): Calculate the angle between the normal vector and the horizontal plane, which can be used to identify the vertical sidewalls of the carriage.

[0048] Finally, a 6-dimensional feature vector is constructed. Input the network so that the model can perceive material differences at the input end.

[0049] To address the shortcomings of traditional networks in edge recognition, this embodiment improves the semantic segmentation network by calculating spatial distances based on the spatial Euclidean distances between points. The characteristic distance is calculated based on the differences in reflection intensity and geometric features. Using spatial distance Distance from features Calculate the attention weights for the local neighborhood: (4) in, This represents the attention weight of the k-th point relative to the center point within the local neighborhood. This represents a multilayer perceptron. This represents the Softmax function. This indicates feature splicing.

[0050] Specifically, this embodiment employs a "Dual-Distance Attentive Pooling" module. In the Local Feature Aggregation (LFA) stage of the network, a dual-distance mechanism is introduced. When calculating the attention weights of local neighborhoods, not only the spatial Euclidean distance between points is calculated, but also... It also simultaneously calculates the feature distance ( (Determined by the difference in reflection intensity and geometric features), when the center point is located on the carriage wall and the neighboring point is located on the cargo, although the spatial distance is small, the feature distance is large due to the difference in materials. The network will automatically assign a very low aggregation weight to the neighboring point, realizing "homogeneous feature aggregation and heterogeneous feature suppression", thereby generating a sharp and clear segmentation boundary and solving the problem of misjudgment of "adhesion" between the carriage wall and the cargo contact surface.

[0051] Furthermore, the improved semantic segmentation network in this embodiment is trained using a hybrid loss function consisting of a weighted sum of Focal Loss and Dice Loss. The calculation expression for the hybrid loss function is as follows: (5) in, Indicates Dice Loss, Indicates Focal Loss, express The weighting coefficients.

[0052] This embodiment utilizes the aforementioned hybrid loss functions. Focal Loss reduces the weight of easily classified samples (such as large, flat cargo areas), forcing the network to focus on learning difficult-to-classify samples (such as the corners of the carriage). Dice Loss directly optimizes the overlap between the predicted and real regions, significantly improving the closure of the segmented regions and reducing volume "missed calculations" caused by missing edges. Thus, by using Focal Loss training and Dice Loss joint training, the problem of missed detections caused by sparse point clouds at the carriage edges (class imbalance) and the problem of extremely high accuracy in volume calculation at boundaries can be solved. This makes the calculated volume model more closed and realistic, achieving preservation of small targets and edges, thereby further improving the accuracy of volume measurement.

[0053] After the above segmentation method, the final segmentation output is the point cloud segmentation based on the maximum probability value output by the network. Divide into four mutually exclusive sets: (Cargo surface point cloud): A collection of bulk material point clouds characterized by high roughness and low reflection intensity; (Point cloud of side wall and bottom of carriage): A collection of point clouds of the vehicle body structure with high flatness, high reflectivity and specific verticality characteristics; (Vehicle cab point cloud): Point cloud in the vehicle cab area (needs to be removed); (Other noise): Interfering point clouds such as suspended dust, tires, and fuel tanks (which need to be removed).

[0054] Finally, after segmentation and denoising, components that interfere with volume calculations, such as the front of the vehicle, fuel tank, and tires, can be automatically removed, leaving only the following components: and Used for subsequent mesh reconstruction and volume calculation, such as Figure 2 As stated above.

[0055] Step S04. Volume Calculation: Based on the point cloud of the cargo surface and the point cloud of the carriage structure obtained by segmentation, a three-dimensional mesh is reconstructed, and the volume of the cargo inside the carriage is calculated based on the projection infinitesimal integral method.

[0056] In this embodiment, high-precision volume calculation is achieved by combining mesh reconstruction and the infinitesimal integral method. Specifically, the steps include: Step S41. Determine the reference plane: Based on the three-dimensional point cloud model of the empty vehicle, the reference plane at the bottom of the carriage is obtained by fitting.

[0057] Specifically, firstly, empty and full-load registration is performed. The two models obtained from the 3D reconstruction of the point cloud, one empty and one full-load, are registered to obtain the translation and rotation matrix. Then, the semantically segmented carriage point cloud is processed. The empty model is fused with the point cloud of "carriage + cargo" using a translation and rotation matrix; then the empty model is based on... Fitting the plane equation of the bottom of the carriage As a reference plane.

[0058] Step S42. Cargo Surface Meshing: Project the segmented cargo surface point cloud onto the horizontal plane to construct a two-dimensional triangular mesh, and map the constructed triangular mesh back into three-dimensional space to form a triangular mesh model of the cargo's upper surface.

[0059] Specifically, the point cloud on the surface of the cargo Projected to In the plane, a two-dimensional triangulation mesh is constructed using the Delaunay triangulation algorithm and mapped back to three-dimensional space to form a closed mesh on the upper surface of the cargo. .

[0060] Step S43. Volume Calculation: Project each triangular facet in the triangular mesh model on the upper surface of the cargo onto the reference plane at the bottom of the carriage to form a triangular prism micro-element. Combine the volumes of all triangular prism micro-elements to obtain the total volume of the cargo inside the carriage.

[0061] Specifically, the mesh on the upper surface of the cargo Each triangular facet in To the reference plane Projection forms a triangular prism element, representing the total volume of the cargo inside the carriage. That is, the sum of the volumes of all effective infinitesimal elements, calculated as follows: (6) (7) (8) in, For the volume of a single triangular prism, M This refers to the number of triangular facets in the grid on the upper surface of the cargo. For triangular facets in Projected area on a plane This is the average height of the three vertices of the triangular facet from the reference plane.

[0062] This embodiment further provides a computer device, including a processor and a memory, wherein the memory is used to store a computer program and the processor is used to execute the computer program to perform the method as described above.

[0063] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0064] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0065] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0066] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for calculating the volume of cargo in a wagon by integrating point cloud semantic segmentation and mesh reconstruction, characterized in that the steps are as follows: include: Step S01. Point cloud data acquisition: Acquire multiple frames of raw point cloud data of the vehicle under test collected by the lidar and preprocess them to obtain preprocessed point cloud data; Step S02. 3D Reconstruction: Perform 3D reconstruction on the preprocessed point cloud data to obtain a 3D point cloud model of the vehicle under test; Step S03. Point cloud semantic segmentation: The three-dimensional point cloud model is input into the improved semantic segmentation network for point cloud semantic segmentation to segment the cargo surface point cloud and the carriage structure point cloud. The improved semantic segmentation network is based on the semantic segmentation network. In the local feature aggregation stage, spatial distance and feature distance are used to calculate the attention weight of the local neighborhood. Step S04. Volume Calculation: Based on the point cloud of the cargo surface obtained by segmentation and the point cloud of the carriage structure, a three-dimensional mesh is reconstructed, and the volume of the cargo inside the carriage is calculated based on the projection infinitesimal integral method.

2. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 1, characterized in that, In step S01, the preprocessing includes outlier filtering and / or ground point removal. The outlier filtering involves calculating the average distance from all points to their nearest neighbors and filtering out outliers based on the calculated average distance. The ground point removal involves calculating the distance from all points to the ground plane based on the ground plane model and filtering out ground points based on the calculated distance to the ground plane.

3. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 1, characterized in that, In step S02, the process of 3D reconstruction of the acquired raw point cloud data also includes motion compensation to correct pose deviations, including: Step S21. Coarse registration: Calculate the fast point feature histogram features of two adjacent frames of point cloud data and match them to obtain the initial transformation matrix; Step S22. Fine registration: Based on the initial transformation matrix, solve for the optimal transformation matrix with the goal of minimizing the distance between point clouds in two adjacent frames; Step S23 Coordinate Transformation: Use the optimal transformation matrix to transform the point cloud data of all frames to the same coordinate system to obtain the reconstructed three-dimensional point cloud model of the vehicle under test.

4. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 1, characterized in that, In the improved semantic segmentation network, the spatial distance is calculated based on the spatial Euclidean distance between points. The characteristic distance is calculated based on the differences in reflection intensity and geometric features. Using spatial distance Distance from features Calculate the attention weights for the local neighborhood: ， in, This represents the attention weight of the k-th point relative to the center point within the local neighborhood. This represents a multilayer perceptron. This represents the Softmax function. This indicates feature splicing.

5. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 1, characterized in that, The input feature vector of the improved semantic segmentation network includes at least the three-dimensional coordinates of each point in the point cloud, the lidar reflection intensity, and the covariance matrix feature values ​​calculated based on the neighborhood of each point. The covariance matrix feature values ​​include flatness features and / or verticality features.

6. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 1, characterized in that, The improved semantic segmentation network is trained using a hybrid loss function consisting of a weighted sum of Focal Loss and Dice Loss. The expression for the hybrid loss function is as follows: ,in, Indicates Dice Loss, Indicates Focal Loss, express The weighting coefficients.

7. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to any one of claims 1 to 6, characterized in that, Step S04 specifically includes: Step S41. Determining the reference plane: Based on the three-dimensional point cloud model of the empty vehicle, the reference plane at the bottom of the vehicle compartment is obtained by fitting. Step S42. Cargo surface meshing: Project the segmented cargo surface point cloud onto a horizontal plane to construct a two-dimensional triangular mesh, and map the constructed triangular mesh back into three-dimensional space to form a triangular mesh model of the cargo upper surface; Step S43. Volume Calculation: Project each triangular facet in the triangular mesh model on the upper surface of the cargo onto the reference plane at the bottom of the carriage to form a triangular prism micro-element. Combine the volumes of all the triangular prism micro-elements to obtain the total volume of the cargo inside the carriage.

8. The method for calculating the volume of cargo in a wagon by fusing point cloud semantic segmentation and mesh reconstruction according to claim 7, characterized in that, In step S43, the total volume of goods inside the carriage. The calculation expression is: in, For the volume of a single triangular prism, M This refers to the number of triangular facets in the grid on the upper surface of the cargo. For triangular facets in Projected area on a plane This is the average height of the three vertices of the triangular facet from the reference plane.

9. A computer device comprising a processor and a memory, the memory being used to store computer programs, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the method as described in any one of claims 1 to 8.

Images