An AI calculation method and system for sand ship measurement

Abstract

The present invention discloses an AI measurement method and system for measuring the volume of a sand ship. The method comprises the following steps: collecting images of the target sand ship to be measured by a high-definition camera of an unmanned aerial vehicle to obtain basic information and an initial image set of the sand ship to be measured; controlling the navigation path of the unmanned aerial vehicle to obtain coordinates of key points in the region; obtaining point cloud data and image data by controlling fine scanning and collection of each region of the navigation path; performing preprocessing on the point cloud image data; establishing a sand ship measurement model based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in a number of regions, and a PointNet++ neural network, and completing the measurement of the target sand ship to be measured according to the sand ship measurement model; the sand ship measurement method based on the PointNet++ neural network model of point cloud image data of the present invention realizes accurate measurement by denoising the wave disturbance noise characteristics of different loads of the point cloud image data and preprocessing the occluded objects, and establishes a point cloud surface fitting model for each load area, thereby improving the accuracy of sand ship transportation measurement.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence sand ship measurement, and in particular to an AI measurement method and system for sand ship measurement. Background Art

[0002] Sand ship measurement refers to the method of measuring or estimating the capacity or weight of the cargo loaded on the sand ship. The commonly used methods include: field measurement method, water gauge method, volume calculation method, weight estimation method, sensor monitoring method, etc. Among them, the field measurement method needs to be measured on site, which is time-consuming and costly, may be restricted by weather conditions and environment, and is easily affected by errors and uncertainties; the accuracy of the water gauge method is affected by the water level reading, and there may be reading errors and inaccurate water gauge calibration problems. At the same time, the weight of the cargo cannot be directly obtained, especially when affected by wind and waves at sea or on a large river, the error is large; the volume calculation method requires accurate data on the size of the cabin and the stacking of cargo, the calculation complexity is high, it takes a long time and there may be estimation errors; the weight estimation method requires the density of the cargo to be measured in advance and combined with the cabin volume to calculate, there may be density measurement errors and inaccurate estimation problems, and the operation is inconvenient. The sensor monitoring method requires the installation and maintenance of sensor equipment, which is costly, and may be limited by sensor accuracy and data processing algorithms. In addition, the irregular structure of each cargo on the sand ship is not considered, resulting in inaccurate measurement. Therefore, a sand ship volume measurement method with fast measurement efficiency, high data accuracy and easy operation is needed to solve the above problems. Summary of the invention

[0003] The purpose of the present invention is to provide an AI calculation method and system for sand ship volume measurement to solve the above-mentioned problems existing in the prior art.

[0004] The specific application is as follows: An AI calculation method for measuring the volume of a sand ship, wherein the AI ​​calculation system for measuring the volume of a sand ship is communicatively connected with a drone and a wave disturbance monitoring device, and the method comprises the following steps: S1. Determine the initial navigation path of the UAV based on navigation technology, and configure a high-definition camera and a three-dimensional laser radar on the UAV; S2. The UAV flies according to the initial navigation path and collects basic information and initial image set of the target sand ship through the high-definition camera and three-dimensional laser radar on board; S3, collecting regional key points of the target sand ship to be tested according to the basic information, and obtaining the coordinates of several regional key points; S4, inputting the initial image set, the basic information, and the coordinates of the several regional key points into a navigation parameter planning model, and outputting a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; S5, controlling the UAV to perform fine scanning and collection of each area of ​​the target sand ship to be tested through the navigation path set to obtain point cloud data and image data; S6, removing the occluding objects of the image data, extracting the wave disturbance noise features for the processed image data, reducing the noise of the point cloud data according to the wave disturbance noise features, and generating pre-processed point cloud calibration data, wherein the wave disturbance noise features have position coordinate identifiers; S7. Based on the point cloud calibration data, the three-dimensional absolute coordinates corresponding to the coordinates of several regional key points, and the PointNet++ neural network, a sand ship measurement model is established, and the measurement of the target sand ship to be measured is completed according to the sand ship measurement model.

[0005] Furthermore, in S3, the target sand ship to be detected is subjected to regional key point collection according to the basic information, and the coordinates of several regional key points are obtained, which also includes: S31, dividing the initial image set into regions according to the coordinates of a number of regional key points, aggregating the loaded objects in each region, and generating a classification of the loaded objects based on the result of the aggregation of the loaded objects; S32, performing object identification based on the classification of the loaded objects, obtaining an object identification result, and using the object identification result as a surface feature of the loaded objects of the first sand ship; S33, classifying the loaded objects and identifying the hardness and softness of the loaded objects to obtain hardness and softness identification results; S34, using the softness and hardness identification result as the surface feature of the objects loaded on the second sand ship; S35. Extract and obtain first wave disturbance noise features based on the surface features of the first sand ship loaded objects and the surface features of the second sand ship loaded objects.

[0006] Furthermore, step S35 further includes: The wave disturbance monitoring device is used to monitor the wave disturbance of the target sand ship to be tested, and a mapping relationship between the wave disturbance data and the point cloud data is established according to the wave disturbance time, the first wave disturbance noise characteristics, and the three-dimensional laser radar acquisition parameters; Performing disturbance characteristic analysis on the wave disturbance data to generate a second wave disturbance noise characteristic; According to the established mapping relationship, performing wave disturbance denoising compensation on the point cloud data based on the second wave disturbance noise feature; The performing disturbance characteristic analysis on the wave disturbance data to generate the second wave disturbance noise characteristic specifically comprises: The image frames collected by the wave disturbance monitoring device are subjected to fast Fourier transform to obtain the spectral values ​​of the corresponding speech frames, and a spectrogram is generated using the spectral values ​​of the speech frames; the abscissa of the spectrogram is the time corresponding to the speech frame, the ordinate of the spectrogram is the frequency component contained in the speech frame, the coordinate point value of the spectrogram is the spectral value, a wave disturbance threshold is preset, and if the spectral value is greater than the wave disturbance threshold, the coordinate point position is judged to be a wave disturbance coordinate point, and the wave disturbance threshold is determined according to the first wave disturbance noise feature.

[0007] Furthermore, in S6, the occluding objects of the image data are removed, wave disturbance noise features are extracted for the processed image data, and denoising the point cloud data according to the wave disturbance noise features includes: S61, correcting, enhancing and normalizing the image data of the target sand ship to be tested; S62, performing KD-tree algorithm filtering, downsampling and normalization processing on the point cloud data of the target sand ship to be tested; S63, converting the point cloud in the point cloud data into the coordinate system of the high-definition camera, and using the conversion formula to obtain the pixel coordinates, and removing the point cloud projected outside the target sand ship image to be tested; S64, constructing a Thiessen polygon network using the pixel coordinates of the point cloud projected onto the target sand ship image, and constructing a local spatial relationship for each point cloud, while distinguishing between occluded object nodes and non-occluded object nodes.

[0008] Furthermore, in S6, denoising the point cloud data by using the wave disturbance noise characteristics further includes: After the point cloud data is denoised, the initial area position distribution result of the point cloud is obtained through the coordinates of the key points in the area; Performing first regional clustering of point cloud data based on the initial regional position distribution result of the point cloud; After the first region clustering is completed, the point clouds of the same region are spliced ​​with feature point clouds. After the splicing is completed, the point clouds of different regions are spliced ​​between clusters to generate several key regions.

[0009] Furthermore, S7 establishes a sand ship measurement model based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several regions, and PointNet++ neural network, including: Based on the initial image set and the sand ship modeling, the sand ship loading volume feature evaluation is performed to obtain the sand ship evaluation feature and feature value; Construct sand ship measurement database; Optimizing the measurement method of the sand ship measurement database based on the sand ship evaluation characteristics and the characteristic values ​​to obtain a sand ship modeling optimization result; The load measurement of the target sand ship to be tested is completed through the sand ship modeling optimization result and the final model of the sand ship modeling.

[0010] Furthermore, the sand ship modeling includes: According to the extension direction of the horizontal plane of the sand ship, the load is divided into multiple partitions according to the coordinates of several regional key points, and at least one common regional key point coordinate is contained between two adjacent partitions; The three-dimensional absolute coordinates of the key points of each region of each partition are used as samples, corresponding to a determined point cloud surface fitting model function as a label, and the PointNet++ neural network is trained until the error of the PointNet++ neural network is lower than a preset threshold and tends to be stable, thereby obtaining a trained PointNet++ neural network; Input the two-dimensional coordinates of multiple regional key points of each partition into the trained PointNet++ neural network to obtain the point cloud surface fitting model of all partitions; Calculate the total volume of the point cloud surface fitting model of each partition to complete the measurement of the target sand ship to be measured; The number of inputs of the PointNet++ neural network is set to the sum of the number of all partitions, each partition corresponds to an independent PointNet++ neural network input, the point cloud surface fitting model of each partition corresponds to an independent PointNet++ neural network output, and the PointNet++ neural network input of a partition corresponds to the PointNet++ neural network output one by one.

[0011] Furthermore, the total volume of the point cloud surface fitting model of each partition is calculated by the grid method including: , Among them, V represents the total volume of the vertical cargo part of the sand ship, k represents the sequence number of the grid, and M represents the number of grids; Respectively represent the k-th point cloud surface fitting model, represents the coordinates of the center point of the kth square, Represents the side length of the square, and S represents the projection area of ​​the vertical load part on the horizontal plane.

[0012] Furthermore, completing the load measurement of the target sand ship to be tested through the sand ship modeling optimization result and the final model of the sand ship modeling also includes: Verifying the measurement result of the target sand ship to be measured to obtain a deviation value of the measurement result; Determine whether the deviation value of the measurement result is within a preset deviation threshold range; When the deviation value of the measurement result is not within the preset deviation threshold range, an error source analysis of the modeling measurement of the target sand ship to be measured is performed to obtain an error source analysis result; A sand ship modeling measurement compensation database is constructed based on the error source analysis results.

[0013] An AI measurement system for measuring the volume of sand ships, the system is connected to a high-definition camera and a three-dimensional laser radar on a drone, and a wave disturbance monitoring device, and the system includes: A basic information acquisition module, which is used to acquire basic information of the target sand ship to be tested; An initial image acquisition module, wherein the initial image acquisition module is used to acquire images of the target sand ship to be tested through the high-definition camera of the drone to obtain an initial image set; A regional key point acquisition module, which is used to acquire regional key points of the target sand ship to be tested and obtain the coordinates of the regional key points; A control navigation path module, wherein the control navigation path module is used to input the initial image set, the basic information, and the coordinates of several regional key points into a navigation parameter planning model, and output a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; A point cloud image data acquisition module, which is used to control the UAV to perform fine scanning and acquisition of various areas of the target sand ship to be tested through the control navigation path module to obtain point cloud data and image data; A point cloud image data preprocessing module, which is used to fuse the point cloud data and image data to obtain a fused image, remove occluded objects in the fused image, extract wave disturbance noise features for the processed fused image, reduce noise on the point cloud data by using the wave disturbance noise features, and generate preprocessed point cloud calibration data; The sand ship modeling and measurement module is used to establish a sand ship measurement model based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several areas, and a PointNet++ neural network, and complete the measurement of the target sand ship to be measured according to the sand ship measurement model.

[0014] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects: The embodiment of the present invention provides a method for determining an initial navigation path of an unmanned aerial vehicle based on navigation technology, and configuring a high-definition camera and a three-dimensional laser radar on the unmanned aerial vehicle; the unmanned aerial vehicle flies according to the initial navigation path, and collects basic information and an initial image set of a target sand ship to be tested through the high-definition camera and the three-dimensional laser radar carried by the unmanned aerial vehicle; regional key points of the target sand ship to be tested are collected according to the basic information to obtain coordinates of several regional key points; the initial image set, the basic information, and the coordinates of the several regional key points are input into a navigation parameter planning model, and a navigation path set is output, wherein the navigation parameter planning model executes a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping of regional key points; the navigation path set is controlled by The drone performs fine scanning and collection of each area of ​​the target sand ship to be measured to obtain point cloud data and image data; removes the occluding objects of the image data, extracts wave disturbance noise features for the processed image data, reduces noise on the point cloud data through the wave disturbance noise features, and generates pre-processed point cloud calibration data; establishes a sand ship measurement model based on the point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several areas, and a PointNet++ neural network, and completes the measurement of the target sand ship to be measured according to the sand ship measurement model; the sand ship measurement method based on the PointNet++ neural network model of point cloud image data of the present invention realizes accurate measurement by reducing the wave disturbance noise of the point cloud image data and pre-processing the occluding objects, and establishes a point cloud surface fitting model for each cargo area, thereby improving the accuracy of sand ship transportation measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a flow chart of an AI calculation method for sand ship volume measurement provided by an embodiment of the present invention; Figure 2 This is a system architecture diagram of an AI calculation system for sand ship measurement provided by an embodiment of the present invention; Figure 3 It is a multi-scale grouping schematic diagram of an AI measurement system for sand ship measurement provided by an embodiment of the present invention; Figure 4 It is a multi-resolution grouping schematic diagram of an AI measurement system for sand ship measurement provided by an embodiment of the present invention; Figure 5 It is a PointNet++ network schematic diagram of an AI measurement system for sand ship measurement provided by an embodiment of the present invention.

[0016] Among them, 10-basic information acquisition module, 11-initial image acquisition module, 12-regional key point acquisition module, 13-sand ship modeling and measurement module, 14-point cloud image data preprocessing module, 15-point cloud image data acquisition module, 16-control navigation path module. DETAILED DESCRIPTION

[0017] The present invention will be described in detail below in conjunction with the accompanying drawings.

[0018] Example 1

[0019] First, the technical terms involved in the embodiments of the present application are explained.

[0020] (1) KD-tree algorithm KD-tree (K-Dimensional Tree) is a tree data structure that stores instance points in k-dimensional space for fast retrieval. It is mainly used for searching key data in multidimensional space, such as range search and nearest neighbor search. KD-tree is essentially a balanced binary tree. As an efficient multidimensional data structure, KD-tree performs well in low to medium dimensional spaces, and is particularly suitable for application scenarios that require frequent nearest neighbor and range searches. Its simple structure and high search efficiency have made it widely used in many fields of computer science.

[0021] The embodiment of the present invention provides an AI calculation method for sand ship volume measurement, wherein the AI ​​calculation system for sand ship volume measurement is communicatively connected with a drone and a wave disturbance monitoring device, and the method comprises the following steps: S1. Determine the initial navigation path of the UAV based on navigation technology, and configure a high-definition camera and a three-dimensional laser radar on the UAV; S2. The UAV flies according to the initial navigation path and collects basic information and initial image set of the target sand ship through the high-definition camera and three-dimensional laser radar on board; It should be noted that the coverage area, obstacle distribution and other information of each loading area of ​​the target sand ship to be tested are collected as the basic information for measuring the target sand ship to be tested. The basic information is the basic basis for measuring the loading capacity of the target sand ship to be tested, and then the key points of each different loading area are divided according to the basic information.

[0022] S3, collecting regional key points of the target sand ship to be tested according to the basic information, and obtaining the coordinates of several regional key points; Specifically, the drone image acquisition is arranged for each cargo area of ​​the target sand ship to be tested, and all-round multi-angle image acquisition is performed on each cargo area of ​​the target sand ship to be tested based on the drone high-definition camera, and the acquired images are sequentially adjusted based on the movement of the acquisition navigation path to generate the initial image set. Preferably, the image acquisition distance can be set and adjusted, and the global image and the image of each area are acquired separately to ensure the accuracy of image acquisition and the completeness of image coverage information.

[0023] It should be noted that, based on each cargo area of ​​the target sand ship to be tested, the coordinate axis is determined based on the three-dimensional space position, and then a three-dimensional space coordinate system is constructed. Structural recognition is performed on each cargo area of ​​the target sand ship to be tested, and multiple positioning points that can characterize the structural conditions of each cargo area of ​​the target sand ship to be tested are identified and extracted, such as Canny edge detection positioning, YOLOv5 positioning, SSD positioning algorithm, etc. The specific density of the distribution of positioning points depends on the structural complexity of different positions. Based on the three-dimensional space coordinate system, the position mapping correspondence is performed on the determined multiple positioning points, and the key point coordinates in the three-dimensional space coordinate system are determined to constitute the key point distribution coordinates. Based on the key point distribution coordinates, the main frame of each cargo area of ​​the target sand ship to be tested can be determined.

[0024] S4, inputting the initial image set, the basic information, and the coordinates of the several regional key points into a navigation parameter planning model, and outputting a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; Specifically, the navigation parameter planning model is constructed, and the navigation parameter planning model is an analysis model for the control parameters of the three-dimensional laser radar point cloud scanning of the unmanned aerial vehicle. Exemplarily, the following is an implementable modeling method: conduct big data survey statistics to determine the project data of multiple sand ship cargo projects, including sample image sets and sample basic information, perform positioning point analysis on this basic information, determine the distribution coordinates of sample key points, and extract the control parameters corresponding to each sample. Since the engineering data is a constructed project, the above parameter data can be directly obtained. The sample image set, the sample basic information and the sample key point distribution coordinates are used as identification information to determine the regional identification key node, and the corresponding sample control parameter is used as the regional key point decision node. The regional identification key node and the regional key point decision node are mapped and connected to generate a navigation parameter decision tree. Based on the navigation parameter decision tree, the navigation parameter planning model is generated by training the PointNet++ neural network. The initial image set, the basic information, and the coordinates of the several regional key points are input into the navigation parameter planning model. By performing data matching mapping, the corresponding control navigation parameters can be directly determined and output as the navigation path set. Constructing the navigation parameter planning model to determine the navigation path set can improve the analysis speed and the accuracy of the analysis results.

[0025] It should be noted that the specific process of determining the regional key point decision node is as follows: PointNet++ network proposes two adaptive density grouping clustering methods, namely Multi-Scale Grouping (MSG) and Multi-Resolution Grouping (MRG). Figure 3 He Ru Figure 4 As shown in the figure, by capturing the multi-scale features of dense and sparse areas through different radius grouping, its adaptability to non-uniformly sampled point clouds is enhanced, and the robustness of the model to non-uniform sampling is improved, thereby improving its performance in classification and segmentation tasks.

[0026] Multi-scale grouping (MSG): Figure 3 As shown in Figure 1, the basic idea is to generate groups of different scales, i.e. different radii, use feature extraction to extract the features of each scale, and finally connect the features extracted from each scale to form multi-scale features. Figure 4As shown in the figure, the central idea is to make the features of each layer consist of two vectors. The vector on the left uses the geometric abstraction layer to summarize the features of the sub-region, and the vector on the right directly uses RintNet to extract features from the original point cloud, assigning different weights to the two sides according to the density of the point cloud. After the feature extraction is completed, the PointNet++ network can use the global features for regional key point decision node classification according to the needs of the specific task, or pass the high-level features back to the original point through the feature propagation mechanism to achieve fine prediction of regional key point decision nodes. In the classification task, the global features are compressed into a fixed-size vector through feature abstraction. Finally, the nonlinear transformation is completed through the fully connected layer, and the prediction results of the regional key point decision node category are output. In the segmentation task, the feature propagation module gradually transfers the high-level features back to the original high-resolution point set to generate point-by-point features. And generate the semantic label of each regional key point decision node through point-by-point classification, and then determine the regional key point decision node.

[0027] S5, controlling the UAV to perform fine scanning and collection of each area of ​​the target sand ship to be tested through the navigation path set to obtain point cloud data and image data; S6, removing the occluding objects of the image data, extracting the wave disturbance noise features for the processed image data, reducing the noise of the point cloud data according to the wave disturbance noise features, and generating pre-processed point cloud calibration data, wherein the wave disturbance noise features have position coordinate identifiers; S7. Based on the point cloud calibration data, the three-dimensional absolute coordinates corresponding to the coordinates of several regional key points, and the PointNet++ neural network, a sand ship measurement model is established, and the measurement of the target sand ship to be measured is completed according to the sand ship measurement model.

[0028] Specifically, based on navigation technology, an initial navigation path of a UAV is determined, and a high-definition camera and a three-dimensional laser radar are configured on the UAV; the UAV flies according to the initial navigation path, and collects basic information and an initial image set of a target sand ship to be tested through the high-definition camera and the three-dimensional laser radar carried by the UAV; regional key points of the target sand ship to be tested are collected according to the basic information to obtain coordinates of several regional key points; the initial image set, the basic information, and the coordinates of the several regional key points are input into a navigation parameter planning model, and a navigation path set is output, wherein the navigation parameter planning model executes a first decision tree neural network training and construction based on a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping of regional key points; the UAV is controlled by the navigation path set. The human-machine system performs fine scanning and collection of each area of ​​the target sand ship to be measured to obtain point cloud data and image data; removes the occluding objects of the image data, extracts wave disturbance noise features for the processed image data, reduces noise on the point cloud data through the wave disturbance noise features, and generates pre-processed point cloud calibration data; establishes a sand ship measurement model based on the point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several areas, and a PointNet++ neural network, and completes the measurement of the target sand ship to be measured according to the sand ship measurement model; the sand ship measurement method based on the PointNet++ neural network model of point cloud image data of the present invention reduces the wave disturbance noise of the point cloud image data and pre-processes the occluding objects, and establishes a point cloud surface fitting model for each cargo area to achieve accurate measurement, thereby improving the accuracy of sand ship transportation measurement.

[0029] In the above embodiment, specifically, in S3, the target sand ship to be detected is collected according to the basic information, and the coordinates of several regional key points are obtained, which further includes: S31, dividing the initial image set into regions according to the coordinates of a number of regional key points, aggregating the loaded objects in each region, and generating a classification of the loaded objects based on the result of the aggregation of the loaded objects; S32, performing object identification based on the classification of the loaded objects, obtaining an object identification result, and using the object identification result as a surface feature of the loaded objects of the first sand ship; S33, classifying the loaded objects and identifying the hardness and softness of the loaded objects to obtain hardness and softness identification results; S34, using the softness and hardness identification result as the surface feature of the objects loaded on the second sand ship; S35. Extract and obtain a first wave disturbance noise feature based on the surface feature of the first sand ship loaded objects and the surface feature of the second sand ship loaded objects.

[0030] It should be noted that the initial image set is divided into different storage areas according to the key points of each area, and the different storage areas are, for example, undulating features, obstacles, etc. Material identification is performed for different storage areas, such as sand, stone, wood, coal, etc., and the material identification results are mapped and corresponded with the storage areas, as the surface features of the first sand ship loaded items. The soft and hardness of the surface of the loaded items is further identified for the classification of the loaded items to obtain the soft and hardness identification results; the soft and hardness identification results are used as the surface features of the second sand ship loaded items. For example, the surface soft and hardness can be measured according to the image recognition algorithm, and the measurement results are mapped and corresponded with the storage areas to generate the soft and hardness identification results. The soft and hardness identification results are used as the surface features of the second sand ship loaded items. Based on the surface features of the first sand ship loaded items, the surface features of the second sand ship loaded items, and the wave disturbance noise, the corresponding wave disturbance noise features are determined, such as the phase deviation of the three-dimensional laser radar echo signal, etc., the wave disturbance noise points are located, and then the position coordinates of the wave disturbance noise features are marked. The wave disturbance noise characteristics are determined through accurate identification, so as to facilitate subsequent targeted wave disturbance noise feature positioning and removal.

[0031] In the above embodiment, specifically, step S35 further includes: The wave disturbance monitoring device is used to monitor the wave disturbance of the target sand ship to be tested, and a mapping relationship between the wave disturbance data and the point cloud data is established according to the wave disturbance time, the first wave disturbance noise characteristics, and the three-dimensional laser radar acquisition parameters; Performing disturbance characteristic analysis on the wave disturbance data to generate a second wave disturbance noise characteristic; According to the established mapping relationship, performing wave disturbance denoising compensation on the point cloud data based on the second wave disturbance noise feature; The performing disturbance characteristic analysis on the wave disturbance data to generate the second wave disturbance noise characteristic specifically comprises: The image frames collected by the wave disturbance monitoring device are subjected to fast Fourier transform to obtain the spectral values ​​of the corresponding speech frames, and a spectrogram is generated using the spectral values ​​of the speech frames; the abscissa of the spectrogram is the time corresponding to the speech frame, the ordinate of the spectrogram is the frequency component contained in the speech frame, the coordinate point value of the spectrogram is the spectral value, a wave disturbance threshold is preset, and if the spectral value is greater than the wave disturbance threshold, the coordinate point position is judged to be a wave disturbance coordinate point, and the wave disturbance threshold is determined according to the first wave disturbance noise feature.

[0032] It should be noted that, based on the mapping relationship between wave disturbance data and point cloud data, the wave disturbance features are matched with the point cloud data to adjust and correct the spatial position of the point cloud data, so as to complete the wave disturbance denoising compensation of the point cloud data and further improve the data accuracy. The wave disturbance threshold is determined according to the first wave disturbance noise feature. Specifically, if the load in the area corresponding to the first wave disturbance noise feature is sand, stone, wood, and coal, respectively, the corresponding thresholds are set to 2, 3, 4, and 5, respectively.

[0033] In the above embodiment, specifically, removing the occluding object of the image data in S6, extracting the wave disturbance noise feature for the processed image data, and reducing the noise of the point cloud data by using the wave disturbance noise feature includes: S61, correcting, enhancing and normalizing the image data of the target sand ship to be tested; S62, performing KD-tree algorithm filtering, downsampling and normalization processing on the point cloud data of the target sand ship to be tested; S63, converting the point cloud in the point cloud data into the coordinate system of the high-definition camera, and using the conversion formula to obtain the pixel coordinates, and removing the point cloud projected outside the target sand ship image to be tested; S64, constructing a Thiessen polygon network using the pixel coordinates of the point cloud projected onto the target sand ship image, and constructing a local spatial relationship for each point cloud, while distinguishing between occluded object nodes and non-occluded object nodes.

[0034] It should be noted that, further to distinguish the occluding object and the non-occluding object nodes in step S64, the point cloud nodes of the occluding object may be removed.

[0035] In the above embodiment, specifically, the denoising of the point cloud data by using the wave disturbance noise feature in S6 further includes: After the point cloud data is denoised, the initial area position distribution result of the point cloud is obtained through the coordinates of the key points in the area; Performing first regional clustering of point cloud data based on the initial regional position distribution result of the point cloud; After the first region clustering is completed, the point clouds of the same region are spliced ​​with feature point clouds. After the splicing is completed, the point clouds of different regions are spliced ​​between clusters to generate several key regions.

[0036] In the above embodiment, specifically, in S7, based on the point cloud calibration data, the three-dimensional absolute coordinates corresponding to the coordinates of the key points in the region, and the PointNet++ neural network, establishing the sand ship measurement model also includes: Based on the initial image set and the sand ship modeling, the sand ship loading volume feature evaluation is performed to obtain the sand ship evaluation feature and feature value; Construct sand ship measurement database; Optimizing the measurement method of the sand ship measurement database based on the sand ship evaluation characteristics and the characteristic values ​​to obtain a sand ship modeling optimization result; The load measurement of the target sand ship to be tested is completed through the sand ship modeling optimization result and the final model of the sand ship modeling.

[0037] It should be noted that the calculation of the volume of cargo carried is based on the sand ship modeling. Due to the differences in specific cargo carrying conditions, the metering efficiency and effect of different metering methods are different. Based on the different characteristics of the cargo storage position of the target sand ship to be measured, the surface cumulative volume measurement method of the storage area at different positions based on the PointNet++ neural network is adopted. Specifically, based on the initial image set and the sand ship modeling, the key point position features of each area of ​​the target sand ship to be measured are identified and extracted, such as the degree of regional undulation, etc., as the evaluation features of the sand ship loading volume, and the specific feature value is used as the feature value, such as the specific amplitude of the loading area undulation. A big data mining algorithm is used to collect and obtain a variety of applicable sand ship loading volume features, which are correspondingly integrated to generate the sand ship loading volume measurement database, which is used for the feature input of the PointNet++ neural network to generate an accurate fitting surface. For example, the cross-section method is suitable for the storage area with large undulations in the loading area and irregular items.

[0038] In the above embodiment, specifically, the sand ship modeling includes: According to the extension direction of the horizontal plane of the sand ship, the load is divided into multiple partitions according to the coordinates of several regional key points, and at least one common regional key point coordinate is contained between two adjacent partitions; The three-dimensional absolute coordinates of the key points of each region of each partition are used as samples, corresponding to a determined point cloud surface fitting model function as a label, and the PointNet++ neural network is trained until the error of the PointNet++ neural network is lower than a preset threshold and tends to be stable, thereby obtaining a trained PointNet++ neural network; Input the two-dimensional coordinates of multiple regional key points of each partition into the trained PointNet++ neural network to obtain the point cloud surface fitting model of all partitions; Calculate the total volume of the point cloud surface fitting model of each partition to complete the measurement of the target sand ship to be measured; The number of inputs of the PointNet++ neural network is set to the sum of the number of all partitions, each partition corresponds to an independent PointNet++ neural network input, the point cloud surface fitting model of each partition corresponds to an independent PointNet++ neural network output, and the PointNet++ neural network input of a partition corresponds to the PointNet++ neural network output one by one.

[0039] Specifically, the regional point cloud surface fitting model can also be implemented using, but not limited to, the following steps: Determining a surface in a 3D point cloud space requires a point on the surface and the surface normal vector , obtain point cloud data for regional key points , where the number of key points m is not less than 3, then all points The distance to the surface formula is: , The objective function equation is: , The centers of all points in the surface point cloud fall on the regional point cloud surface, and the difference of the objective function is equal to 0, that is: , The regional point cloud surface fitting formula is obtained through the above formula: , Get the point It falls on the desired fitting surface. In this embodiment, the normal vector of the regional point cloud surface fitting is solved according to the least squares method.

[0040] In the above embodiment, specifically, the total volume of the point cloud surface fitting model of each region is calculated by the grid method, including: , Among them, V represents the total volume of the vertical cargo part of the sand ship, k represents the sequence number of the grid, and M represents the number of grids; Respectively represent the k-th point cloud surface fitting model, represents the coordinates of the center point of the kth square, Represents the side length of the square, and S represents the projection area of ​​the vertical load part on the horizontal plane.

[0041] In the above embodiment, specifically, completing the load measurement of the target sand ship to be tested through the sand ship modeling optimization result and the final model of the sand ship modeling also includes: Verifying the measurement result of the target sand ship to be measured to obtain a deviation value of the measurement result; Determine whether the deviation value of the measurement result is within a preset deviation threshold range; When the deviation value of the measurement result is not within the preset deviation threshold range, an error source analysis of the modeling measurement of the target sand ship to be measured is performed to obtain an error source analysis result; A sand ship modeling measurement compensation database is constructed based on the error source analysis results.

[0042] It should be noted that the measurement results are verified to determine the measurement accuracy and obtain the deviation value of the measurement results. Exemplarily, multiple verification points are determined for the sand ship cargo to be measured, which can be set randomly, and actual measurements are performed, and the corresponding measurement results in the sand ship modeling are checked to determine the deviation value of the measurement results. The expected deviation value threshold is set, that is, the critical data for determining the degree of deviation of the measurement result, such as a deviation amplitude of 5%, and further determines whether the deviation value of the measurement result is within the expected deviation value range. When it is, it indicates that the degree of deviation is small and can be appropriately ignored; when it is not, it indicates that the degree of deviation of the measurement data is large, and data compensation is required to ensure data accuracy. The error source analysis of the modeling measurement of the sand ship cargo to be measured is carried out, and the data deviation direction and deviation amplitude are determined for specific judgment. Specifically, the data collection and processing flow is checked to locate the source of error, determine the source of error, error direction and error scale as the error source analysis result, the error source analysis result is a complete analysis result of the volume of the sand ship carrying cargo to be measured, determine the error source analysis result, based on the error source analysis result, determine the adjustment data of different error positions, integrate and generate the sand ship modeling measurement compensation database, based on the sand ship modeling measurement compensation database, adjust the measurement results, and further improve the accuracy of the measurement results.

[0043] Example 2

[0044] An AI measurement system for measuring the volume of sand ships, the system is connected to a high-definition camera and a three-dimensional laser radar on a drone, and a wave disturbance monitoring device, and the system includes: A basic information acquisition module 10, wherein the basic information acquisition module 10 is used to acquire basic information of the target sand ship to be tested; An initial image acquisition module 11, wherein the initial image acquisition module 11 is used to acquire an image of the target sand ship to be tested through the high-definition camera of the drone to obtain an initial image set; A regional key point acquisition module 12, wherein the regional key point acquisition module 12 is used to acquire regional key points of the target sand ship to be detected and obtain the coordinates of the regional key points; A control navigation path module 16, wherein the control navigation path module 16 is used to input the initial image set, the basic information, and the coordinates of several regional key points into a navigation parameter planning model, and output a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; A point cloud image data acquisition module 15, which is used to control the UAV to perform fine scanning and acquisition of each area of ​​the target sand ship to be tested through the control navigation path module 16 to obtain point cloud data and image data; A point cloud image data preprocessing module 14, which is used to fuse the point cloud data and the image data to obtain a fused image, remove occluded objects in the fused image, extract wave disturbance noise features from the processed fused image, reduce noise on the point cloud data through the wave disturbance noise features, and generate preprocessed point cloud calibration data; The sand ship modeling and measurement module 13 is used to establish a sand ship measurement model based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several areas, and a PointNet++ neural network, and complete the measurement of the target sand ship to be measured according to the sand ship measurement model.

[0045] The algorithm and display provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious that the structure required for constructing such systems. In addition, the present invention is not directed to any specific programming language either. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description of the above specific languages ​​is for disclosing the best mode of the present invention.

[0046] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0047] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. More specifically, as reflected in the claims below, inventive aspects lie in less than all the features of the individual embodiments disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the present invention.

[0048] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device so disclosed may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0049] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present invention and form different embodiments. For example, in the claims below, any one of the claimed embodiments may be used in any combination.

[0050] The various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in the apparatus according to an embodiment of the present invention. The present invention may also be implemented as a device or device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

Claims

1. An AI calculation method for sand ship measurement, characterized in that: The AI ​​calculation system of the sand ship is connected to the drone and the wave disturbance monitoring device in communication, and the method includes the following steps: S1. Determine the initial navigation path of the UAV based on navigation technology, and configure a high-definition camera and a three-dimensional laser radar on the UAV; S2. The UAV flies according to the initial navigation path and collects basic information and initial image set of the target sand ship through the high-definition camera and three-dimensional laser radar on board; S3, collecting regional key points of the target sand ship to be tested according to the basic information, and obtaining the coordinates of several regional key points; S4, inputting the initial image set, the basic information, and the coordinates of the several regional key points into a navigation parameter planning model, and outputting a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; S5, controlling the UAV to perform fine scanning and collection of each area of ​​the target sand ship to be tested through the navigation path set to obtain point cloud data and image data; S6, removing the occluding objects of the image data, extracting the wave disturbance noise features for the processed image data, reducing the noise of the point cloud data according to the wave disturbance noise features, and generating pre-processed point cloud calibration data, wherein the wave disturbance noise features have position coordinate identifiers; S7. Based on the point cloud calibration data, the three-dimensional absolute coordinates corresponding to the coordinates of several regional key points, and the PointNet++ neural network, a sand ship measurement model is established, and the measurement of the target sand ship to be measured is completed according to the sand ship measurement model.

2. The AI ​​calculation method for sand ship measurement according to claim 1 is characterized in that: In S3, the target sand ship to be detected is subjected to regional key point collection according to the basic information, and the coordinates of several regional key points are obtained, which also includes: S31, dividing the initial image set into regions according to the coordinates of a number of regional key points, aggregating the loaded objects in each region, and generating a classification of the loaded objects based on the result of the aggregation of the loaded objects; S32, performing object identification based on the classification of the loaded objects, obtaining an object identification result, and using the object identification result as a surface feature of the loaded objects of the first sand ship; S33, classifying the loaded objects and identifying the hardness and softness of the loaded objects to obtain hardness and softness identification results; S34, using the softness and hardness identification result as the surface feature of the objects loaded on the second sand ship; S35. Extract and obtain first wave disturbance noise features based on the surface features of the first sand ship loaded objects and the surface features of the second sand ship loaded objects.

3. The AI ​​calculation method for sand ship measurement according to claim 2 is characterized in that: Step S35 also includes: The wave disturbance monitoring device is used to monitor the wave disturbance of the target sand ship to be tested, and a mapping relationship between the wave disturbance data and the point cloud data is established according to the wave disturbance time, the first wave disturbance noise characteristics, and the three-dimensional laser radar acquisition parameters; Performing disturbance characteristic analysis on the wave disturbance data to generate a second wave disturbance noise characteristic; According to the established mapping relationship, performing wave disturbance denoising compensation on the point cloud data based on the second wave disturbance noise feature; Performing disturbance feature analysis on the wave disturbance data to generate a second wave disturbance noise feature specifically includes: The image frames collected by the wave disturbance monitoring device are subjected to fast Fourier transform to obtain the spectral values ​​of the corresponding speech frames, and a spectrogram is generated using the spectral values ​​of the speech frames; the abscissa of the spectrogram is the time corresponding to the speech frame, the ordinate of the spectrogram is the frequency component contained in the speech frame, the coordinate point value of the spectrogram is the spectral value, a wave disturbance threshold is preset, and if the spectral value is greater than the wave disturbance threshold, the coordinate point position is judged to be a wave disturbance coordinate point, and the wave disturbance threshold is determined according to the first wave disturbance noise feature.

4. The AI ​​calculation method for sand ship measurement according to claim 1 is characterized in that: In S6, the occluded object of the image data is removed, wave disturbance noise features are extracted for the processed image data, and noise reduction is performed on the point cloud data according to the wave disturbance noise features, including: S61, correcting, enhancing and normalizing the image data of the target sand ship to be measured; S62, performing KD-tree algorithm filtering, downsampling and normalization processing on the point cloud data of the target sand ship to be tested; S63, converting the point cloud in the point cloud data into the coordinate system of the high-definition camera, and using the conversion formula to obtain the pixel coordinates, and removing the point cloud projected outside the target sand ship image to be tested; S64, constructing a Thiessen polygon network using the pixel coordinates of the point cloud projected onto the target sand ship image, and constructing a local spatial relationship for each point cloud, while distinguishing between occluded object nodes and non-occluded object nodes.

5. The AI ​​calculation method for sand ship measurement according to claim 1 is characterized in that: In S6, the step of reducing the noise of the point cloud data by using the wave disturbance noise characteristics further includes: After the point cloud data is denoised, the initial area position distribution result of the point cloud is obtained through the coordinates of the key points in the area; Performing first regional clustering of point cloud data based on the initial regional position distribution result of the point cloud; After the first region clustering is completed, the point clouds of the same region are spliced ​​with feature point clouds. After the splicing is completed, the point clouds of different regions are spliced ​​between clusters to generate several key regions.

6. The AI ​​calculation method for sand ship measurement according to claim 1 is characterized in that: In S7, a sand ship measurement model is established based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several regions, and PointNet++ neural network, including: Based on the initial image set and the sand ship modeling, the sand ship loading volume feature evaluation is performed to obtain the sand ship evaluation feature and feature value; Construct sand ship measurement database; Optimizing the measurement method of the sand ship measurement database based on the sand ship evaluation characteristics and the characteristic values ​​to obtain a sand ship modeling optimization result; The load measurement of the target sand ship to be tested is completed through the sand ship modeling optimization result and the final model of the sand ship modeling.

7. The AI ​​calculation method for sand ship measurement according to claim 6 is characterized in that: The sand ship modeling includes: According to the extension direction of the horizontal plane of the sand ship, the load is divided into multiple partitions according to the coordinates of several regional key points, and at least one common regional key point coordinate is contained between two adjacent partitions; The three-dimensional absolute coordinates of the key points of each region of each partition are used as samples, corresponding to a determined point cloud surface fitting model function as a label, and the PointNet++ neural network is trained until the error of the PointNet++ neural network is lower than a preset threshold and tends to be stable, thereby obtaining a trained PointNet++ neural network; Input the two-dimensional coordinates of multiple regional key points of each partition into the trained PointNet++ neural network to obtain the point cloud surface fitting model of all partitions; Calculate the total volume of the point cloud surface fitting model of each partition to complete the measurement of the target sand ship to be measured; The number of inputs of the PointNet++ neural network is set to the sum of the number of all partitions, each partition corresponds to an independent PointNet++ neural network input, the point cloud surface fitting model of each partition corresponds to an independent PointNet++ neural network output, and the PointNet++ neural network input of a partition corresponds to the PointNet++ neural network output one by one.

8. The AI ​​calculation method for sand ship measurement according to claim 7 is characterized in that: Calculate the total volume of the point cloud surface fitting model of each partition, using the grid method to calculate: ， Among them, V represents the total volume of the vertical cargo part of the sand ship, k represents the sequence number of the grid, and M represents the number of grids; Respectively represent the k-th point cloud surface fitting model, represents the coordinates of the center point of the k-th square, Represents the side length of the square, and S represents the projection area of ​​the vertical load part on the horizontal plane.

9. The AI ​​calculation method for sand ship measurement according to claim 6 is characterized in that: Completing the load measurement of the target sand ship to be tested through the sand ship modeling optimization result and the final model of the sand ship modeling also includes: Verifying the measurement result of the target sand ship to be measured to obtain a deviation value of the measurement result; Determine whether the deviation value of the measurement result is within a preset deviation threshold range; When the deviation value of the measurement result is not within the preset deviation threshold range, an error source analysis of the modeling measurement of the target sand ship to be measured is performed to obtain an error source analysis result; A sand ship modeling measurement compensation database is constructed based on the error source analysis results.

10. An AI calculation system for sand ship measurement, characterized in that: The system is connected to the high-definition camera, three-dimensional laser radar and wave disturbance monitoring device on the drone, and the system includes: A basic information acquisition module, which is used to acquire basic information of the target sand ship to be tested; An initial image acquisition module, wherein the initial image acquisition module is used to acquire images of the target sand ship to be tested through the high-definition camera of the drone to obtain an initial image set; A regional key point acquisition module, which is used to acquire regional key points of the target sand ship to be tested and obtain the coordinates of the regional key points; A control navigation path module, wherein the control navigation path module is used to input the initial image set, the basic information, and the coordinates of several regional key points into a navigation parameter planning model, and output a navigation path set, wherein the navigation parameter planning model performs a first decision tree neural network training and construction according to a navigation parameter decision tree, and the navigation parameter decision tree is constructed by associating the regional identification nodes configured by the decision processing of historical sample data with the regional mapping association of the regional key points; A point cloud image data acquisition module, which is used to control the UAV to perform fine scanning and acquisition of various areas of the target sand ship to be tested through the control navigation path module to obtain point cloud data and image data; A point cloud image data preprocessing module, which is used to fuse the point cloud data and image data to obtain a fused image, remove occluded objects in the fused image, extract wave disturbance noise features for the processed fused image, reduce noise on the point cloud data by using the wave disturbance noise features, and generate preprocessed point cloud calibration data; The sand ship modeling and measurement module is used to establish a sand ship measurement model based on point cloud calibration data, three-dimensional absolute coordinates corresponding to the coordinates of key points in several areas, and a PointNet++ neural network, and complete the measurement of the target sand ship to be measured according to the sand ship measurement model.

Images