Aggregate particle size prediction method based on three-dimensional multi-view

Abstract

The invention discloses an aggregate particle size prediction method based on three-dimensional multiple views, which comprises the following steps: generating a multi-view projection from three-dimensional point cloud data of an aggregate, and fusing and processing multi-view information in combination with a deep learning technology so as to realize higher-precision and more stable particle size classification; according to the method, the defects of a two-dimensional single-view method can be effectively overcome, the advantages of three-dimensional data can be fully utilized, the complexity and viewpoint dependence of aggregate multi-view information acquisition are effectively reduced, and the consistency and accuracy of information under complex illumination and environment conditions in an industrial environment are ensured.

Description

Technical Field

[0001] The invention belongs to the technical field of material detection and data analysis, and in particular relates to an aggregate particle size prediction method based on three-dimensional multi-views. Background Art

[0002] Classifying aggregate particle size is an important step in predicting aggregate gradation, which determines the accuracy of gradation calculation and affects the quality and performance evaluation of asphalt pavement. In construction projects with high continuity requirements such as continuous asphalt paving and dam construction, accurate prediction of aggregate particle size can ensure that particle size classification before aggregate mixing or gradation measurement can be achieved without interrupting the operation of the production line, so as to meet the high requirements of continuous asphalt concrete construction on mineral mixture productivity and quality.

[0003] Most of the existing aggregate particle size prediction methods perform particle size classification based on two-dimensional images or three-dimensional point cloud data. The basic idea is: first, collect two-dimensional images or three-dimensional point cloud data of aggregates through sensors (such as cameras, laser scanners, etc.); then, extract the geometric features of aggregates from the collected data, such as the aspect ratio and elongation in two-dimensional images, or the volume and maximum diameter in three-dimensional point clouds; finally, use machine learning algorithms or deep learning models to analyze and classify these features, and output the particle size range or grading results of aggregates.

[0004] The existing methods have the following disadvantages:

[0005] (1) Two-dimensional methods can only reflect the morphological information of aggregates from a specific viewpoint and cannot obtain key information such as height. Especially for needle-shaped aggregates, the accuracy of particle size classification is greatly reduced. (2) Single-view methods are greatly affected by viewpoint constraints and noise. The same aggregate particles have different appearances at different angles or lighting. A single view cannot verify or correct information from other angles and has poor stability. (3) Three-dimensional methods mostly rely on morphological and feature engineering techniques to process three-dimensional data. Although they provide more information, they often ignore the recovery and utilization of multi-view information, resulting in insufficient understanding and classification of the overall morphology of aggregates. (4) Although three-dimensional methods can provide more comprehensive geometric information, the computational complexity of processing three-dimensional point cloud data is high, especially in applications with large data volumes and high real-time requirements, and it is difficult to respond quickly. Summary of the invention

[0006] The purpose of the present invention is to provide an aggregate particle size prediction method based on three-dimensional multi-views, which combines deep learning technology to fuse and process multi-view information, thereby achieving higher precision and more stable particle size classification; the present invention can not only effectively make up for the shortcomings of the two-dimensional single-view method, but also make full use of the advantages of three-dimensional data, effectively reduce the complexity and viewpoint dependence of aggregate multi-view information acquisition, and ensure the consistency and accuracy of information under complex lighting and environmental conditions in industrial environments.

[0007] The technical solution adopted by the present invention is a method for predicting aggregate particle size based on three-dimensional multi-views, comprising:

[0008] 3D point cloud data collection, collecting 3D point cloud data of aggregates;

[0009] Image restoration and segmentation: using an interpolation method to restore the three-dimensional point cloud data of the aggregate, segmenting the restored three-dimensional point cloud data of the aggregate into individual particles to obtain an aggregate sample point cloud database;

[0010] Image matching, performing aggregate multi-view acquisition based on the aggregate sample point cloud database to obtain an aggregate sample image set; selecting a test aggregate point cloud from the aggregate sample point cloud database, and then selecting an image corresponding to the test aggregate point cloud from the aggregate sample image set;

[0011] Feature extraction: After image matching is completed, feature extraction is performed on the matched image to obtain a high-dimensional feature representation of the aggregate point cloud and output the feature shape;

[0012] Category classification: classify the particles according to the characteristic shapes to complete the particle size classification.

[0013] Furthermore, image restoration and segmentation specifically include:

[0014] The B-spline interpolation method is used to restore 3D point cloud data. First, the unknown control points are found using the inverse algorithm based on the known point cloud data, the B sample curve is determined, and then the missing point cloud positions are substituted to calculate the missing point cloud values.

[0015] The global threshold is used to segment the restored three-dimensional point cloud data of the aggregate into individual particles. Specifically, the pixel points with grayscale between the minimum grayscale value and the maximum grayscale value are selected, and the selected pixel points are combined into a continuous connected area, that is, the Region area.

[0016] Furthermore, the aggregate multi-view collection specifically includes:

[0017] Select two view projection methods, one is the three-view method, including the front view, top view and side view;

[0018] The other is to add additional views focusing on the upper surface of the aggregate and oblique views on the basis of the three views;

[0019] The above two view projection methods are used to obtain the maximum projection contour image library.

[0020] Furthermore, the image background in the maximum projection contour image library is black, and the image size is a view of 256x256.

[0021] Furthermore, image matching includes:

[0022] The projection contour images of the test aggregate point cloud in the projection contour images respectively obtained by the two view projection methods need to correspond one to one.

[0023] Furthermore, feature extraction includes:

[0024] The projection contour map corresponding to the selected test aggregate point cloud is subjected to feature extraction through a convolutional neural network to obtain multi-view features, and then the multi-view features are hierarchically aggregated to obtain a high-dimensional feature representation of the test aggregate point cloud and output the feature shape.

[0025] Further, the category classification includes:

[0026] Repeat the feature extraction until all aggregate samples in the aggregate sample point cloud database are processed to obtain a plurality of characteristic shapes, and complete the aggregate particle size classification according to the categories of the plurality of characteristic shapes.

[0027] The beneficial effects of the present invention are:

[0028] The present invention discloses an aggregate particle size prediction method based on three-dimensional multi-views. By adopting a multi-view projection combined with a graph convolutional network, the present invention can make full use of three-dimensional data, enrich data features and enhance the interaction between feature expression and viewing angle information. Compared with the existing methods, the present invention has more accurate and comprehensive morphological characterization capabilities. At the same time, the flexible view quantity selection and feature fusion strategy overcome the problems of insufficient viewing angle information utilization and insufficient multi-view processing capabilities of the existing methods, improves the accuracy of aggregate particle size classification prediction, and provides a more efficient and flexible solution for aggregate size classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of a method for predicting aggregate particle size based on three-dimensional multi-views of the present invention;

[0030] Figure 2 It is a three-view projection perspective diagram in a three-dimensional multi-view based aggregate particle size prediction method of the present invention;

[0031] Figure 3 It is a nine-view projection perspective diagram in a three-dimensional multi-view based aggregate particle size prediction method of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0034] This embodiment provides a method for predicting aggregate particle size based on three-dimensional multi-views. Figure 1 As shown, specifically including:

[0035] Aggregate multi-view acquisition:

[0036] Selecting a suitable viewing angle can fully represent a 3D object with the least amount of projection. In this embodiment, two view projection methods are selected and designed; Figure 2 As shown in Figure 1, method 1 uses three views, namely, the main view, the top view and the side view; these three views can provide a simple and intuitive representation of the main geometric information of the aggregate while minimizing the number of projected images to save computing resources; as shown in Figure 1 Figure 3 As shown, method 2 adds an additional view focused on the upper surface of the aggregate on the basis of the three-view scheme to provide a more comprehensive aggregate representation. This includes views focused on the upper surface of the aggregate to avoid occlusion, and the introduction of tilted views to enrich view disparity information and enhance the depth perception of the model. Specifically, there are nine views. When determining the projection strategy, the position of the viewpoint is used to determine the direction of the coordinate area, and the viewpoint is specified by azimuth and elevation to obtain the desired projected image.

[0037] 3D point cloud data collection, 3D point cloud data interpolation and segmentation:

[0038] Data interpolation is applied to restore the missing 3D point cloud data. In this embodiment, the B-spline interpolation method is used for data recovery. When performing B-sample interpolation on the point cloud data, the unknown control points are first found using an inverse algorithm based on the known point cloud data, the B-sample curve is determined, and then the missing point cloud position is substituted to calculate the missing point cloud value.

[0039] The interpolated data needs to be segmented into individual aggregates; global threshold segmentation is applied to select pixels with grayscale between the minimum grayscale value and the maximum grayscale value. These pixels will be combined into a continuous connected area, that is, a Region. This Region represents a specific part of the image, and all pixels with grayscale values ​​not within this range will be excluded from this area and will not participate in the formation of the Region. Finally, the aggregate sample point cloud database is obtained.

[0040] The aggregate samples selected in this example include four particle size categories, which are divided according to the "Test Methods for Aggregates in Highway Engineering" (JTG 3432-2024), specifically 4.75 mm, 9.5 mm, 13.2 mm and 16 mm. Each particle size category represents a specific particle size range:

[0041] 4.75mm grade: 100% of the aggregate particles pass through the 9.5mm sieve, and less than 10% of the particles are retained on the 4.75mm sieve;

[0042] 9.5mm grade: 100% of the aggregate particles pass through the 13.2mm sieve, and less than 10% of the particles are retained on the 9.5mm sieve;

[0043] 13.2mm grade: 100% of the aggregate particles pass through the 16mm sieve, and less than 10% of the particles are retained on the 13.2mm sieve.

[0044] 16mm grade: 100% of the aggregate particles pass through the 19mm sieve, and less than 10% of the aggregate particles are retained on the 16mm sieve.

[0045] A 3D point cloud data set is generated by collecting aggregate samples that meet the above-mentioned particle size standards and using a 3D scanning device. There will be M 3D point cloud data samples in each category to ensure coverage of different forms and distribution characteristics to improve the generalization ability of the model.

[0046] Image Matching:

[0047] The two projection methods designed for aggregate multi-view acquisition are used to process the restored and segmented 3D point cloud data respectively to obtain the nine-view and three-view maximum projection contour image libraries; then the background of the image is replaced with black, and the field of view size of the virtual camera is set to obtain a view size of 256x256;

[0048] From the point cloud database of M aggregate samples containing four particle size categories, the test aggregate point cloud is selected. The test aggregate point cloud is randomly selected from the entire point cloud database according to a certain proportion to verify the prediction performance of the model and ensure that the sample is representative and can fully reflect the characteristics of different particle size levels.

[0049] The three-view and nine-view projection contour images corresponding to the point cloud are selected from the nine-view and three-view maximum projection contour libraries, where the correspondence between the nine-view and three-view images means that the three-view and nine-view projection contour images of each test aggregate point cloud are generated based on the same aggregate sample point cloud.

[0050] Feature extraction:

[0051] Each projection of the test set 3D point cloud is input into the graph convolutional neural network to extract features. The features obtained from all projections are represented by the structure of the graph. Each view is regarded as a node in the graph, and the spatial dependencies between views are captured by the edges. The high-dimensional feature representation of the test set point cloud is aggregated. The specific steps of extracting deep feature representation and constructing the graph structure using the graph convolutional neural network are as follows:

[0052] This embodiment adopts a deep residual learning framework as the basic network structure to perform feature extraction on three-view and nine-view projection contour data. The basic network stacks multiple residual units, each of which is composed of several convolutional layers and an identity mapping (jump connection) to achieve effective feature extraction. Specifically, the jump connection in the residual unit can enable the input feature to be directly passed to the output, thereby alleviating the gradient vanishing problem and accelerating network convergence.

[0053] In this network structure, the projection contour data of three views and nine views are input into the convolutional network for feature extraction. The extracted features are processed by multiple residual units to obtain rich and efficient deep feature representation. These deep features are further optimized through the self-attention mechanism. The self-attention mechanism captures the dependencies between features through the interaction and re-weighting between feature vectors, highlights important features, and suppresses irrelevant information, so that the view obtains enhanced feature representation. This optimization process significantly improves the accuracy and robustness of the feature representation.

[0054] This embodiment uses a view-based graph convolutional network structure to process the enhanced feature representation view, which is specifically used for 3D shape analysis; the processing flow is as follows:

[0055] First, the aggregate point cloud data of each view is projected onto a two-dimensional plane to generate feature representations of multiple views. Then, these features are input into a graph convolutional network for processing. The graph convolutional network constructs a multi-view structure and uses graph convolution operations to capture the correlation and feature interaction between views. Each view is regarded as a node, and the edges between nodes represent the correlation between views, thereby transmitting and fusing information on the features of each view. After feature processing, the features of all views are gradually fused into a unified high-dimensional feature representation through a multi-level feature aggregation strategy. This process can effectively capture the spatial information and global geometric features of multi-view data, providing a richer and more complete feature description for subsequent analysis.

[0056] The model output vector of the view-based graph convolutional network structure is used as the extracted feature shape for category classification, and feature extraction and classification are repeated until the 3D multi-view data of all M aggregate samples in the aggregate multi-view sample set are processed.

[0057] The method of the present invention was evaluated:

[0058] The number of positive samples correctly predicted as positive is recorded as TP, the number of positive samples incorrectly predicted as negative is recorded as FN, the number of negative samples correctly predicted as negative is recorded as TN, and the number of negative samples incorrectly predicted as positive is recorded as FP; the prediction accuracy of aggregate particle size is calculated according to the following formula, and the classification performance of the two types of data sets is evaluated using the Accuracy, Precision, Recall and F1-score indicators:

[0059]

[0060] The method of the present invention is subjected to simulation experiment:

[0061] Simulation conditions: The present invention uses the PyTorch framework for model training and experiments on a Windows 10 operating system with an Intel(R) Xeon(R) Gold 5118 CPU, an NVIDIA QuadroP5000 GPU, and 64GB of RAM. The aggregates in the data set come from the silos of a continuous mixing station and an aggregate production base, and the types of aggregate materials include granite, limestone, diabase, and gneiss.

[0062] The methods compared in the experiment are as follows:

[0063] The existing methods include: a multi-view classification method based on deep convolutional neural network, denoted as MVCNN in the experiment, and a multi-view classification method based on graph convolutional network, which uses graph structure to model the relationship between multiple perspectives, denoted as View-GCN in the experiment;

[0064] Simulation content: According to the specific implementation method of this embodiment, the accuracy of aggregate particle size classification is calculated and compared with the MVCNN model and the View-GCN model. The results are shown in Tables 1 and 2.

[0065] Table 1 Comparison of different multi-view classification models on nine-view projection data

[0066]

[0067] Table 2 Comparison of different multi-view classification models on three-view projection data

[0068]

[0069] It can be seen from the experimental results in the above table that the model used in this embodiment effectively models the relationship between views through the message passing mechanism, and fuses multi-view features using the graph structure, so as to fully mine additional view information; through the three-layer GCN structure, the details and texture information are further integrated to generate an accurate global feature representation, which significantly improves the classification accuracy and verifies the advantages of the present invention.

[0070] The principles and implementation methods of the present invention are described in this article using specific embodiments. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for predicting aggregate particle size based on three-dimensional multi-views, characterized in that: include: 3D point cloud data collection, collecting 3D point cloud data of aggregates; Image restoration and segmentation: using an interpolation method to restore the three-dimensional point cloud data of the aggregate, segmenting the restored three-dimensional point cloud data of the aggregate into individual particles to obtain an aggregate sample point cloud database; Image matching, performing aggregate multi-view acquisition based on the aggregate sample point cloud database to obtain an aggregate sample image set; selecting a test aggregate point cloud from the aggregate sample point cloud database, and then selecting an image corresponding to the test aggregate point cloud from the aggregate sample image set; Feature extraction: After image matching is completed, feature extraction is performed on the matched image to obtain a high-dimensional feature representation of the aggregate point cloud and output the feature shape; Category classification: classify the particles according to the characteristic shapes to complete the particle size classification.

2. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 1, characterized in that: The image restoration and segmentation specifically include: The B-spline interpolation method is used to restore 3D point cloud data. First, the unknown control points are found using the inverse algorithm based on the known point cloud data, the B sample curve is determined, and then the missing point cloud positions are substituted to calculate the missing point cloud values. The global threshold is used to segment the restored three-dimensional point cloud data of the aggregate into individual particles. Specifically, the pixel points with grayscale between the minimum grayscale value and the maximum grayscale value are selected, and the selected pixel points are combined into a continuous connected area, that is, the Region area.

3. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 1, characterized in that: The aggregate multi-view acquisition specifically includes: Select two view projection methods, one is the three-view method, including the front view, top view and side view; The other is to add additional views focusing on the upper surface of the aggregate and oblique views on the basis of the three views; The above two view projection methods are used to obtain the maximum projection contour image library.

4. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 3, characterized in that: The image background in the maximum projection contour image library is black, and the image size is a view of 256x256.

5. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 3, characterized in that: The image matching includes: The projection contour images of the test aggregate point cloud in the projection contour images respectively obtained by the two view projection methods need to correspond one to one.

6. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 5, characterized in that: The feature extraction comprises: The projection contour map corresponding to the selected test aggregate point cloud is subjected to feature extraction through a convolutional neural network to obtain multi-view features, and then the multi-view features are hierarchically aggregated to obtain a high-dimensional feature representation of the test aggregate point cloud and output the feature shape.

7. The method for predicting aggregate particle size based on three-dimensional multi-view according to claim 6, characterized in that: The categories include: Repeat the feature extraction until all aggregate samples in the aggregate sample point cloud database are processed to obtain a plurality of characteristic shapes, and complete the aggregate particle size classification according to the categories of the plurality of characteristic shapes.

Images