Stone particle size classification method and system based on deep learning, and storage medium

Abstract

The application discloses a kind of stone particle size classification method, system and storage medium based on deep learning.The stone pile in simulated environment is acquired and the height information of stone pile is acquired;N*N grid image splicing is used to remove the interference of height difference existing in different regions of stone pile in combination with laser triangulation;Fourier transform is used to convert N grid stone image to frequency domain;The three-dimensional spectral graph of stone after high-pass filter is used as classification model dataset;The ResNet34 model is improved, the residual structure is optimized, the hollow convolution is added, the ECA module is introduced, the three-dimensional spectral graph dataset of stone after high-pass filter is put into the improved ResNet34 model for training, and finally used for stone particle size classification.The application is accurate in stone particle size classification task under the condition that the height difference of stone pile is different, the light weather and other factors are different, suitable for various stone, discriminates correctly, and has good robustness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a stone particle size classification method and system based on deep learning and a storage medium. BACKGROUND

[0002] In the actual scene of a construction site, in order to ensure the construction quality, the particle size of the stone must be clearly required. When the stone is transported into the site by a truck, it should be stacked according to different particle size grades. In subsequent use, the stones of the three particle size grades are mixed according to a specific ratio so as to achieve the required density of the site in actual application, which is called graded stone. The traditional stone particle size detection method is generally through manual sampling, which indeed has good accuracy for a small number of samples. However, in a large-scale complex stone transportation scene, the manual sampling method has great limitations and is easily affected by worker fatigue, weather conditions and poor visibility at night, which can cause a large error in the result. Obviously, the manual sampling method cannot meet the actual needs of today's construction site.

[0003] In recent years, with the continuous improvement and development of computer vision, deep learning and other technologies, their application fields are also expanding. At present, the mainstream target classification technology has been widely used in other fields, but it is still lacking in the task of stone particle size classification in complex environments. The main reason is that there are many types of stones in these scenes, which are greatly affected by distance factors and disturbed by factors such as light and weather. At present, some research institutions and enterprises are exploring the use of digital image processing technology and machine vision technology to detect stone aggregates. For example, Li Xu et al. used image processing technology to extract the length-width ratio coefficient of concrete coarse aggregate to realize the detection of concrete coarse aggregate. Liu Chun et al. used morphological processing method to segment the stone particles and pores, and then extracted the contour information of the stone particles to further calculate their size and other characteristics. However, the pure digital image processing method has strict requirements on the light environment and is not suitable for processing the task of stone particle size classification in complex environments.

[0004] CNN is widely used in target classification field due to its good robustness. Zhu Daqing, Cao Guo, etc. proposed a two-stage deep learning-based sandstone image particle size detection method in 2020. This algorithm can effectively segment closely adhered sandstone targets and improve the accuracy of sandstone target particle size calculation. However, the accuracy of this algorithm is affected by the particle size of the stone particles and the construction environment, and in the actual site scene, it is not necessary to accurately calculate the specific value of the stone particle size, but only to distinguish the grade of the stone particle size, which can greatly reduce the consumption of hardware resources and time. Yan Ran et al. applied CNN model to the sandstone aggregate classification task in 2021 and achieved a classification accuracy of more than 99%.

[0005] The latest research is that Yan Ran et al. applied CNN model to the sand and gravel aggregate classification task in 2021 and achieved more than 99% classification accuracy. Although this method solves the influence of surface wetness on the classification result, it does not consider the influence of stone type, stone pile height difference information and light information on the final classification result in the actual application scene. Therefore, the current problems are summarized as follows:

[0006] (1) Because the distance between the stones in different positions in the truck carriage and the camera lens is not the same during image shooting, there is a height difference, so the pixels occupied by stones of the same particle size in the final image have a large deviation. In addition, the particle size distribution of the required fine sieve is relatively close. If the influence of the stone pile height difference factor is not considered, the final classification result of the stone particle size will be misjudged, the three grades of stone will be placed in disorder, and finally irreversible economic losses will be caused.

[0007] (2) Because the color and edge shape of the stones transported by different gravel manufacturers are quite different, and the weather and light conditions of the construction site are quite different, the accuracy of the general image classification model in this scene is poor. SUMMARY

[0008] 1. The purpose of the present application

[0009] In order to solve the problem of poor accuracy of existing computer vision and deep learning stone classification algorithms, and to solve the errors caused by factors such as multiple types of stones, different stone pile height differences, and different light and weather conditions in complex construction site scenarios, a stone particle size classification method, system and storage medium based on deep learning are proposed.

[0010] 2. The technical solution adopted by the present application

[0011] The present application discloses a stone particle size classification method based on deep learning, specifically:

[0012] Obtain the stone pile in the actual construction site environment and obtain the stone pile height information;

[0013] Use N*N grid image splicing combined with laser triangulation to remove the height difference interference in different areas of the stone pile;

[0014] Convert the N*N grid stone image to the frequency domain using Fourier transform;

[0015] Use the stone three-dimensional frequency spectrum graph after high-pass filtering as the classification model data set;

[0016] The ResNet34 model is improved, the residual structure is optimized, that is, two convolution kernels are optimized into three convolution kernels, a hollow convolution is added, an ECA module is added for each channel to adjust the weight, and the stone three-dimensional spectral data set after passing through the high-pass filter is put into the improved ResNet34 model for training, and finally the stone particle size is classified.

[0017] Specifically, the residual structure is optimized, two convolution kernel sizes of 3*3 convolution layers originally used for feature transformation are optimized into a 1*1 convolution layer reducing the number of channels, a 3*3 convolution layer for feature transformation, and another 1*1 convolution layer restoring the number of channels in series, the hollow convolution adopts a hollow convolution with a hollow rate of 2.

[0018] Specifically, the stone pile in the simulated environment is photographed and the height information of the stone pile is obtained, the stone pile passes below the industrial camera at a speed v, the laser is turned on and the industrial camera is controlled to continuously take N*N positions of the stone pile under short exposure time and normal exposure time, a total of 2N*N stone images.

[0019] Specifically, the height information of the laser spot center position O is obtained from the short exposure time image, and the height information of the laser spot center position is calculated by using the laser triangulation method in the corresponding normal exposure time image.

[0020] Further, the position of the laser is recorded as A, the position of the industrial camera is recorded as B, the center point in the camera imaging plane is recorded as C, and the image of the laser spot center in the camera imaging plane is recorded as D, so according to the normal incidence laser triangulation, the following relationship can be obtained:

[0021]

[0022] From equation (1), it can be deduced that the distance OA between the laser point and the plane of the camera and the laser is:

[0023]

[0024] Where AB represents the horizontal distance between the camera and the laser, BC represents the focal length of the camera, and CD represents the distance between the laser spot imaging point and the image pixel center in the imaging plane, which can be calculated by multiplying the pixel difference between the laser spot imaging point and the image pixel center by the pixel size. In this step, the height information of N*N laser spot positions can be calculated.

[0025] Further, the N*N grid stone image is obtained by intercepting and magnifying and interpolating the region of the stone image, and then merging; an M*M pixel square region centered on the laser spot position is intercepted in the stone image under normal exposure time, and the M*M pixel images under N*N heights are merged into a (N*M)*(N*M) pixel nine-grid image.

[0026] Further, before image merging, the M*M pixel stone image under different heights needs to be magnified and interpolated, the obtained height information is scaled with the ratio of the reference height, the magnified image pixels are calculated, the M*M pixel image is magnified by using the bilinear interpolation method, and then the M*M pixel image is intercepted for image merging.

[0027] Further, the spatial domain information of the N*N grid stone image is converted into frequency domain information through discrete Fourier transform to obtain a three-dimensional spectrum of the N*N grid stone image for more comprehensive analysis and processing.

[0028] Further, a high-pass filter is used to filter the three-dimensional spectrum of the N*N grid stone image, the maximum particle size of the stone in the construction site is D, and a pixel K occupied by a stone with a particle size of D in the image is calculated by using a variant (3) of the normal incidence laser triangulation method.

[0029]

[0030] Wherein f represents the focal length of the camera, H represents the value of the distance OA of the stone from the plane of the industrial camera and the laser, size represents the pixel size, and the pixel K occupied by a stone with a particle size of D in the image can be obtained by inputting the data, and the cutoff frequency F can be calculated by inputting the Fourier transform formula, so it can be considered that the signal in the interval range of u, v≤F is interfering for classification, and therefore the cutoff frequency of the ideal high-pass filter is set to F. The three-dimensional spectrum of all N*N grid stone images after passing through the ideal high-pass filter is used as a data set of the classification model.

[0031] The improved ResNet34 model is improved, the residual structure is optimized, that is, two convolution kernels are optimized to three convolution kernels, a hollow convolution is added, an ECA module is added to each channel to adjust the weight, and the stone three-dimensional spectrum data set after passing through the high-pass filter is put into the improved ResNet34 model for training, and finally the classification of the target stone particle size is realized.

[0032] The application also provides a stone particle size classification system based on deep learning, which is characterized by: an industrial camera and a laser are used to scan a stationary or moving stone pile to be subdivided, and the method is used for classification and identification.

[0033] The application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the method steps.

[0034] 3、The beneficial effects of the application

[0035] (1) The method has excellent performance in the stone particle size classification task under different stone pile height differences, light weather and other factors, and the accuracy on the test set is 99.8%, and the loss function value is 0.012.

[0036] (2) The method is suitable for stone classification of various colors, and the stone image of different colors is classified and predicted in the frequency domain classification algorithm based on the stone pile height information, and the discrimination result is correct, and the method has good robustness. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 It is a stone particle size classification algorithm flowchart based on deep learning;

[0038] Figure 2 It is a device method schematic diagram;

[0039] Figure 3 It is a stone image acquisition method and a schematic diagram of intercepting and splicing after scaling according to height information;

[0040] Figure 4 It is a normal exposure and short exposure time image example, (a) normal exposure time image, (b) short exposure time image;

[0041] Figure 5 It is a schematic diagram of intercepting M*M pixel stone image;

[0042] Figure 6 It is a stone N*N grid image example under different light environments and whether the stone surface is wet or not;

[0043] Figure 7 It is a stone N*N grid image of different particle size grades;

[0044] Figure 8 It is a three-dimensional frequency spectrum diagram corresponding to the stone of different particle size grades;

[0045] Figure 9 It is a three-dimensional frequency spectrum diagram of large stone before and after ideal high-pass filter spectrum comparison diagram;

[0046] Figure 10 It is an improved ResNet34 network structure diagram;

[0047] Figure 11The training result of the stone three-dimensional spectrum data set after passing through the ideal high-pass filter in the ResNet34 network model;

[0048] Figure 12 The stone images of different particle size grades and various colors;

[0049] Figure 13 The stone image classification results of different particle size grades and different colors;

[0050] 1. Laser, 2. Industrial camera, 3. Laser path, 4. Camera main optical axis, 5. Laser indication spot, 6. Stone pile, 7. Truck. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0052] Figure 1 The flowchart of the stone particle size classification method in the present application is shown in the following detailed description of the embodiments of the present application in conjunction with the drawings.

[0053] 1. Obtain stone images:

[0054] As shown in Figure 2 , when the truck 7 loaded with stones passes under the industrial camera 2 at a speed v, the laser 1 is turned on and the industrial camera 2 is controlled to continuously take N*N positions (as shown in Figure 3 -a) of the stone pile 6 under the conditions of short exposure time and normal exposure time, a total of 2N*N stone images, and N*N M*M pixel regions 12 are intercepted; the height of the region is scaled 13 according to the ratio of the reference height, and the M*M pixel image in the scaled image c n is intercepted.

[0055] A specific image example is shown in Figure 4 . Figure 4 (a) The O' point shown in the short exposure time image is the imaging of the laser indication spot 5 in the industrial camera 2.

[0056] 2. Obtain the height information of the stone image laser spot by laser triangulation:

[0057] In order to solve the interference of the height difference h of the stone pile in the truck compartment on the final particle size classification result, the height information of the above photographed stone image area is first obtained. The laser spot center position O is obtained from the short exposure time image, and the height information of the laser spot center position is calculated in the corresponding normal exposure time image by using the laser triangulation method. The position of the laser is denoted as A, the position of the industrial camera is denoted as B, the center point in the camera imaging plane is denoted as C, and the image of the laser spot center in the camera imaging plane is denoted as D, so according to the normal incidence laser triangulation method, the following relationship can be obtained:

[0058]

[0059] It can be deduced from formula (1) that the distance OA of the laser point from the camera and the laser plane is:

[0060]

[0061] Where AB represents the horizontal distance between the camera and the laser, BC represents the focal length of the camera, and CD represents the distance between the laser spot imaging point and the image pixel center in the imaging plane, which can be calculated by multiplying the pixel difference value of the laser spot imaging point and the image pixel center by the pixel size. In this step, the height information of N*N laser spot positions can be calculated.

[0062] 3. The stone image in the intercepted area is scaled and then merged into an N*N grid image

[0063] A M*M pixel square area centered on the laser spot position is intercepted in the stone image with normal exposure time, as shown in Figure 5 .

[0064] In order to reflect the particle size information of stones in different position areas through one image in the subsequent training process, the present application adopts an image merging method to merge the M*M pixel images at N*N heights into an N grid image with (N*M)*(N*M) pixels as shown in Figure 3 -c. Before image merging, the M*M pixel stone images at different heights need to be scaled, and the present application calculates the scaled image pixels and uses the bilinear interpolation method to enlarge the M*M pixel image, and then intercepts the M*M pixel image for image merging.

[0065] In order to avoid the influence of factors such as light weather on the result, the present application obtains multiple groups of images in different light environments and whether the stone surface is wet or not as shown in Figure 6 , and scales and merges them into N*N grid images after calculating their height information according to the above method.

[0066] All the above combined (N*M)*(N*M) pixel images are named big, medium and small according to the particle size grade, and 60% of the images are set as the training set train and 40% of the images are set as the test set val, and finally the stone image dataset considering the height information and light weather factors is obtained, Figure 7 are different particle size grade stone N*N grid images.

[0067] 4. The N*N grid stone image is converted from the spatial domain to the three-dimensional spectrum graph in the frequency domain

[0068] The present application considers converting the spatial domain information of the N*N grid stone image into frequency domain information through discrete Fourier transform, so as to more comprehensively analyze and process the image. The original shape of the three-dimensional spectrum graph can reflect more information about the particle size of the stone from the frequency domain.

[0069] 5. The stone three-dimensional spectrum graph is filtered through a high-pass filter

[0070] The present application filters the three-dimensional spectrum graph in step 4 using a high-pass filter. The present application considers that the particle size of the stone in the construction site is at most D. A stone with a particle size of D occupies K pixels in the image, which is calculated using a variant of the normal incidence laser triangulation method (3).

[0071]

[0072] Where f represents the camera focal length, H represents the value of the distance OA of the stone from the industrial camera and the laser plane, size represents the pixel size, and the data can be brought in to obtain the pixel K occupied by a stone with a particle size of D in the image. The Fourier transform formula can be used to calculate the cutoff frequency F, so it can be considered that the signal in the interval range of u, v≤F is interfering with the classification, so the cutoff frequency of the ideal high-pass filter is set to F. Taking a large stone as an example, the stone spectrum before and after the above high-pass filter is as shown in the figure Figure 9 From the figure, it can be seen that the three-dimensional spectrum graph obtained by the ideal high-pass filter with a cutoff frequency of F can show more details, and there is no problem of spectrum compression and distortion. Therefore, after the stone N*N grid image dataset is subjected to the above operation, the stone three-dimensional spectrum graph is named big, medium and small according to the particle size grade, and 60% of the images are set as the training set train and 40% of the images are set as the test set val, to obtain the stone three-dimensional spectrum graph dataset after the high-pass filter.

[0073] 6. Improvement of ResNet34 model

[0074] The model of ResNet34 is improved from three aspects of optimizing residual structure, adding a hollow convolution and introducing an ECA module to increase the classification accuracy of the model and reduce the operation parameter amount, and the improved ResNet34 model structure is as shown in Figure 8

[0075] (1) Optimizing residual structure: two convolution layers with a kernel size of 3*3 originally used for feature transformation are optimized into a 1*1 convolution layer, a 3*3 convolution layer and another 1*1 convolution layer in series. The first 1*1 convolution layer is used to reduce the number of channels of the input feature map, so as to reduce the number of network parameters and the amount of calculation. The 3*3 convolution layer after that realizes feature transformation and maps the features through a nonlinear activation function. The last 1*1 convolution layer is used to restore the number of channels of the feature map to the original dimension.

[0076] (2) Adding a hollow convolution: generally, the receptive field can also be expanded by increasing the number of convolution layers and reducing the down-sampling step, but this will increase the amount of calculation and reduce the resolution of the feature map, while the hollow convolution can obtain a larger receptive field without increasing the amount of calculation. The implementation method of the hollow convolution is to add a certain number of holes in the convolution kernel of the ordinary convolution, and the holes are usually filled with zeros. Assuming that the convolution kernel size is k*k and the hole rate is r, r-1 zeros are filled in the middle of the convolution kernel, that is, the original convolution kernel becomes (k+(k-1)×(r-1))×(k+(k-1)×(r-1)) in size, and the hollow convolution with a hole rate of 2 is adopted in the application.

[0077] (3) Introducing an ECA module: ECA is an efficient channel attention module, which can adaptively adjust the weight of each channel in the convolution feature map, so that the network pays more attention to important channel information, thereby improving the detection accuracy. It mainly includes three aspects of strengthening channel features, suppressing redundant features and improving feature expression ability. Global average pooling is used to compress the feature values of each channel to obtain the importance of the channel features, which are then input into the sigmoid function to obtain the channel attention coefficient between 0 and 1. This mechanism can strengthen the features of each channel, so that the network pays more attention to important channel information. The ECA module can suppress redundant features, that is, features that do not contribute to classification or detection results. Because in the process of global average pooling, the feature values of the channels corresponding to the redundant features will be compressed to small values, so that the corresponding channel weight is low, so the influence of the redundant features can be suppressed. The ECA module can improve the expression ability of the feature map, so as to better capture the key information in the image. In the convolutional neural network, the feature map has a very important influence on the classification or detection result, and the ECA module can improve the expression ability of the feature map, so as to better capture the key information and improve the detection accuracy. ​

[0078] 7. Train the improved ResNet34 model by feeding the 3D spectrogram dataset of stones after passing through the high-pass filter into the model:

[0079] The 3D spectrogram dataset of stones after passing through a high-pass filter is fed into an improved ResNet34 network for training, resulting in a deep learning-based stone size classification algorithm that can effectively detect and classify stone sizes under different lighting and weather conditions.

[0080] Example:

[0081] The height difference h during truck transport of gravel is 0.95, N is 3, M is 200, the maximum gravel size is D, which is 6cm, the camera focal length BC is f, which is 12mm, and the pixel size is 2.4μm. The horizontal distance AB between the camera and the laser is 90mm.

[0082] ① Obtain the image of the stone:

[0083] The hardware components of this invention include a Hikvision MV-CE060-10UM industrial camera, a Hikvision MVL-HF1228M-6MPE camera lens, and a 650nm, 30mW red dot laser from Shenzhen Infrared Laser Technology Co., Ltd.

[0084] Based on the Guangzhou Municipal Construction Waste Transport Vehicle Marking and Monitoring Terminal, and the specifications and sealing standards of the truck bodies, it is known that the maximum height difference h when a truck transports gravel in an actual construction site scenario should be around 0.95 meters. This project arranged the gravel scenario in a laboratory simulation environment as follows: three particle size grades were placed sequentially on an inclined surface, with the height difference between the lowest and highest points of the gravel controlled at 0.95 meters. The gravel pile was pushed at a certain rate, and the laser switch was turned on. This invention subsequently utilized a nine-grid image for analysis, repeatedly controlling the exposure time in the MVS client, and using a Hikvision industrial camera to capture and store eighteen images from the bottom to the top.

[0085] ②Use laser triangulation to obtain height information of different stone areas.

[0086] The pixel coordinates of the laser center point were obtained from the short exposure time image. Then, the height information of the nine laser spot positions was calculated from the normal exposure time image using laser triangulation. The results are summarized in the table below, in mm:

[0087] Table 1. Summary of Height Information from Nine Points for Stones of Different Size Grades

[0088] [A1] [A2] [A3] [A4] [A5] [A6] [A7] [A8] [A9] Small stones 1845 1936 2047 2145 2204 2310 2507 2609 2795 Medium stones 1868 2020 2155 2242 2326 2476 2536 2648 2818 Large stones 1867 2033 2100 2320 2417 2500 2602 2701 2817

[0089] ③ After scaling, synthesize an N*N grid of pebble images.

[0090] To address the aforementioned scenario, this invention calculates that regions within a 200*200 pixel area have consistent height information. A 200*200 region centered on the laser point is extracted, and stone images at the same height are also extracted into a stone image folder at that height.

[0091] After the above steps, this invention has obtained 200*200 pixel images of stones of different sizes at different heights. To enable a single image to reflect information about stones in different locations during subsequent training, this invention employs an image merging method, combining the nine 200*200 pixel images at different heights into a single 600*600 pixel nine-grid image. Before image merging, the 200*200 pixel stone images at different heights need to be scaled, using a plane 1845mm away from the laser and industrial camera plane as a reference. By calculating the height information ratio, this invention calculates the magnified image pixels and uses bilinear interpolation to magnify the 200*200 pixel image, then extracts a 200*200 pixel portion for image merging.

[0092] To avoid the influence of factors such as lighting and weather on the results, this invention acquired multiple sets of images under different lighting conditions and with varying degrees of moisture on the stone surface. The height information was calculated using the method described above, and then the images were scaled and merged into a nine-grid image. All the merged 600*600 pixel images were named "big," "medium," and "small" according to their particle size. 60% of the images were used as the training set (train), and 40% were used as the test set (val), resulting in a dataset of stone images that considers height information and lighting / weather factors.

[0093] ④ Transform the nine-square grid of stones from the spatial domain into a three-dimensional spectrum in the frequency domain.

[0094] The spatial domain information of the image is transformed into frequency domain information using a two-dimensional discrete Fourier transform, allowing for further processing of the nine-square grid pebble image.

[0095] ⑤ Pass the three-dimensional spectrum of the stones through a high-pass filter

[0096] The application adopts a high-pass filter to filter the three-dimensional spectrum diagram. In the classification task, the maximum particle size of the stone is 4 cm, and considering a certain error, it is considered that the particle size D of the stone in the construction site cannot exceed 6 cm. By inputting the data, it can be obtained that the pixels M occupied by a stone with a particle size of 6 cm in the image is about 100, that is, T, Z = 50. It can be considered that the interval range of u, v ≤ 50 is the signal that interferes with the classification, and therefore the cut-off frequency of the ideal high-pass filter is set to 50. Therefore, the stone nine-square image data set obtained after the above operation is named big, medium and small according to the particle size grade, and 60% of the images are set as the training set train, and 40% of the images are set as the test set val, to obtain the stone three-dimensional spectrum diagram data set after the high-pass filter.

[0097] ⑥Put the stone three-dimensional spectrum diagram data set after the high-pass filter into the improved ResNet34 model for training

[0098] The improved ResNet34 network structure is as shown in Figure 10 The stone three-dimensional spectrum diagram data set after the high-pass filter is put into the improved ResNet34 model for training, the Adam optimizer is selected as the optimizer, and the initial learning rate is set to 0.0001. The batch size is set to 16, and the iteration number is set to 50. And real-time monitoring of various indicators in the training and testing process, including accuracy and loss function, etc., and finally output in the form of a curve.

[0099] Figure 11 The loss function value and accuracy of the current training set and the loss function value and accuracy of the test set after each epoch of the improved ResNet34 network model training are given.

[0100] From Figure 11 It can be obtained that the stone three-dimensional spectrum diagram data set after the high-pass filter reaches the optimal result in the improved ResNet34 network model after about 14 rounds of training, and the final accuracy on the training set is 99.9%, and the loss function value is 0.010, and the accuracy on the test set is 99.8%, and the loss function value is 0.012.

[0101] ⑦Result analysis of stone particle size classification algorithm based on deep learning

[0102] From the above experimental results, it can be seen that the stone particle size classification algorithm based on deep learning proposed by the application has excellent performance in the stone particle size classification task under the condition that the stone pile height difference, light weather and other factors are different.

[0103] In order to test the robustness of the algorithm, different particle size grades and various colors of stones are purchased, such asFigure 12 As shown.

[0104] In the case of not training the pebble color factor, the images of pebbles of multiple colors are put into the frequency domain classification algorithm based on the height information of the pebble pile for classification prediction, and the prediction result is as shown in Figure 13 As shown.

[0105] As shown in Figure 11 It can be seen that the algorithm correctly distinguishes pebbles of different colors, which shows that the pebble size classification algorithm based on deep learning proposed by the present application has good robustness, and the algorithm can solve the problems caused by the height difference of the pebble pile, the light condition, the weather and other factors in the actual construction site.

[0106] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A deep learning-based stone particle size classification method, characterized in that: acquiring a stone pile in an actual construction site environment and acquiring stone pile height information; Utilizing The grid image splicing combines with the laser triangulation method to remove the height difference interference existing in different regions of the stone pile ; the height information of the laser spot center position is calculated by using the laser triangulation method; and the height information is used as the position height information of the central region of the corresponding normal exposure time image. Using Fourier transform Convert the grid-like pebble image to the frequency domain. The spatial domain information of the grid-like pebble image is transformed into frequency domain information through discrete Fourier transform, yielding... 3D spectrogram of a grid of pebbles; improve the ResNet34 model by optimizing the residual structure, adding a hollow convolution, and introducing an ECA module; put the stone three-dimensional spectrum graph after the high-pass filter into the improved ResNet34 model for training, and finally realize the classification of the target stone particle size. 2.The deep learning-based stone particle size classification method according to claim 1, characterized in that: The optimization residual structure is specifically: two convolution kernel sizes of used for feature transformation are optimized into a convolution layer with a reduced channel number of , a convolution layer for feature transformation with a size of , and another convolution layer with a restored channel number of , which are connected in series, and the hollow convolution adopts a hollow rate of 2. 3.The deep learning-based stone particle size classification method according to claim 1, characterized in that: The acquisition of the stone pile in the actual construction site environment and the acquisition of the stone pile height information are specifically: the stone pile moves at a speed Passing under the industrial camera, turning on the laser and controlling the industrial camera to continuously take pictures of the stone pile Each position is respectively photographed with a short exposure time and a normal exposure time, and a total of Stone images.

4. The deep learning-based stone particle size classification method according to claim 1, characterized in that: The height information of the laser spot center position is calculated by using the laser triangulation method, specifically, the position of the laser is recorded as , the position of the industrial camera is recorded as , the center point in the camera imaging plane is recorded as , and the image of the laser spot center in the camera imaging plane is recorded as , so according to the normal incidence laser triangulation method, the following relationship is obtained: From equation (1) the distance of the laser point from the camera and the laser plane is derived is: wherein represents the horizontal distance of the camera from the laser; represents the focal length of the camera; represents the distance of the laser spot imaging point from the center of the image pixel in the imaging plane, CD is calculated by the pixel difference of the laser spot imaging point from the center of the image pixel multiplied by the pixel size; A corresponding OA is calculated for each position from N A corresponding OA is calculated for each position from N The height information of N laser spot positions. 5.The deep learning-based stone particle size classification method according to claim 1, wherein: The use of Grid image stitching, specifically: cropping an image centered on the laser spot position from a pebble image with a normal exposure time. Pixel square area, as Pixel-level stone images; at different heights The pixel-level stone image is scaled based on the ratio of the acquired height information to a reference height to calculate the magnified image pixels, and then bilinear interpolation is used to... The image of the pebble was enlarged to a pixel size, and then a portion of it was extracted. A pixel image, as a magnified version A pixel-sized image of pebbles; Magnified at a height The images of the pebbles were combined into one. A pixelated image of pebbles. 6.The deep learning-based stone particle size classification method according to claim 1, wherein: the stone three-dimensional spectrum graph after the high-pass filter is put into the improved ResNet34 model for training, specifically: Using a high-pass filter The 3D spectrogram of the grid-like pebble image was filtered, and the maximum pebble size at the construction site was [missing value]. D Using a variation (3) of the normal incident laser triangulation method, a particle with a diameter of [missing value] was calculated. D The number of pixels occupied by the pebbles in the image K ; wherein f represents the camera focal length, H represents the distance of the stone from the plane of the industrial camera and the laser OA , size represents the pixel size, and the data is brought in to obtain a block of stone with a particle size of D pixels in the image K , the Fourier transform formula is brought in to calculate the cut-off frequency F , so it is considered that the signal in the interval range of u, v≤ F interferes with classification, so the cut-off frequency of the ideal high-pass filter is set to F ; All through the high-pass filter after The three-dimensional spectrum of the grid stone image is put into the improved ResNet34 model for training.

7. A computer readable storage medium having stored thereon a computer program, characterized in that: The computer program is executed by the processor to realize the method steps of any one of claims 1-6.

Images