An aggregate detection and classification method based on AI intelligent machine vision technology

Abstract

The present invention discloses an aggregate detection and classification method based on AI intelligent machine vision technology, comprising the following steps: S1, using a high-definition camera to shoot aggregate transportation video, and transmitting it in real time through a 5G network to obtain an aggregate image; S2, processing the aggregate image through an image edge segmentation processing method to extract edge features; S3, manually marking the aggregate category of the processed aggregate image; S4, inputting the marked aggregate image into a machine vision algorithm, training it, extracting features, and outputting it as a weight file; S5, placing the trained weights in a detection system to detect and identify the aggregate. Aggregate images are obtained in real time through a 5G network using a camera, computer image processing technology is applied to process the video, and different types of aggregate status images are detected and identified in combination with an AI intelligent algorithm to solve the problem of complex operation of existing aggregate monitoring methods, while also realizing the function of real-time processing.

Description

Technical Field

[0001] The present invention belongs to the technical field of aggregate detection, and in particular relates to an aggregate detection and classification method based on AI intelligent machine vision technology. Background Art

[0002] Aggregate is an important component of concrete. The classification of sand and gravel aggregate is an important factor in determining the performance and quality of concrete. Ordinary concrete (referred to as concrete) is composed of cement, sand, stone and water, and aggregate generally accounts for 70% to 80% of concrete. There are many factors that affect the strength of concrete, mainly the influence of cement strength and water-cement ratio, the influence of aggregate grading and shape, the influence of curing temperature, the influence of curing cycle, etc. The aggregate processing system is one of the main auxiliary production systems in the construction of large-scale water conservancy and hydropower projects. The quality improvement of sand and gravel aggregates is of great significance to promoting the healthy development of the engineering construction industry. It is also extremely important to improve the quality of the project and optimize the cost of the project. Different sand and gravel aggregates have different effects on the performance of concrete. For the particle size and shape of aggregates, the current specifications for coarse aggregate needle-shaped particles are relatively broad, and good quality aggregates must have standardized particle size and shape. Therefore, we must ensure the quality requirements of sand and gravel aggregates and reasonably select raw materials to ensure the quality of concrete. Therefore, it is particularly important to find a suitable aggregate classification detection method.

[0003] The classification of sand and gravel aggregates is an important prerequisite for solving the proportion of concrete raw materials. Many technical issues related to image processing will be involved in the preparation and operation of the work. There are many types of aggregates in concrete raw materials, complex detection conditions, inconsistent detection distances, weather reasons, light changes, changes in dryness and humidity, and distance changes will affect the detection effect of aggregates. Among them, finding aggregate characteristics is one of the important issues, and it is necessary to find a detection method suitable for identifying aggregate categories. Traditional methods for aggregate detection generally include screening methods and particle size recognition, but these methods have limitations in image recognition and real-time processing, are time-consuming and labor-intensive, and difficult to solve practical problems. They do not meet the requirements of fast, accurate and real-time processing, and cannot be fully promoted and applied. In recent years, aggregate detection and identification technology has developed methods with modern scientific and technological technologies, such as the application of convolutional neural networks, imaging methods, backlight and high-resolution camera measurements, and application overviews based on laser detection analysis. Since various methods and technologies have their own application conditions and limitations, no single method is omnipotent. Sometimes in practice, if only one parameter is used as the basis for interpretation, it may be difficult to achieve the desired effect. Therefore, in practice, a comprehensive aggregate testing method is generally used to improve the accuracy of aggregate testing.

[0004] With the development of science and technology, aggregate detection methods and detection equipment have made great progress and development, and the technical level of aggregate detection has been continuously improved. However, various detection methods have their own application premises and limitations. In practical applications, most of the current methods are just modernizing the instruments, and the methods themselves are not innovative. Therefore, it is sometimes difficult to achieve good results by using a single method in aggregate detection. Therefore, in the application of aggregate detection, scientific and technological personnel should select a reasonable combination of aggregate detection methods based on the combination of aggregate image features and machine vision technology.

[0005] At present, there is no special real-time monitoring design for aggregate detection in concrete production projects. In engineering construction, high-definition industrial monitoring equipment is generally installed to monitor the location and unloading status of aggregate transport vehicles. However, the existing monitoring equipment is generally distributed on the upper end of the transport vehicle and can only monitor, but cannot recognize and process images; in addition, after the monitoring data is obtained, it only requires the staff to judge the type of aggregate by human eyes, without more effective processing and analysis.

[0006] In view of the shortcomings of the prior art, the present invention provides an aggregate detection and classification method based on AI intelligent machine vision technology. The method uses a camera to obtain aggregate images in real time through a 5G network, and then applies computer image processing technology to process the video. Combined with AI intelligent algorithms, different types of aggregate status images are detected and identified to solve the problem of complex operation of existing aggregate monitoring methods, while also realizing real-time processing functions. Summary of the invention

[0007] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an aggregate detection and classification method based on AI intelligent machine vision technology.

[0008] The present invention provides the following technical solutions:

[0009] An aggregate detection and classification method based on AI intelligent machine vision technology includes the following steps:

[0010] S1, uses a high-definition camera to shoot aggregate transportation videos and transmits them in real time through the 5G network to obtain aggregate images;

[0011] S2, processing the aggregate image by an image edge segmentation processing method to extract edge features;

[0012] S3, manually labeling the aggregate categories of the processed aggregate images;

[0013] S4, inputting the annotated aggregate image into the machine vision algorithm for training, feature extraction, and outputting it as a weight file;

[0014] S5, put the trained weights into the detection system to detect and identify the aggregates.

[0015] Preferably, in step S2, the image edge segmentation processing method specifically includes the following steps:

[0016] A, import the aggregate image, grayscale the aggregate image, and then calculate the gradient map;

[0017] B, based on the gradient map, the watershed algorithm is used to process the image and obtain the edge line of the segmented image;

[0018] C, use image opening operation to remove small objects and burr interference that affects the target in the image;

[0019] D, perform adhesion segmentation to separate the objects that are stuck together.

[0020] Preferably, in step S4, the machine vision algorithm training process specifically includes the following steps:

[0021] a, 5 detection areas are selected for each annotated aggregate image;

[0022] b. Perform image convolution on each detection area to extract image features;

[0023] c. Upsample the image and restore the convolved feature map to the image;

[0024] d, use tensor splicing algorithm to integrate image features;

[0025] e. Output to the detection head. The detection head classifies the image and determines the classification result of the recognition image through the probability function;

[0026] f, recognition result, output the detection result of the image.

[0027] Preferably, in step a, the five detection areas are respectively the center point, upper left, upper right, lower left and lower right areas of the aggregate image.

[0028] Preferably, in step b, the image features include edge features, color features and texture features of the aggregate.

[0029] Preferably, in step c, the image is upsampled so that the image fits the size of the display area, and an interpolation method is used, that is, new elements are inserted between pixels based on the original image pixels using a suitable interpolation algorithm.

[0030] Preferably, in step C, a processing method is adopted in which a gradient function is modified so that the water collection basin only responds to the target to be detected, and threshold processing is performed on the gradient image to reduce over-segmentation caused by grayscale changes.

[0031] Preferably, the method further includes step: S6, testing new images, judging whether the sample size is sufficient according to the test results, and keeping the sample size as uniform as possible. If the accuracy of the weight for a certain category is lower than 90%, the corresponding sample size is increased and the features are re-extracted until the recognition rate of each type of aggregate reaches more than 90%, which meets the requirements.

[0032] Preferably, in step B, the calculation process of the watershed is divided into two steps, one is a sorting process, and the other is a flooding process. First, the grayscale level of each pixel is sorted from low to high, and then in the flooding process from low to high, the influence domain of each local minimum at the h-order height is judged and marked using a first-in-first-out (FIFO) structure. The watershed transform obtains the watershed image of the input image, and the boundary points between the watersheds are the watersheds. Obviously, the watershed represents the maximum point of the input image. Therefore, in order to obtain the edge information of the image, the gradient image is usually taken as the input image, that is,

[0033] g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5

[0034] In the formula, f(x,y) represents the original image and grad represents the gradient operation.

[0035] Preferably, in step C, in order to reduce the over-segmentation caused by the watershed algorithm, the gradient function needs to be modified. A simple method is to perform threshold processing on the gradient image to eliminate the over-segmentation caused by slight changes in grayscale.

[0036] g(x,y)=max(grad(f(x,y)),gθ)

[0037] Where gθ represents the threshold value.

[0038] The program adopts the method: use threshold to limit the gradient image to eliminate over-segmentation caused by small changes in grayscale values, obtain an appropriate amount of areas, and then sort the gray levels of the edge points of these areas from low to high, and then implement the flooding process from low to high. The gradient image is calculated using the Sobel operator.

[0039] The operator contains two sets of 3x3 matrices, one for horizontal and one for vertical. By performing a planar convolution with the image, the approximate values ​​of the horizontal and vertical brightness differences can be obtained respectively. The formula is as follows:

[0040]

[0041]

[0042] The approximate horizontal and vertical gradients of each pixel of the image can be combined using the following formula to calculate the magnitude of the gradient.

[0043]

[0044] Where A represents the original image, G x and G y Represent the images after horizontal and vertical edge detection respectively.

[0045] After preprocessing, the edge information is obtained and the minimum rectangular frame is selected.

[0046] Preferably, in step S4, the algorithm is used to perform the following operations:

[0047] Image classification: Input the image into a multi-layer convolutional neural network, which will output a feature vector and feed it back to the softmax unit to predict the type of image.

[0048] Positioning and classification: Use an algorithm to determine whether there is a detection target in the image, mark the location of the target, mark it with a bounding box, and identify and locate the target.

[0049] Object detection: Images can contain multiple objects, and some single images may have multiple objects of different classifications. Multiple objects can be detected.

[0050] The Softmax function, a normalized exponential function, is actually the logarithmic normalization of the gradient of a finite discrete probability distribution. In multinomial logistic regression and linear discriminant analysis, the input of the function is the result obtained from K different linear functions, and the probability formula for the sample vector x belonging to the jth category is:

[0051]

[0052] This can be viewed as a composite of K linear functions called the Softmax function.

[0053] The target positioning is represented by anchor boxes. The input image is divided into S*S grids, and n anchor boxes are generated for each small grid. The real frame of the image is IOU-calculated with the anchor box generated by the small grid where the center point of the image is located. The regressed box is the Bounding Box. When the anchor boxes are closer to the real width and height, the performance of the model is better. The k-means algorithm is used to cluster the width and height of representative shapes in the ground truth boxes of all samples in the training set, and dimension clustering is performed. Multiple anchor box groups with different numbers are clustered and applied to the model respectively, and finally the optimal anchor box group that compromises between the complexity of the model and high recall is found.

[0054] The Bounding Box formula is:

[0055]

[0056]

[0057] Where a w and a h is the width and height of the anchor box,

[0058] t w and t h The width and height directly predicted by the bounding box,

[0059] b w and b h is the actual width and height predicted after conversion,

[0060] This is the width and height of the output in the final prediction.

[0061] The loss function for object detection is as follows:

[0062]

[0063] In the above formula, N is the total number of categories, yi is the probability of the current category after the activation function, and yi* determines whether the prior box is responsible for the target (0 or 1), if so, it is 0, otherwise it is 1.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] (1) The present invention provides an aggregate detection and classification method based on AI intelligent machine vision technology. The method uses a camera to obtain aggregate images in real time through a 5G network, and then applies computer image processing technology to process the video. The AI ​​intelligent algorithm is combined to detect and identify different types of aggregate status images to solve the problem of complex operation of existing aggregate monitoring methods, while also realizing the function of real-time processing.

[0066] (2) The present invention provides an aggregate detection and classification method based on AI intelligent machine vision technology, which extracts aggregate edge information, texture features and aggregate particle size features from the collected aggregate images, processes the images through image processing technology, and pre-processes the images to make the features of the aggregate more obvious.

[0067] (3) The present invention provides an aggregate detection and classification method based on AI intelligent machine vision technology. By collecting each image and dividing it into five detection areas, it is possible to realize one image and five judgments, thereby realizing one-time recognition and multiple judgments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0069] Figure 1 It is a data collection roadmap for the present invention;

[0070] Figure 2 This is a flow chart of the intelligent aggregate detection and identification method of the present invention;

[0071] Figure 3 This is a schematic diagram of image processing and recognition of the present invention;

[0072] Figure 4 This is a flow chart of the edge segmentation of an aggregate image according to the present invention;

[0073] Figure 5 It is a schematic diagram of edge segmentation of the present invention;

[0074] Figure 6 It is a schematic diagram of the machine vision model of the present invention; DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] Example 1

[0078] See also Figure 1-6 As shown, the detection objects of the present invention are five types of aggregates, namely gravel, pebbles, machine-made sand, small machine-made sand and surface sand, among which the particle size of gravel is in the range of 3-4 cm; the particle size of pebbles is in the range of 2-3 cm; the particle size of machine-made sand is in the range of 1-2 cm; the particle size of surface sand is in the range of 0.1-0.5 cm; the collected aggregate images include those under different light conditions, dry and wet conditions of aggregates and different shooting distances.

[0079] Figure 1 This is a scene diagram of the present invention. The high-definition camera is located diagonally above the vehicle. When the transport vehicle is in place, the video is shot and the frame sequence pictures are extracted. The aggregate detection model system consists of two parts, one is the transport vehicle in place and the acquisition of high-definition camera images; the other part is the image data processing and result display part.

[0080] An aggregate detection and classification method based on AI intelligent machine vision technology includes the following steps:

[0081] S1, uses a high-definition camera to shoot aggregate transportation videos and transmits them in real time through the 5G network to obtain aggregate images;

[0082] S2, processing the aggregate image by an image edge segmentation processing method to extract edge features;

[0083] S3, manually labeling the aggregate categories of the processed aggregate images;

[0084] S4, inputting the annotated aggregate image into the machine vision algorithm for training, feature extraction, and outputting it as a weight file;

[0085] S5, put the trained weights into the detection system to detect and identify the aggregates.

[0086] In step S2, the image edge segmentation processing method specifically includes the following steps:

[0087] A, import the aggregate image, grayscale the aggregate image, and then calculate the gradient map;

[0088] B, based on the gradient map, the watershed algorithm is used to process the image and obtain the edge line of the segmented image;

[0089] C, use image opening operation to remove small objects and burr interference that affects the target in the image;

[0090] D, perform adhesion segmentation to separate the objects that are stuck together.

[0091] Figure 4 The following is a flow chart of the image edge segmentation technology of the present invention. First, the captured image is imported. The image is grayed. The second step is to process the image using the watershed algorithm to make the target edge more obvious. The third step is to use the image opening operation to remove small objects in the image and burr interference that affects the target. Finally, adhesion segmentation is performed to separate the objects that are stuck together.

[0092] Figure 5 This is a schematic diagram of the adhesion segmentation of the present invention, which can segment the edges of the aggregate.

[0093] In practical applications, considering the accumulation of aggregates, edge information is mainly distinguished by color light and dark boundaries. In the segmentation process, the similarity between adjacent pixels is used as an important reference, so that pixels with similar spatial positions and similar grayscale values ​​(gradient) are connected to form a closed contour.

[0094] Operation steps: grayscale the color image, then calculate the gradient map, and finally perform the watershed algorithm based on the gradient map to calculate the edge lines of the segmented image.

[0095] In real images, due to the presence of noise points or other interference factors, the watershed algorithm often results in over-segmentation, which is due to the presence of many small local extreme points. In order to solve the problem of over-segmentation, a watershed algorithm based on a marked image can be used, which is to guide the watershed algorithm through prior knowledge to obtain better image segmentation results.

[0096] In step B, the calculation process of the watershed is divided into two steps, one is the sorting process, and the other is the flooding process. First, the gray level of each pixel is sorted from low to high, and then in the flooding process from low to high, the influence domain of each local minimum at the h-order height is judged and marked using a first-in-first-out (FIFO) structure. The watershed transform obtains the watershed image of the input image, and the boundary points between the watersheds are the watersheds. Obviously, the watershed represents the maximum point of the input image. Therefore, in order to obtain the edge information of the image, the gradient image is usually taken as the input image, that is,

[0097] g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5

[0098] In the formula, f(x,y) represents the original image and grad represents the gradient operation.

[0099] The watershed algorithm has a good response to weak edges. Noise in the image and slight grayscale changes on the surface of the object will cause over-segmentation. But at the same time, it should be noted that the watershed algorithm has a good response to weak edges, which is the guarantee of obtaining closed continuous edges. In addition, the closed water basin obtained by the watershed algorithm provides the possibility of analyzing the regional characteristics of the image.

[0100] In step C, in order to reduce the over-segmentation caused by the watershed algorithm, the gradient function needs to be modified. A simple method is to perform threshold processing on the gradient image to eliminate the over-segmentation caused by slight changes in grayscale.

[0101] g(x,y)=max(grad(f(x,y)),gθ)

[0102] Where gθ represents the threshold value.

[0103] The program adopts the method: use threshold to limit the gradient image to eliminate over-segmentation caused by small changes in grayscale values, obtain an appropriate amount of areas, and then sort the gray levels of the edge points of these areas from low to high, and then implement the flooding process from low to high. The gradient image is calculated using the Sobel operator.

[0104] The operator contains two sets of 3x3 matrices, one for horizontal and one for vertical. By performing a planar convolution with the image, the approximate values ​​of the horizontal and vertical brightness differences can be obtained respectively. The formula is as follows:

[0105]

[0106]

[0107] The approximate horizontal and vertical gradients of each pixel of the image can be combined using the following formula to calculate the magnitude of the gradient.

[0108]

[0109] Where A represents the original image, G x and G y Represent the images after horizontal and vertical edge detection respectively.

[0110] After preprocessing, the edge information is obtained and the minimum rectangular frame is selected.

[0111] In step S4, the machine vision algorithm training process specifically includes the following steps:

[0112] a, 5 detection areas are selected for each annotated aggregate image;

[0113] b. Perform image convolution on each detection area to extract image features;

[0114] c. Upsample the image and restore the convolved feature map to the image;

[0115] d, use tensor splicing algorithm to integrate image features;

[0116] e. Output to the detection head. The detection head classifies the image and determines the classification result of the recognition image through the probability function;

[0117] f, recognition result, output the detection result of the image.

[0118] Since the ultimate goal of this patent is to detect and identify aggregates in real time, and the detection background is complex, it is necessary to identify under different lighting, different distances and different moisture contents of aggregates. The features are diverse, and simple image processing may not be able to meet the judgment of sample features. Therefore, machine vision algorithms are used to input and train samples under different states.

[0119] After the collected images are processed as above, they are manually labeled with the aggregate category and input into the machine vision algorithm for feature extraction.

[0120] The machine vision algorithm used in this patent mainly has 6 working steps. First: Select 5 detection areas, which is to divide an image into 5 working areas, use an odd number of areas to judge the final result, and select 5 recognition categories with a recognition probability greater than 50% as the final result. Second: Image convolution, by performing image convolution on each area, the purpose of extracting image features is achieved, mainly extracting the edge features, color features and texture features of aggregates. Third: Image upsampling, so that the image conforms to the size of the display area, using the interpolation method, that is, inserting new elements between pixels based on the original image pixels using a suitable interpolation algorithm. Fourth: Tensor splicing, because before this we need to process the three channels (RGB) of the image separately, and then reassemble them to generate a new image, this requires tensor splicing, and tensor splicing is used to integrate the image features. Fifth: Output to the detection head, the detection head part is to classify the image, and judge the classification result of the recognition image through the probability function. Sixth: Recognition result, output the detection result of the image, that is, the algorithm believes that the image is the maximum probability result of a certain category.

[0121] Figure 2 This is a main flow chart of an aggregate detection method and system based on AI intelligent visual processing technology of the present invention. The process includes four parts: aggregate area image acquisition, image preprocessing, image feature extraction, and aggregate detection judgment. The system first determines whether the vehicle is in place. If it is in place, it obtains real-time image information, performs area selection and image preprocessing; then uses machine vision algorithms to detect images and perform recognition; records the recognition results, and outputs the final results if the probability of the selected 5 image results is greater than 50%; determines whether the vehicle has started to unload materials, and if so, stops judging.

[0122] Figure 3 The technical route of this patent is to first pre-process the training sample images, segment the aggregate edges, perform feature extraction and training learning, and establish an intelligent processing model for efficient aggregate detection; then process and compare the test sample images to obtain the actual aggregate recognition processing results.

[0123] Figure 6 This is a flow chart of the machine vision algorithm processing used in the present invention. First, five areas are extracted from each image, namely the center point of the image, the upper left, upper right, lower left, and lower right areas, so as to achieve one-time recognition and multiple judgments, and each area is processed; image features such as light, angle, texture and other features are extracted through image convolution; then the image is upsampled and the convolved feature map is restored to the image; then the tensor splicing algorithm is used to expand the image dimension; finally, the detection head is used to predict the results. The final recognition results of the five areas are the most likely consistent results, and more than 50% of the five areas have the same prediction results. The detection ends when it is determined that all five areas have been recognized.

[0124] In step a, the five detection areas are respectively the center point, upper left, upper right, lower left, and lower right areas of the aggregate image.

[0125] In step b, the image features include edge features, color features and texture features of the aggregate.

[0126] In step c, the image is upsampled so that the image fits the size of the display area, and an interpolation method is used, that is, a new element is inserted between pixels based on the original image pixels using a suitable interpolation algorithm.

[0127] In step C, a processing method is used to modify the gradient function so that the water collection basin only responds to the target to be detected, and the gradient image is threshold processed to reduce over-segmentation caused by grayscale changes.

[0128] Example 2

[0129] On the basis of Example 1, the method further includes the step: S6, testing new images, judging whether the sample size is sufficient according to the test results, and keeping the sample size as uniform as possible. If the accuracy of the weight for a certain category is lower than 90%, the corresponding sample size is increased and the features are re-extracted until the recognition rate of each type of aggregate reaches more than 90%, which meets the requirements.

[0130] In step S4, the algorithm will do the following:

[0131] Image classification: Input the image into a multi-layer convolutional neural network, which will output a feature vector and feed it back to the softmax unit to predict the type of image.

[0132] Positioning and classification: Use an algorithm to determine whether there is a detection target in the image, mark the location of the target, mark it with a bounding box, and identify and locate the target.

[0133] Object detection: Images can contain multiple objects, and some single images may have multiple objects of different classifications. Multiple objects can be detected.

[0134] The Softmax function, a normalized exponential function, is actually the logarithmic normalization of the gradient of a finite discrete probability distribution. In multinomial logistic regression and linear discriminant analysis, the input of the function is the result obtained from K different linear functions, and the probability formula for the sample vector x belonging to the jth category is:

[0135]

[0136] This can be viewed as a composite of K linear functions called the Softmax function.

[0137] The target positioning is represented by anchor boxes. The input image is divided into S*S grids, and n anchor boxes are generated for each small grid. The real frame of the image is IOU-calculated with the anchor box generated by the small grid where the center point of the image is located. The regressed box is the Bounding Box. When the anchor boxes are closer to the real width and height, the performance of the model is better. The k-means algorithm is used to cluster the width and height of representative shapes in the ground truth boxes of all samples in the training set, and dimension clustering is performed. Multiple anchor box groups with different numbers are clustered and applied to the model respectively, and finally the optimal anchor box group that compromises between the complexity of the model and high recall is found.

[0138] The Bounding Box formula is:

[0139]

[0140]

[0141] Where a w and a h is the width and height of the anchor box,

[0142] t w and t h The width and height directly predicted by the bounding box,

[0143] b w and b h is the actual width and height predicted after conversion,

[0144] This is the width and height of the output in the final prediction.

[0145] The loss function for object detection is as follows:

[0146]

[0147] In the above formula, N is the total number of categories, yi is the probability of the current category after the activation function, and yi* determines whether the prior box is responsible for the target (0 or 1), if so, it is 0, otherwise it is 1.

[0148] After obtaining the aggregate image, the present invention extracts edge features through an image processing method to enhance the aggregate features. First, the image is grayed, and then the image edge is extracted through a watershed algorithm. After all the collected images are preprocessed, the images are manually annotated, input into the machine vision algorithm, feature extraction is performed, and output as a weight file. The working process is to use a high-definition camera to shoot the aggregate transportation video, transmit it in real time through the 5G network, extract the key areas in the image frame sequence of the monitoring area, start preliminary processing of the image, use the image processing method for preprocessing, perform edge detection, manually annotate the collected image after processing, input it into the machine vision algorithm, train it, and finally put the trained weights in the detection system to detect and identify the aggregate. The aggregate image is obtained in real time through the 5G network using a camera, and then the video is processed using computer image processing technology, and the state images of different types of aggregates are detected and identified in combination with the AI ​​intelligent algorithm to solve the problem of complex operation of the existing aggregate monitoring method, and also realize the function of real-time processing.

[0149] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and modifications. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for aggregate detection and classification based on AI intelligent machine vision technology, characterized in that: The following steps are involved: S1, uses a high-definition camera to shoot aggregate transportation videos and transmits them in real time through the 5G network to obtain aggregate images; S2, processing the aggregate image by an image edge segmentation processing method to extract edge features; S3, manually labeling the aggregate categories of the processed aggregate images; S4, inputting the annotated aggregate image into the machine vision algorithm for training, feature extraction, and outputting it as a weight file; S5, put the trained weights into the detection system to detect and identify the aggregate; In step S2, the image edge segmentation processing method specifically includes the following steps: A, import the aggregate image, convert it into grayscale, and then calculate the gradient map; B, based on the gradient map, the watershed algorithm is used to process the image and obtain the edge line of the segmented image; C, use image opening operation to remove small objects and burr interference that affects the target in the image; D, perform adhesion segmentation to separate the objects that are stuck together; In step S4, the machine vision algorithm training process specifically includes the following steps: a, 5 detection areas are selected for each annotated aggregate image; b. Perform image convolution on each detection area to extract image features; c. Upsample the image and restore the convolved feature map to the image; d, use tensor splicing algorithm to integrate image features; e. Output to the detection head. The detection head classifies the image and determines the classification result of the recognized image through the probability function; f, recognition result, output the detection result of the image; In step C, a processing method is used to modify the gradient function so that the water collection basin only responds to the target to be detected, and the gradient image is threshold processed to reduce over-segmentation caused by grayscale changes; In step B, the calculation process of the watershed is divided into two steps, one is the sorting process, and the other is the flooding process. First, the gray level of each pixel is sorted from low to high, and then in the flooding process from low to high, the influence domain of each local minimum value at the h-order height is judged and marked using a first-in-first-out structure; In order to obtain the edge information of the image, the gradient image is usually used as the input image, that is, g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5 In the formula, f(x,y) represents the original image, and grad represents the gradient operation; The specific method of threshold processing of the gradient image is: g(x,y)=max(grad(f(x,y)),gθ) In the formula, gθ represents the threshold value; The program adopts the method: use the threshold to limit the gradient image to eliminate the over-segmentation caused by the slight change of the gray value, obtain the appropriate area, and then sort the gray levels of the edge points of these areas from low to high, and then realize the flooding process from low to high. The gradient image is calculated using the Sobel operator; The operator contains two sets of 3x3 matrices, one for horizontal and one for vertical. By performing a planar convolution with the image, the approximate horizontal and vertical brightness difference values ​​can be obtained respectively. The formula is as follows: The approximate horizontal and vertical gradients of each pixel in the image can be combined with the following formula to calculate the gradient size: Where A represents the original image, G x and G y Represent the images after horizontal and vertical edge detection respectively; After preprocessing, the edge information is obtained and the minimum rectangular frame is selected.

2. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that: In step a, the five detection areas are respectively the center point, upper left, upper right, lower left, and lower right areas of the aggregate image.

3. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that: In step b, the image features include edge features, color features and texture features of the aggregate.

4. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that: In step c, the image is upsampled so that the image fits the size of the display area, and an interpolation method is used, that is, a new element is inserted between pixels based on the original image pixels using a suitable interpolation algorithm.

Images