Sample particle size detection method, system, electronic device and storage medium

Abstract

The present disclosure provides a sample particle size detection method, system, electronic device and computer-readable storage medium, the method comprising: obtaining a data set containing multiple sample images; using a watershed segmentation algorithm based on distance transformation to perform image segmentation on the processed data set to obtain a rendering; pre-training the renderings with pre-labeled labels in the data set and migrating them to extract features from the renderings without labeled labels to label them; performing digital morphological processing on the labels, demarcating positive and negative instances, completing label reprocessing, and obtaining a training set; using a preset network model to train the training set, and obtaining a detection model when the model converges; estimating the particle size of the sample to be detected through the detection model. The technical solution disclosed in the present disclosure can solve the problems of low image segmentation accuracy, difficult labeling and large errors in the particle size detection process; it can adapt to particle size detection in complex environments, with strong robustness and high accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of computer vision technology, and in particular to a sample particle size detection method, a sample particle size detection system, an electronic device, and a computer-readable storage medium. Background Art

[0002] Ore particle size detection is an important part of the entire mining operation of today's mining enterprises, and in the entire detection process, accurate segmentation of ore images is the most important link. Accurate image segmentation is conducive to the subsequent detection system to further analyze the geometric parameters such as ore contours and circumscribed rectangles, and facilitate the subsequent particle size statistical processing, thereby maximizing the productivity of ore crushers and improving the economic benefits of mining enterprises.

[0003] The main detection methods of traditional ore particle size include screening, sedimentation, ultrasonic and diffraction methods. On the one hand, these detection methods are time-consuming and cannot provide timely feedback on the results; on the other hand, the detection results are affected by human subjective judgment, and the overall robustness is difficult to guarantee. At present, most of the research on the segmentation of ore images at home and abroad is based on traditional image processing methods, such as threshold-based segmentation algorithms, edge detection-based segmentation algorithms, clustering-based segmentation algorithms, and graph theory-based segmentation algorithms. The accuracy and robustness are low, especially for the segmentation accuracy of medium-sized stones. It is not possible to effectively solve the effective segmentation of ore under the adhesion of ore images. The most widely used watershed-based segmentation algorithm is prone to over-segmentation or under-segmentation problems, especially for the particle size calculation errors of adhesion stones. The segmentation algorithm based on deep learning requires a large number of data sets in different field environments, is difficult to label, and takes a long time to train the overall algorithm. The above particle size detection methods cannot meet the industrial production needs of mining enterprises. Therefore, it is necessary to design a method for detecting ore particle size that can adapt to complex production environments, has strong robustness and high accuracy. Summary of the invention

[0004] In order to at least solve the technical problems in the prior art of low image segmentation accuracy for particle size detection, great difficulty in data set annotation, long overall algorithm training time, and large particle size calculation error, the present disclosure provides a sample particle size detection method, a sample particle size detection system, an electronic device, and a computer-readable storage medium, which can adapt to particle size detection in complex production environments, and have strong robustness and high accuracy.

[0005] In a first aspect, the present disclosure provides a sample particle size detection method, the method comprising:

[0006] Get a data set containing multiple sample images;

[0007] A watershed segmentation algorithm based on distance transformation is used to find seed points to perform image segmentation on the sample images in the data set to obtain the effect graph of the data set, which includes the effect graph with pre-labeled labels and the effect graph without labeled labels;

[0008] Pre-train the renderings that are pre-labeled in the data set, and transfer the pre-training results to the renderings without labels to extract features and label the renderings without labels;

[0009] Perform digital morphological processing on the labels of all renderings, identify positive and negative instances, complete label reprocessing operations, and obtain training sets;

[0010] The training set is trained using a preset network model, and a detection model is obtained when the network model converges;

[0011] The particle size of the sample to be detected is estimated by the detection model.

[0012] Furthermore, the step of obtaining a data set including a plurality of sample images includes:

[0013] Collect original sample images under various preset situations;

[0014] The collected original sample images are preprocessed by grayscale dimension reduction and denoising to obtain the data set.

[0015] Furthermore, the grayscale dimension reduction and denoising preprocessing of the collected original sample image includes:

[0016] The weighted average method is used to set the weights to grayscale the original sample image and reduce its dimension; and,

[0017] Histogram equalization is used to enhance the contrast of the overall sample image after grayscale dimensionality reduction, and then the sample image with enhanced contrast is denoised using the bilateral filtering algorithm.

[0018] Furthermore, after the collected original sample image is subjected to grayscale dimension reduction and denoising preprocessing, the following steps are also included:

[0019] The method of detecting straight lines using Hough transform excludes sample images collected in the data set when the equipment is running without load; and

[0020] The frame difference method is used to exclude sample images collected when the device is in a stationary state from the data set.

[0021] Furthermore, after obtaining the effect diagram of the data set, the method further includes:

[0022] Performing image enhancement processing on the effect graph of the data set, and adding the image-enhanced effect graph to the data set;

[0023] The image enhancement processing may be performed in the following manners: any one or more of image scaling, image rotation, image mirroring, image brightness adjustment and image contrast adjustment.

[0024] Furthermore, the digital morphological processing is performed on the labels of all the effect images, and positive and negative instances are demarcated to complete the label reprocessing operation, including:

[0025] For the mask image generated after labeling the effect map without labeled labels, the expansion algorithm is used to expand a preset number of pixels outward according to the results of multiple calculations to generate a new mask image;

[0026] For the new mask image, the white edge of the object to be detected is set as the positive example, the black border of the preset number of pixels expanded after the dilation operation and the object to be detected itself are set as the negative example, and the outermost gray background area is not involved in the calculation, thus completing the label reprocessing operation.

[0027] Furthermore, the preset network model is obtained by reducing the number of channels of the original U-Net network model according to a preset ratio and introducing a residual network; the algorithm used for pre-training the effect graphs with pre-labeled labels in the data set is a semi-supervised learning method.

[0028] Furthermore, the method further comprises:

[0029] When the preset network model is used to train the training set, the cross entropy loss function and L2 regularization are used to construct the objective function as the performance evaluation index, and the Adam method optimization algorithm is used to minimize the objective function.

[0030] In a second aspect, the present disclosure provides a sample particle size detection system, the system comprising:

[0031] An acquisition module, which is configured to acquire a data set including a plurality of sample images;

[0032] An image segmentation module is configured to use a watershed segmentation algorithm based on distance transformation to find seed points to perform image segmentation on sample images in a data set, and obtain effect graphs of the data set, wherein the effect graphs include effect graphs with pre-labeled labels and effect graphs without labeled labels;

[0033] A migration module is configured to pre-train the renderings pre-labeled in the data set, and migrate the pre-training results to the renderings without label to extract features and label the renderings without label;

[0034] A label processing module is configured to perform digital morphological processing on the labels of all renderings, and to demarcate positive and negative instances, complete the label reprocessing operation, and obtain a training set;

[0035] A training module, which is configured to train the training set using a preset network model and obtain a detection model when the network model converges;

[0036] A detection module is configured to estimate the particle size of the sample to be detected by using the detection model.

[0037] In a third aspect, the present disclosure provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the sample particle size detection method as described in any one of the first aspects.

[0038] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the sample granularity detection method described in any one of the first aspects is implemented.

[0039] Beneficial effects:

[0040] The sample particle size detection method, sample particle size detection system, electronic device and computer-readable storage medium provided by the present disclosure obtain a data set containing multiple sample images; use a watershed segmentation algorithm based on distance transformation to find seed points to perform image segmentation on the sample images in the data set to obtain the effect map of the data set, and the effect map includes the effect map with pre-labeled labels and the effect map without labeled labels; pre-train the effect map with pre-labeled labels in the data set, and transfer the pre-training results to the effect map without labeled labels to extract features to label the effect map without labeled labels; perform digital morphological processing on the labels of all effect maps, and define positive and negative instances, complete the label reprocessing operation, and obtain a training set; use a preset network model to train the training set, and obtain a detection model when the network model converges; and estimate the particle size of the sample to be detected by the detection model. The technical solution disclosed in the present disclosure can solve the problems of low image segmentation accuracy for particle size detection, high difficulty in data set annotation, long overall algorithm training time, and large particle size calculation error in the prior art, and can adapt to particle size detection in complex production environments, with strong robustness and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A schematic diagram of a flow chart of a sample particle size detection method provided in the first embodiment of the present disclosure;

[0042] Figure 2 A schematic diagram of pre-training an existing annotated image provided in the first embodiment of the present disclosure;

[0043] Figure 3A schematic diagram of the effect of performing digital morphological processing on a label provided in the first embodiment of the present disclosure;

[0044] Figure 4 A schematic diagram of a method for detecting straight lines using Hough transform to exclude images collected when the device is in a no-load running state in a data set provided in the first embodiment of the present disclosure;

[0045] Figure 5 A schematic diagram of a flow chart of a sample particle size detection method provided in the second embodiment of the present disclosure;

[0046] Figure 6 A comparison chart of the results of ore particle size detection training under different environments provided in the second embodiment of the present disclosure;

[0047] Figure 7 This is an architecture diagram of a sample particle size detection system provided in Embodiment 3 of the present disclosure;

[0048] Figure 8 This is an architecture diagram of an electronic device provided in Embodiment 4 of the present disclosure. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the present disclosure is further described in detail below in conjunction with the drawings and embodiments. It should be understood that the specific embodiments and drawings described herein are only used to explain the present invention, rather than to limit the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence; and, in the absence of conflict, the embodiments in the present disclosure and the features in the embodiments can be arbitrarily combined with each other.

[0051] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms of "a", "said" and "the" used in the embodiments of the present disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings.

[0052] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present disclosure, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0053] The following specific embodiments are used to describe in detail the technical solution of the present disclosure and how the technical solution of the present disclosure solves the problems of low accuracy of image segmentation for particle size detection, difficulty in data set annotation, long overall algorithm training time, and large particle size calculation error in the prior art. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0054] Figure 1 A flow chart of a sample particle size detection method provided in the first embodiment of the present disclosure is shown in FIG. Figure 1 As shown, the method includes:

[0055] Step S101: obtaining a data set including a plurality of sample images;

[0056] Step S102: using a watershed segmentation algorithm based on distance transformation to find seed points to perform image segmentation on the sample images in the data set, and obtaining effect graphs of the data set, wherein the effect graphs include effect graphs with pre-labeled labels and effect graphs without labeled labels;

[0057] Step S103: pre-training the effect pictures pre-labeled in the data set, and transferring the pre-training results to the effect pictures without labeling to extract features and label the effect pictures without labeling;

[0058] Step S104: Perform digital morphological processing on the labels of all effect images, and define positive and negative instances, complete the label reprocessing operation, and obtain a training set;

[0059] Step S105: using a preset network model to train the training set, and obtaining a detection model when the network model converges;

[0060] Step S106: Estimate the particle size of the sample to be detected by using the detection model.

[0061] The disclosed embodiment obtains the original ore image by decoding the video stream through the camera when collecting the original image samples, and selects samples of different working conditions at different time points as data sets according to the image collection principle. The collected sample images contain characteristics such as stones of different sizes, serious adhesion between ores, difficult to distinguish boundaries, and serious environmental impact. To this end, it is necessary to pre-process the collected images for dimensionality reduction and denoising. After pre-processing the sample data set, the image needs to be segmented. The traditional watershed segmentation method is a segmentation method based on mathematical morphology of topological theory. Its basic idea is to regard the image as a topological landform in geodesy. The gray value of each pixel in the image represents the altitude of the point. Each local minimum and its affected area are called a watershed, and the boundary of the watershed forms a watershed. The concept and formation of a watershed can be explained by simulating the immersion process. A small hole is pierced on the surface of each local minimum, and then the entire model is slowly immersed in water. As the immersion deepens, the influence domain of each local minimum slowly expands outward, and a dam is built at the confluence of two water collection basins, that is, a watershed is formed. However, for sample images collected from ore, in the traditional watershed segmentation algorithm, the pixel values ​​between valleys are close to the pixel values ​​between watersheds, and it is difficult to select local minimum points as seed points, which can easily cause over-segmentation or under-segmentation problems, especially for the large error in the calculation of the particle size of adhered stones; therefore, the embodiment of the present disclosure adopts a watershed segmentation algorithm based on distance transformation to find seed points, and adopts the distance transformation method to divide the instance into foreground images and the rest into background images. By replacing pixel values ​​with distance values, the brighter the instance pixels are, the easier it is to find the center to be segmented. By normalizing pixels and finding seed points for ore segmentation, a better segmentation effect is achieved in the test set with higher image quality, and an effect diagram is obtained.

[0062] In order to solve the problems of serious adhesion and difficulty in defining standards in existing data sets, after annotating some images in the current data set, we use existing labels for pre-training, migrate to unlabeled labels to extract features, adopt a semi-supervised learning strategy and define a loss function calculation range to accelerate the convergence of deep networks. In the calculation of the loss function in the pre-training process, only the image range after secondary label processing is calculated; the pre-training process only trains images in a certain area, such as Figure 2 As shown in Figure 2 In (a), some stones have been labeled, and training and updating loss function are performed in the existing labeled area, that is, Figure 2As shown in (b), only the non-black areas in the image (for the marked labels) are trained; the existing label data is fully utilized to extract edge features, which reduces the calculation scope and accelerates the network convergence speed. After migrating the labels to the renderings without labeled labels, the existing labels need to be processed again. By performing digital morphological processing on the labels of all renderings and defining positive and negative instances, the label reprocessing operation is completed to obtain the training set. The significance of digital morphological processing is to use the expansion algorithm to expand the edge area of ​​the labeled image and reduce the training scope as much as possible; as shown in (b). Figure 3 As shown, Figure 3 The left image (a) is the original mask image after annotation, and the right image (b) is the image after digital morphology processing, which highlights the boundary area. Then the preset network model is used for training, and the training set is put into training. When the network converges, the corresponding detection model is obtained. The test set is input into the detection model to obtain the final segmentation effect of the test set and estimate the ore particle size.

[0063] The disclosed embodiment achieves better segmentation effect by finding seed points using a watershed segmentation algorithm based on distance transformation; based on a semi-supervised learning strategy, the overall image training area is changed to solve the problem of difficulty in data set annotation, and the boundary area is highlighted through morphological processing, which greatly improves the training efficiency. The model training can be completed in a short time using relatively low resources, so as to achieve accurate segmentation of ore particle size in industrial scenarios.

[0064] Furthermore, the step of obtaining a data set including a plurality of sample images includes:

[0065] Collect original sample images under various preset situations;

[0066] The collected original sample images are preprocessed by grayscale dimension reduction and denoising to obtain the data set.

[0067] At the mine site, we select specific angles to place cameras according to the needs, collect video streams to obtain original sample images under various conditions, including images under various weather and lighting conditions. We use the weighted average method to set weights for the collected images, grayscale the dataset and reduce the dimension, then use histogram equalization to enhance the overall image contrast and highlight the instance edges, and then use the bilateral filtering algorithm to achieve denoising effect. The preprocessed dataset is obtained.

[0068] Furthermore, the grayscale dimension reduction and denoising preprocessing of the collected original sample image includes:

[0069] The weighted average method is used to set the weights to grayscale the original sample image and reduce its dimension; and,

[0070] Histogram equalization is used to enhance the contrast of the overall sample image after grayscale dimensionality reduction, and then the sample image with enhanced contrast is denoised using the bilateral filtering algorithm.

[0071] The weighted average method is used to set the weights of the RGB three dimensions of the original color image according to the empirical values ​​to meet the actual needs. While retaining the key information of the original image, the computational complexity of the semantic segmentation network training is reduced, and the processing speed of the entire image segmentation process and the entire application system is improved. In view of the impact of light, shadow and dust on the quality of the collected images in the on-site working conditions, the histogram equalization method is used to improve the contrast of the overall image, highlight the edge range of the stone, and eliminate the difficulty of distinguishing some foreground and background. For the image noise of the collected image and the clutter of the surface texture characteristics of the ore itself, the bilateral filtering algorithm is used to consider the grayscale similarity and spatial proximity of the pixel points, and the denoising effect is achieved while retaining the edge of the ore.

[0072] Furthermore, after the collected original sample image is subjected to grayscale dimension reduction and denoising preprocessing, the following steps are also included:

[0073] The method of detecting straight lines using Hough transform excludes sample images collected in the data set when the equipment is running without load; and

[0074] The frame difference method is used to exclude sample images collected when the device is in a stationary state from the data set.

[0075] When the ore particle size detection system is unloaded, a large number of ore particle sizes will be counted as 0, which will cause serious errors in the subsequent overall ore particle size calculation. In order to eliminate the influence of this special working condition, the edge detection algorithm Canny algorithm is used to detect the edge of the image, and finally Hough is used to detect the straight line and output it. A threshold is set for the intersection of the detected straight lines. Figure 4 As shown in the figure, an empty conveyor belt can detect more horizontal straight lines (non-intersecting) than a non-empty conveyor belt. If the number of intersections is less than the threshold, it means that the belt contour can be clearly detected, indicating that it is in an empty state. If the number of intersections is greater than the threshold and there are many, it means that there are objects of different shapes on the conveyor belt, indicating that it is not empty.

[0076] When the conveyor belt gets stuck or an overall construction failure occurs, the camera will continue to collect redundant images when the conveyor belt is stationary. Continuous repeated counting will cause large errors in the statistical results of the ore particle size. To address this problem, the inter-frame difference method is used to judge and provide feedback on the conveyor belt condition for ore images at different times in a close time period to eliminate special working conditions.

[0077] Furthermore, after obtaining the effect diagram of the data set, the method further includes:

[0078] Performing image enhancement processing on the effect graph of the data set, and adding the image-enhanced effect graph to the data set;

[0079] The image enhancement processing may be performed in the following manners: any one or more of image scaling, image rotation, image mirroring, image brightness adjustment and image contrast adjustment.

[0080] In order to solve the problem of network overfitting caused by insufficient data and the lack of generalization ability of the training model, the data enhancement method is used to expand the existing data set. The online enhancement method is adopted, and the GPU is used to train the previous batch of models. The pre-processing of data enhancement is completed by the CPU to improve the convergence efficiency of the model. Then the data set is subjected to image scaling, image rotation, image mirroring and other transformations. The rotation angles include 90°, 180°, 270° and other angles, and the mirror transformation includes up, down, left, right, and reverse transformation methods. Image scaling facilitates the overall network to have multi-scale learning capabilities. In addition, operations including changing the image brightness value and contrast are used to make the network more adaptable to images in different lighting environments. Multiple methods can be performed separately or in combination.

[0081] Furthermore, the digital morphological processing is performed on the labels of all the effect images, and positive and negative instances are demarcated to complete the label reprocessing operation, including:

[0082] For the mask image generated after labeling the effect map without labeled labels, the expansion algorithm is used to expand a preset number of pixels outward according to the results of multiple calculations to generate a new mask image;

[0083] For the new mask image, the white edge of the object to be detected is set as the positive example, the black border of the preset number of pixels expanded after the dilation operation and the object to be detected itself are set as the negative example, and the outermost gray background area is not involved in the calculation, thus completing the label reprocessing operation.

[0084] Do digital morphology processing such as Figure 3 As shown, the edge area of ​​the annotated image is expanded by the expansion algorithm to reduce the training range as much as possible. By using the expansion algorithm, a preset number of pixels are selected to expand outward according to the results of multiple calculations. The preset number can be 5, 8 or other numbers to generate a new mask image, which can fully cover the existing annotated instances on the one hand and reduce the training range on the other hand. The edge of the white ore is set as a positive example, the black border of the preset number of pixels expanded after the expansion operation and the ore individual itself are set as negative examples, and the outermost gray background area does not involve calculation, so as to achieve the purpose of label reprocessing, which is in line with the semi-supervised learning strategy.

[0085] Furthermore, the preset network model is obtained by reducing the number of channels of the original U-Net network model according to a preset ratio and introducing a residual network; the algorithm used for pre-training the effect graphs with pre-labeled labels in the data set is a semi-supervised learning method.

[0086] In view of the network degradation phenomenon that may occur when the existing data set is put into the deep network, the model can be improved. For example, for the semantic segmentation network U-Net network structure, the original network channels are proportionally reduced to reduce the complexity, save video memory usage and avoid server overload. The residual network (Res Net) is introduced to eliminate the network degradation problem caused by the network output jumping out of the optimal solution due to the nonlinear activation function in the network structure.

[0087] Furthermore, the method further comprises:

[0088] When the preset network model is used to train the training set, the cross entropy loss function and L2 regularization are used to construct the objective function as the performance evaluation index, and the Adam method optimization algorithm is used to minimize the objective function.

[0089] By adding L2 regularization to construct the objective function, the weights can be kept as small as possible during the fitting process, and a model with relatively small parameters can be constructed, making the model simpler and adaptable to different data sets, and avoiding overfitting to a certain extent; the cross entropy loss function and the Adam method optimization algorithm are widely used in deep learning applications in computer vision and natural language processing. The objective function can be expressed as follows, with the first half being the cross entropy and the second half being the L2 regularization term:

[0090]

[0091] Among them, y k represents the indicator variable. If the category is the same as the sample category, it is 1, otherwise it is 0, indicating the probability that the pixel x belongs to category k. λ is the L2 regularization term coefficient. is the L2 regularization term. By improving the U-Net network structure, redefining the cross entropy loss function, designing performance evaluation indicators, and training the model, a simple deep network U-Net is used. The network structure is simpler and the model complexity is low. After the improvement, the training accuracy and speed are faster.

[0092] The particle size detection method based on semi-supervised learning of the disclosed embodiment can adapt to particle size detection in various complex production environments, and has strong robustness and high accuracy. The watershed segmentation algorithm based on distance value is used to more effectively find seed points for image segmentation, achieving better segmentation effect; through image enhancement, the data set is expanded to make the network more adaptable to images in different lighting environments; based on the semi-supervised learning strategy, the overall image training area is changed, and the boundary area is highlighted through morphological processing, which greatly improves the training efficiency while expanding the training set, and can complete model training in a short time with relatively low resources, so as to achieve accurate segmentation of ore particle size in industrial scenarios.

[0093] In order to more clearly describe the technical solution of the present disclosure, Figure 5 As shown, the second embodiment of the present disclosure provides a sample particle size detection method, including:

[0094] Step S1: Select a specific angle to place the camera at the mine site according to the needs, collect the video stream and connect it to the video monitoring room through the on-site industrial bus; after receiving the video stream, the operator first decodes the video stream, and selects a specific frame rate to collect the original ore image based on the responsibility of collecting irrelevant, non-overlapping and continuous screenshots; the collected ore image contains the characteristics of different sizes of stones, serious adhesion between ores, difficult to distinguish boundaries, and serious impact of the environment;

[0095] Step S2: Preprocessing the ore image, including:

[0096] S2a: The weighted average method is used to set the weights of the RGB three dimensions of the original color image according to the empirical values ​​to meet the actual needs, while retaining the key information of the original image, reducing the computational complexity of the semantic segmentation network training, and improving the processing speed of the entire image segmentation process and the entire application system;

[0097] S2b: In view of the influence of light, shadow and dust on the quality of collected images in the field working conditions, the histogram equalization method is used to improve the contrast of the overall image, highlight the edge range of the stone, and eliminate the difficulty in distinguishing some foreground and background;

[0098] S2c: For the image noise of the collected images and the clutter of the surface texture characteristics of the ore itself, a bilateral filtering algorithm is used to consider the grayscale similarity and spatial proximity of the pixel points to achieve denoising effect while retaining the edge of the ore;

[0099] Step S3: Check special working conditions, including no-load check and static state check; when the ore particle size detection system is no-load, a large number of ore particle size statistics of 0 will be counted, which will cause serious errors in the subsequent overall ore particle size calculation; to eliminate the impact of this special working condition, the edge detection algorithm Canny algorithm should be used to detect the edge of the image, and finally Hough should be used to detect the straight line and output; set the threshold of the intersection between the detected straight lines, and fewer intersections indicate that the belt contour can be clearly detected, and fewer intersections than the threshold indicate that it is in a no-load state; when the conveyor belt is stuck or the entire construction fails, the static state of the conveyor belt will cause the camera to continue to collect redundant images, and continuous repeated counting will cause large errors in the ore particle size statistics. To address this problem, the inter-frame difference method is used to judge and feedback the conveyor belt status on the ore images at different times in close time periods, eliminating special working conditions;

[0100] Step S3: Segment the image using a watershed segmentation algorithm based on distance segmentation; in the traditional watershed segmentation algorithm, the pixel values ​​between valleys are close to the pixel values ​​between watersheds, and it is difficult for the seed point to select the local minimum point; the distance transformation method is used to divide the instance into a foreground image and the rest into a background image. The distance value is used to replace the pixel value, which makes the instance pixel brighter and easier to find the center to be segmented. By normalizing the pixels, the seed point is found for ore segmentation to obtain the effect image;

[0101] Step S5: In order to solve the problem that insufficient data may cause network overfitting and training model lacks generalization ability, the existing data set is expanded by data enhancement method; the online enhancement method is adopted, and the GPU is used to train the previous batch of models, and the pre-processing of data enhancement is completed by the CPU to improve the convergence efficiency of the model; the data set is then subjected to image scaling, image rotation, and image mirroring; the rotation angles include 90°, 180°, 270° and other angles, and the mirroring transformation includes up, down, left, right, and reverse transformation methods. Image scaling facilitates the overall network to have multi-scale learning ability; in addition, operations including changing the image brightness value and contrast are adopted to make the network more adaptable to images under different lighting environments;

[0102] Step S6: Based on the network training of semi-supervised learning, the image segmentation of the ore image data set has the following difficulties: first, the ores are seriously adhered to each other, and it is difficult to accurately distinguish the edges of the ore instances from the perspective of the annotator; second, the ore images taken in different environments around the clock are seriously covered by shadows; third, it is difficult for the annotator to define the size relationship of the stones, and labeling is more difficult; In response to the above problems, by extracting the existing annotated stones for pre-training, the features are transferred to the unlabeled stones. In the calculation of the loss function during the training process, only the image range after the secondary label processing is calculated, and the training and update of the loss function are performed in the existing annotated area. On the one hand, it can make full use of the existing label data to extract edge features, and on the other hand, it reduces the calculation scope and accelerates the network convergence speed. Based on this strategy, the existing labels need to be processed again. For the generated mask image, the expansion algorithm is used. According to the results of multiple calculations, 5 pixels are selected to expand outward to generate a new mask image. On the one hand, it can fully cover the existing labeled instances, and on the other hand, it can reduce the training scope. The edge of the white ore is set as a positive example, and the black border of the 5-pixel range expanded after the expansion operation and the ore individual itself are set as negative examples. The outermost gray background area does not involve calculation, so as to achieve the purpose of label reprocessing, which is in line with the semi-supervised learning strategy.

[0103] Step S7: In view of the problem that the existing semantic segmentation network U-Net structure is prone to overfitting, the number of original network channels is proportionally reduced to reduce complexity, save video memory usage and avoid server overload, and a residual network (ResNet) is introduced to eliminate the network degradation problem caused by the network output jumping out of the optimal solution due to the nonlinear activation function in the network structure;

[0104] Step S8: The objective function is constructed as the performance evaluation index through the cross entropy loss function and L2 regularization, the training set is put into training, the corresponding model is obtained when the network converges, the test set is input to obtain the final segmentation effect, and the ore particle size is estimated.

[0105] Comparison of ore particle size detection in different environments in the embodiment of the present disclosure Figure 6 As shown, Figure 6 a is the original image under different situations, Figure 6 b is the corresponding label annotation map, Figure 6 c is the corresponding side view after image processing under the original strategy; Figure 6 b is the corresponding side view after the image is processed using the training strategy in this embodiment. It can be seen that the training strategy in this embodiment can adapt to complex production environments, and the detection results are robust and accurate.

[0106] Figure 7 This is an architecture diagram of a sample particle size detection system provided in Embodiment 3 of the present disclosure, such as Figure 7 As shown, the system comprises:

[0107] In a second aspect, the present disclosure provides a sample particle size detection system, the system comprising:

[0108] An acquisition module 11, which is configured to acquire a data set including a plurality of sample images;

[0109] An image segmentation module 12 is configured to use a watershed segmentation algorithm based on distance transformation to find seed points to perform image segmentation on sample images in a data set, and obtain effect graphs of the data set, wherein the effect graphs include effect graphs with pre-labeled labels and effect graphs without labeled labels;

[0110] A migration module 13, which is configured to pre-train the effect pictures pre-labeled with labels in the data set, and migrate the pre-training results to the effect pictures without labeling labels to extract features for labeling the effect pictures without labeling labels;

[0111] The label processing module 14 is configured to perform digital morphological processing on the labels of all renderings, and to delineate positive and negative instances, complete the label reprocessing operation, and obtain a training set;

[0112] A training module 15, which is configured to train the training set using a preset network model and obtain a detection model when the network model converges;

[0113] The detection module 16 is configured to estimate the particle size of the sample to be detected by using the detection model.

[0114] Furthermore, the acquisition module 11 includes:

[0115] A collection unit, which is configured to collect original sample images under various preset situations;

[0116] The preprocessing unit is configured to perform grayscale dimension reduction and denoising preprocessing on the collected original sample images to obtain a data set.

[0117] Furthermore, the preprocessing unit is specifically configured as follows:

[0118] The weighted average method is used to set the weights to grayscale the original sample image and reduce its dimension; and,

[0119] Histogram equalization is used to enhance the contrast of the overall sample image after grayscale dimensionality reduction, and then the sample image with enhanced contrast is denoised using the bilateral filtering algorithm.

[0120] Furthermore, the system also includes a troubleshooting module 17;

[0121] The screening module 17 is configured to use a Hough transform method to detect straight lines to exclude sample images collected in the data set when the device is in a no-load operation state; and

[0122] The frame difference method is used to exclude sample images collected when the device is in a stationary state from the data set.

[0123] Furthermore, the system also includes an image enhancement module 18;

[0124] The image enhancement module 18 is configured to perform image enhancement processing on the effect image of the data set, and add the image-enhanced effect image to the data set;

[0125] The image enhancement processing may be performed in the following manners: any one or more of image scaling, image rotation, image mirroring, image brightness adjustment and image contrast adjustment.

[0126] Furthermore, the tag processing module 14 is specifically configured as follows:

[0127] For the mask image generated after labeling the effect map without the label, the expansion algorithm is used to expand the preset number of pixels outward according to the results of multiple calculations to generate a new mask image; and

[0128] For the new mask image, the white edge of the object to be detected is set as the positive example, the black border of the preset number of pixels expanded after the dilation operation and the object to be detected itself are set as the negative example, and the outermost gray background area is not involved in the calculation, thus completing the label reprocessing operation.

[0129] Furthermore, the preset network model is obtained by reducing the number of channels of the original U-Net network model according to a preset ratio and introducing a residual network; the algorithm used for pre-training the effect graphs with pre-labeled labels in the data set is a semi-supervised learning method.

[0130] Furthermore, the training module 15 is specifically configured as follows:

[0131] When the preset network model is used to train the training set, the cross entropy loss function and L2 regularization are used to construct the objective function as the performance evaluation index, and the Adam method optimization algorithm is used to minimize the objective function.

[0132] The sample particle size detection system of the disclosed embodiment is used to implement the sample particle size detection method in method embodiment 1 and embodiment 2, so the description is relatively simple. For details, please refer to the relevant description in the previous method embodiment 1 and embodiment 2, which will not be repeated here.

[0133] In addition, if Figure 8As shown, the fourth embodiment of the present disclosure also provides an electronic device, including a memory 10 and a processor 20, wherein the memory 10 stores a computer program, and when the processor 20 runs the computer program stored in the memory 10, the processor 20 executes the above-mentioned various possible methods.

[0134] The memory 10 is connected to the processor 20. The memory 10 may be a flash memory, a read-only memory or other memory. The processor 20 may be a central processing unit or a single-chip microcomputer.

[0135] In addition, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to perform the above-mentioned various possible methods.

[0136] The computer-readable storage medium includes volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules or other data). Computer-readable storage media include, but are not limited to, RAM (Random Access Memory), ROM (Read-Only Memory), EEPROM (Electrically Erasable Programmable read only memory), flash memory or other memory technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer.

[0137] It is to be understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of the present disclosure, but the present disclosure is not limited thereto. For those of ordinary skill in the art, various modifications and improvements can be made without departing from the spirit and substance of the present disclosure, and these modifications and improvements are also considered to be within the scope of protection of the present disclosure.

Claims

1. A sample particle size detection method, characterized in that: The method comprises: Get a data set containing multiple sample images; A watershed segmentation algorithm based on distance transformation is used to find seed points to perform image segmentation on the sample images in the data set to obtain the effect graph of the data set, which includes the effect graph with pre-labeled labels and the effect graph without labeled labels; Pre-train the renderings that are pre-labeled in the data set, and transfer the pre-training results to the renderings without labels to extract features and label the renderings without labels; Perform digital morphological processing on the labels of all renderings, identify positive and negative instances, complete label reprocessing operations, and obtain training sets; The training set is trained using a preset network model, and a detection model is obtained when the preset network model converges, wherein the preset network model is obtained by reducing the number of channels of the original U-Net network model according to a preset ratio and introducing a residual network; The granularity of the sample to be detected is estimated by the detection model, which includes: inputting a test set including the sample to be detected into the detection model to obtain a final segmentation effect, and estimating the granularity of the sample to be detected; The digital morphological processing is performed on the labels of all renderings, and positive and negative instances are demarcated to complete the label reprocessing operation, including: For the mask image generated after labeling the effect map without labeled labels, the expansion algorithm is used to expand a preset number of pixels outward according to the results of multiple calculations to generate a new mask image; For the new mask image, the white edge of the object to be detected is set as the positive example, the black border of the preset number of pixels expanded after the dilation operation and the object to be detected itself are set as the negative example, and the outermost gray background area is not involved in the calculation, thus completing the label reprocessing operation.

2. The method according to claim 1, characterized in that: The step of acquiring a data set including a plurality of sample images comprises: Collect original sample images under various preset situations; The collected original sample images are preprocessed by grayscale dimension reduction and denoising to obtain the data set.

3. The method according to claim 2, characterized in that The grayscale dimension reduction and denoising preprocessing of the collected original sample image includes: The weighted average method is used to set the weights to grayscale the original sample image and reduce its dimension; and, Histogram equalization is used to enhance the contrast of the overall sample image after grayscale dimensionality reduction, and then the sample image with enhanced contrast is denoised using the bilateral filtering algorithm.

4. The method according to claim 2 or 3, characterized in that: After the collected original sample image is grayed out, dimension reduced and denoised, the following steps are also included: The method of detecting straight lines using Hough transform excludes sample images collected in the data set when the equipment is running without load; and The frame difference method is used to exclude sample images collected when the device is in a stationary state from the data set.

5. The method according to claim 1, characterized in that After obtaining the effect diagram of the data set, the method further includes: Performing image enhancement processing on the effect graph of the data set, and adding the image-enhanced effect graph to the data set; The image enhancement processing may be performed in the following manners: any one or more of image scaling, image rotation, image mirroring, image brightness adjustment and image contrast adjustment.

6. The method according to claim 1, characterized in that The algorithm used for pre-training the renderings with pre-labeled labels in the dataset is a semi-supervised learning method.

7. The method according to claim 1 or 6, characterized in that: The method further comprises: When the preset network model is used to train the training set, the cross entropy loss function and L2 regularization are used to construct the objective function as the performance evaluation index, and the Adam method optimization algorithm is used to minimize the objective function.

8. A sample particle size detection system, characterized in that: The system comprises: An acquisition module, which is configured to acquire a data set including a plurality of sample images; An image segmentation module is configured to use a watershed segmentation algorithm based on distance transformation to find seed points to perform image segmentation on sample images in a data set, and obtain effect graphs of the data set, wherein the effect graphs include effect graphs with pre-labeled labels and effect graphs without labeled labels; A migration module is configured to pre-train the renderings pre-labeled in the data set, and migrate the pre-training results to the renderings without label to extract features and label the renderings without label; A label processing module is configured to perform digital morphological processing on the labels of all renderings, and to demarcate positive and negative instances, complete the label reprocessing operation, and obtain a training set; A training module, which is configured to train the training set using a preset network model and obtain a detection model when the network model converges, wherein the preset network model is obtained by reducing the number of channels of the original U-Net network model according to a preset ratio and introducing a residual network; A detection module, which is configured to estimate the granularity of the sample to be detected through the detection model, comprising: inputting a test set including the sample to be detected into the detection model to obtain a final segmentation effect, and estimating the granularity of the sample to be detected; The digital morphological processing is performed on the labels of all renderings, and positive and negative instances are demarcated to complete the label reprocessing operation, including: For the mask image generated after labeling the effect map without labeled labels, the expansion algorithm is used to expand a preset number of pixels outward according to the results of multiple calculations to generate a new mask image; For the new mask image, the white edge of the object to be detected is set as the positive example, the black border of the preset number of pixels expanded after the dilation operation and the object to be detected itself are set as the negative example, and the outermost gray background area is not involved in the calculation, thus completing the label reprocessing operation.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes the sample particle size detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the sample particle size detection method according to any one of claims 1 to 7 is implemented.

Images