Mixture detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a mixture detection method, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a to-be-detected image, carrying out the segmentation of a mixture in the to-be-detected image according to a polygon through a segmentation model, and obtaining a segmented image; wherein the mixture comprises impurities and a target object, the segmented image comprises a mask area of the impurities, the target object corresponds to a coal mine, and the impurities comprise water; acquiring all candidate detection frames matched with the mask regions in which pixels are continuously distributed and the number of the pixels exceeds a number threshold value in the segmented image; and determining a target detection frame of the impurities from all the candidate detection frames based on the overlap ratio among all the candidate detection frames. In this way, the accuracy of water-coal mixture detection can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a mixture detection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the continuous development of image processing technology, the recognition of a certain type of substance in an image has a high recognition accuracy. However, the accuracy rate when identifying a mixture is still low. In coal mining, if the water content of coal exceeds the standard, it will not only reduce the coal quality, but also affect the subsequent washing and transportation processes. The accuracy of existing technologies will drop significantly when identifying a mixture composed of liquid impurities such as water and solid targets such as coal. In view of this, how to improve the accuracy of water-coal mixture detection has become an urgent problem to be solved. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a mixture detection method, an electronic device and a computer-readable storage medium, which can improve the accuracy of water-coal mixture detection.

[0004] To solve the above technical problems, the first aspect of the present application provides a mixture detection method, comprising: obtaining an image to be detected, and using a segmentation model to segment the mixture in the image to be detected according to polygons to obtain a segmented image; wherein the mixture includes impurities and target objects, and the segmented image includes a mask area of ​​the impurities, the target object corresponds to a coal mine and the impurities include water; obtaining all candidate detection frames matched by the mask area in the segmented image where pixels are continuously distributed and the number exceeds a quantity threshold; based on the overlap between all the candidate detection frames, determining the target detection frame of the impurity from all the candidate detection frames.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium on which program data is stored. When the program data is executed by a processor, the method described in the first aspect is implemented.

[0007] The above scheme obtains the image to be detected, and uses the segmentation model to segment the mixture in the image to be detected according to polygons, so as to distinguish impurities and targets in the mixture, wherein the target corresponds to coal mines and the impurities include water, and generates a mask area for the area corresponding to the impurities obtained after segmentation, and obtains a segmented image, thereby enhancing the capture of edge information corresponding to liquid impurities such as water through polygon segmentation, reducing the probability of redundant information including the target in the mask area, and strengthening the features of the impurities through the mask area, so as to facilitate the identification of the impurities in the segmented image. The pixels in the segmented image are traversed, and the mask area where the pixels are continuously distributed and the total number of pixels exceeds the number threshold is determined from the segmented image, and a matching candidate detection frame is generated for the corresponding mask area, so as to facilitate the acquisition of independent and unconnected candidate detection frames corresponding to the mask area from the segmented image, and all candidate detection frames in the segmented image are obtained, so that the marked area corresponding to the impurities is more comprehensive. Based on the overlap between all candidate detection frames, all candidate detection frames are screened to obtain the target detection frame matched by the impurities that need to be independently identified, so as to independently identify the impurities in the mixture in the image to be detected, and improve the accuracy of water-coal mixture detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0009] Figure 1 It is a schematic diagram of a flow chart of an implementation method of a mixture detection method of the present application;

[0010] Figure 2 It is a schematic flow chart of another embodiment of the mixture detection method of the present application;

[0011] Figure 3 This is a schematic diagram of an application scenario of an implementation scheme of a candidate detection frame of the present application;

[0012] Figure 4 This is a schematic diagram of an application scenario of an implementation scheme of the target detection frame of the present application;

[0013] Figure 5 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0014] Figure 6 It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0017] The mixture detection method provided in the present application is used to identify the mixture in the image to be identified and mark the position of impurities. It is particularly suitable for identifying a water-coal mixture composed of liquid impurities and solid target objects. Its corresponding execution subject is a processing unit capable of performing image processing.

[0018] See also Figure 1 , Figure 1 : is a schematic diagram of a flow chart of an embodiment of a mixture detection method of the present application, the method comprising:

[0019] S101: Acquire an image to be detected, and use a segmentation model to segment the mixture in the image to be detected according to polygons to obtain a segmented image; wherein the mixture includes impurities and target objects, the segmented image includes a mask area of ​​the impurities, the target object corresponds to a coal mine, and the impurities include water.

[0020] Specifically, an image to be detected is obtained, and a segmentation model is used to segment the mixture in the image to be detected according to polygons, so as to distinguish impurities and targets in the mixture, wherein the target corresponds to a coal mine and the impurities include water, and a mask area is generated for the area corresponding to the impurities obtained after segmentation to obtain a segmented image.

[0021] In some implementation scenarios, an image to be detected is obtained, a mixture in the image to be detected is segmented according to polygons using a segmentation model, impurities in the mixture are marked using polygons, and a mask area is generated for the marked polygon area to obtain a segmented image. The segmentation model is obtained by training with training samples with impurity labels, and the impurity labels include polygonal labeled areas set for impurities.

[0022] In some implementation scenarios, an image to be detected is obtained, and a mixture in the image to be detected is segmented according to polygons using a segmentation model, and the target object and impurities in the mixture are respectively marked using polygons, and the boundary between the target object and the impurity is determined, and a matching mask area is generated for the polygon area corresponding to the marked impurity to obtain a segmented image. The segmentation model is obtained by training with training samples with boundary labels, and the boundary labels include a curve set for the boundary between the target object and the impurity.

[0023] It can be understood that polygon segmentation enhances the capture of edge information corresponding to liquid impurities such as water, reduces the probability of redundant information of the target object included in the mask area, and strengthens the features of the impurities through the mask area to facilitate the identification of impurities in the segmented image.

[0024] S102: Obtain all candidate detection boxes that match mask regions in which pixels in the segmented image are continuously distributed and whose number exceeds a number threshold.

[0025] Specifically, the pixels in the segmented image are traversed, and the mask areas where the pixels are continuously distributed and the total number of pixels exceeds the number threshold are determined from the segmented image. Matching candidate detection frames are generated for the corresponding mask areas, so that candidate detection frames corresponding to independent and unconnected mask areas can be obtained from the segmented image, and all candidate detection frames in the segmented image are obtained, so that the marked area corresponding to the impurity is more comprehensive.

[0026] In some implementation scenarios, all pixels in the initial segmented image do not have access marks set. Starting from a preset position in the segmented image, the pixels in the segmented image that do not have access marks set are traversed, and access marks are set for the traversed pixels. When the mask value corresponding to the mask area is traversed, other mask values ​​are searched in a preset direction with the current mask value as the center until no mask value is obtained in the preset direction. Access marks are set for other mask values ​​found, and the circumscribed rectangles corresponding to all mask values ​​found this time are obtained to obtain candidate detection boxes until all pixels in the segmented image are traversed.

[0027] In some implementation scenarios, a sliding window is obtained, wherein the sliding window includes multiple pixels, and the sliding window is moved in the segmented image to determine whether the sliding window includes a mask value. When the sliding window includes a mask value, the sliding window is moved in a preset direction with the current mask value as the center to search for other mask values ​​until no mask value is obtained in the preset direction. A candidate detection frame is obtained by fitting all sliding windows based on the mask values ​​found this time, and then the sliding window is moved in the area that has not been traversed in the segmented image to determine whether the sliding window includes the mask value until all pixels in the segmented image are traversed.

[0028] S103: Determine a target detection frame of the impurity from all candidate detection frames based on the overlap between all candidate detection frames.

[0029] Specifically, based on the overlap between all candidate detection frames, all candidate detection frames are screened to obtain target detection frames that match the impurities that need to be independently identified, thereby independently identifying the impurities in the mixture in the image to be detected and improving the accuracy of mixture detection.

[0030] In some implementation scenarios, all candidate detection frames are combined in pairs to obtain a candidate detection frame combination, the intersection area between two candidate detection frames in the candidate detection frame combination is determined, and the ratio between the intersection area and the smaller candidate detection frame is determined. When the ratio exceeds the ratio threshold, the smaller candidate detection frame is eliminated, and the candidate detection frame finally retained is used as the target detection frame of the impurity.

[0031] In some implementation scenarios, the intersection-and-union ratios between all candidate detection frames are obtained, and candidate detection frame combinations whose intersection-and-union ratios are greater than a preset ratio are determined. The candidate detection frames with fewer pixels in the candidate detection frame combinations are eliminated, and the candidate detection frames that are finally retained are used as target detection frames for impurities.

[0032] It should be noted that the above mixture detection method is applicable to the detection of impurities after different substances are mixed, and is particularly applicable to the scene where the impurities are liquid substances and the target substances are solid substances. Among them, the image to be detected corresponds to the image corresponding to the conveyor belt area, the target object corresponds to the coal mine and the impurities include water.

[0033] Specifically, in coal mining, insufficient drainage systems or high groundwater levels may cause the water content of coal to exceed the specified standard. Excessive water content not only reduces the quality of coal, but also affects subsequent washing and transportation processes. Among them, coal is directly transported to the next process through conveyor belts. The development of water-coal mixture detection technology suitable for conveyor belts is crucial to ensure coal quality and improve production efficiency.

[0034] It can be understood that polygon segmentation enhances the capture of water surface edge information, while eliminating a large amount of redundant information in non-water surface areas, and obtains candidate detection frames corresponding to independent and unconnected mask areas from the segmented image, so as to obtain all candidate detection frames in the segmented image, making the water surface area more comprehensive. Finally, all candidate detection frames are screened to obtain the water surface area that needs to be independently identified, so as to accurately detect the water mixed in the coal mine on the conveyor belt, ensure the coal quality and improve production efficiency.

[0035] The above scheme obtains the image to be detected, and uses the segmentation model to segment the mixture in the image to be detected according to polygons, so as to distinguish impurities and targets in the mixture, wherein the target corresponds to coal mines and the impurities include water, and generates a mask area for the area corresponding to the impurities obtained after segmentation, and obtains a segmented image, thereby enhancing the capture of edge information corresponding to liquid impurities such as water through polygon segmentation, reducing the probability of redundant information including the target in the mask area, and strengthening the features of the impurities through the mask area, so as to facilitate the identification of the impurities in the segmented image. The pixels in the segmented image are traversed, and the mask area where the pixels are continuously distributed and the total number of pixels exceeds the number threshold is determined from the segmented image, and a matching candidate detection frame is generated for the corresponding mask area, so as to facilitate the acquisition of independent and unconnected candidate detection frames corresponding to the mask area from the segmented image, and all candidate detection frames in the segmented image are obtained, so that the marked area corresponding to the impurities is more comprehensive. Based on the overlap between all candidate detection frames, all candidate detection frames are screened to obtain the target detection frame matched by the impurities that need to be independently identified, so as to independently identify the impurities in the mixture in the image to be detected, and improve the accuracy of water-coal mixture detection.

[0036] See also Figure 2 , Figure 2 : is a flow chart of another embodiment of the mixture detection method of the present application, the method comprising:

[0037] S201: Acquire an image to be detected, segment the mixture in the image to be detected using a segmentation model, and determine a mask image that matches the mixture in the image to be detected; wherein the segmentation model is trained using training samples with impurity labels, and the impurity labels include polygonal labeled areas set for impurities.

[0038] Specifically, an image to be detected is obtained, and the image to be detected is input into a segmentation model so that the segmentation model segments the mixture in the image to be detected, wherein the segmentation model is obtained by supervised training using training samples with impurity labels, and the impurity labels include polygonal labeled areas set for impurities, so that the segmentation model can segment the impurities in the mixture according to the polygons, effectively improving the efficiency and accuracy of image segmentation.

[0039] Furthermore, a mask image is generated for the image to be detected, so that the mask area in the mask image matches the impurities obtained by segmentation in the image to be detected.

[0040] S202: Fusing the mask image and the image to be detected to obtain a segmented image; wherein the segmentation model is configured with a mask matrix for the image to be detected, and the mask image is generated based on the mask matrix.

[0041] Specifically, the mask image has the same size as the image to be detected, wherein the segmentation model is configured with a mask matrix for the image to be detected, and the mask matrix includes elements indicating whether a mask value is set for each pixel. For example, when the element is 1, it means that the mask value is set, and when the element is 0, the mask value is not set.

[0042] It can be understood that the mask image is fused with the image to be detected, that is, the mask image is superimposed on the image to be detected to obtain a segmented image, thereby enhancing the features of the impurities through the mask image and unifying the pixels of the impurities.

[0043] It should be noted that the image to be detected is obtained based on the following steps: acquiring an initial image captured in the target scene, and using a detection model to extract the image to be detected corresponding to the target area from the initial image; wherein the detection model is trained using training samples with area labels, and the area labels include rectangular annotated areas set for the target area.

[0044] Specifically, an initial image collected from a target scene is obtained, and the initial image is input into a detection model so that the detection model detects a target area from the initial image and outputs a fixed-size image to be detected corresponding to the target area. The detection model is obtained by supervised training using training samples with region labels, and the region labels include a rectangular annotated area set for the target area, so that the detection model can output the image to be detected corresponding to the target area according to the rectangular size, effectively improving the efficiency and accuracy of image extraction.

[0045] It should be noted that the image to be detected corresponds to the image corresponding to the conveyor belt area, and the image to be detected corresponds to a preset size, so as to detect the water-coal mixture during the transmission process, and the mask image and the image to be detected both correspond to preset sizes, so as to ensure that the detection process is stable and continuous. Among them, the area label includes a rectangular annotation area set for the conveyor belt area, so that the detection model can extract the image to be detected corresponding to the conveyor belt area from the initial image.

[0046] S203: Obtain all candidate detection boxes that match mask regions in which pixels in the segmented image are continuously distributed and whose number exceeds a number threshold.

[0047] Specifically, the pixels in the segmented image are traversed, mask regions where pixels are continuously distributed and the total number of pixels exceeds a number threshold are determined from the segmented image, and matching candidate detection boxes are generated for the corresponding mask regions.

[0048] In some implementation scenarios, all candidate detection frames matched by mask areas in a segmented image where pixels are continuously distributed and the number exceeds a quantity threshold are obtained, including: traversing the pixels in the segmented image until the mask value corresponding to the mask area is traversed, and searching for other mask values ​​in a preset direction with the current mask value as the center; determining the total number of mask values ​​found this time, and when the total number of mask values ​​exceeds the quantity threshold, obtaining the circumscribed rectangles corresponding to all mask values ​​found this time to obtain candidate detection frames; returning to the step of traversing the pixels in the segmented image until the mask value corresponding to the mask area is traversed, and searching for other mask values ​​in a preset direction with the current mask value as the center, until all pixels in the segmented image are traversed.

[0049] Specifically, the pixels in the segmented image are traversed from a preset position, wherein the preset position is usually the upper left corner of the image, and the present application does not impose any specific restrictions on this. After the mask value corresponding to the mask area is traversed in the segmented image, the pixel coordinates corresponding to the current mask value are determined, and other mask values ​​are searched one by one in a preset direction with the current mask value as the center until no mask value can be obtained, thereby ensuring that the mask value found in a single time has continuity.

[0050] Furthermore, the total number of mask values ​​corresponding to all the mask values ​​found this time is determined, and it is judged whether the total number of mask values ​​exceeds the quantity threshold. When the total number of mask values ​​exceeds the quantity threshold, the circumscribed rectangles corresponding to all the mask values ​​found this time are obtained to obtain the candidate detection frame. When the total number of mask values ​​does not exceed the quantity threshold, the corresponding area is eliminated to avoid marking of too small areas and reduce the probability of false alarms due to other interference factors.

[0051] It can be understood that by continuing to traverse the pixels that have not been traversed in the segmented image until all the pixels in the segmented image are traversed, all the candidate detection boxes in the segmented image can be obtained.

[0052] For illustration purposes, see Figure 3 , Figure 3 This is a schematic diagram of an application scenario of an implementation scheme of a candidate detection frame of the present application. Taking the outermost rectangular frame as a conveyor belt and the irregular shape as the mask area corresponding to the water surface as an example, the pixels in the segmented image are traversed, and the mask areas whose total mask values ​​exceed the quantity threshold and exist independently are marked using the minimum circumscribed rectangle to obtain candidate detection frames. Among them, when some mask areas are relatively close, there are overlapping areas between the candidate detection frames.

[0053] It should be noted that traversing the pixels in the segmented image until the mask value corresponding to the mask area is traversed, and searching for other mask values ​​in a preset direction with the current mask value as the center, includes: obtaining an access matrix corresponding to the segmented image and initializing the access matrix; wherein the elements in the access matrix match the pixels in the segmented image, and the elements in the initialized access matrix correspond to the initial values; traversing the pixels whose elements in the access matrix are the initial values, and when the elements in the access matrix are the initial values ​​and the mask value is obtained, adjusting the corresponding initial value in the access matrix to the indicated value, searching for other mask values ​​in a preset direction with the current indicated value as the center, and adjusting the initial value of the corresponding position to the indicated value.

[0054] Specifically, an access matrix matching the size of the segmented image is obtained, each element in the access matrix corresponds to a pixel in the segmented image, the access matrix is ​​initialized, and the elements in the initialized access matrix correspond to initial values.

[0055] Furthermore, the pixels whose elements in the access matrix are initial values ​​are traversed. When the elements in the access matrix are initial values ​​and mask values ​​are obtained, the corresponding initial values ​​in the access matrix are adjusted to indication values, and other mask values ​​are searched in a preset direction with the current indication value as the center. When other mask values ​​are found, the corresponding initial values ​​in the access matrix are adjusted to indication values. Therefore, when the mask value is found, by modifying the initial value in the access matrix to the indication value, it is determined that the pixel at the corresponding position has been searched, thereby avoiding repeated searches, and the total number of mask values ​​found this time can be determined by counting the number of indication values ​​obtained this time.

[0056] It can be understood that impurities include water, wherein the segmented image is fused with a mask image, and the mask image is obtained based on the mask matrix mask_water. In the mask matrix mask_water, 0 indicates that no mask is set, and non-0 indicates that a mask is set. First, construct a visit matrix mask_visit of the same size as the mask matrix mask_water, and initialize all to 0, initialize the water surface pixel count pixel_count to 0, and the number of independent and unconnected water surfaces water_count to 0.

[0057] It can be understood that when the mask matrix mask_water is traversed and a coordinate [i, j] is found such that mask_water[i, j] is not 0 and mask_visit[i, j] is 0, it means that the pixel at this point in the mask is classified as the water surface and has not been visited, so mask_visit[i, j] is set to 1, and the water surface pixel count pixel_count is increased by 1. At the same time, with [i, j] as the center, the four directions of forward [i-1, j], backward [i+1, j], left [i, j-1], and right [i, j+1] are used as the preset directions to perform the same judgment and operation. When encountering a mask value that has not been visited, the water surface pixel count pixel_count is increased by 1, and the minimum bounding rectangle is continuously updated until no points that meet the requirements can be found in all directions.

[0058] Furthermore, the minimum bounding rectangle and pixel count pixel_count of the current water surface are output. If pixel_count is less than the quantity threshold, it means that the independent water surface pixel can be discarded. If pixel_count is greater than or equal to the quantity threshold, the water surface count water_count is increased by 1.

[0059] S204: Determine a target detection frame of the impurity from all candidate detection frames based on the overlap between all candidate detection frames.

[0060] Specifically, based on the overlap between all candidate detection frames, all candidate detection frames are screened to obtain target detection frames that match the impurities that need to be independently identified, thereby independently identifying the impurities in the mixture in the image to be detected and improving the accuracy of mixture detection.

[0061] In some implementation scenarios, based on the overlap between all candidate detection frames, a target detection frame of the impurity is determined from all candidate detection frames, including: sorting all candidate detection frames according to the total number of pixels in the detection frame to obtain a candidate detection frame set; wherein the candidate detection frame in the candidate detection frame set that has not been matched and has the largest total number of pixels is the detection frame to be matched; based on the overlap between the detection frame to be matched and other unmatched candidate detection frames, the candidate detection frame set is updated; wherein the candidate detection frame whose overlap with the detection frame to be matched meets a preset condition is eliminated; returning to the step of sorting all candidate detection frames according to the total number of pixels in the detection frame to obtain the candidate detection frame set, until all candidate detection frames in the candidate detection frame set are traversed, and the candidate detection frame in the final candidate detection frame set is used as the target detection frame.

[0062] Specifically, all candidate detection frames are sorted according to the total number of pixels covered by the detection frames to obtain a candidate detection frame set, and the candidate detection frame that has not been matched and has the largest number of pixels in the candidate detection frame set is used as the detection frame to be matched. Therefore, when the candidate detection frame set is sorted in descending order according to the total number of pixels, the detection frame to be matched in the initial state is the first candidate detection frame.

[0063] Furthermore, the degree of overlap between the detection frame to be matched and other unmatched candidate detection frames is determined, that is, the degree of overlap between the detection frame to be matched and other candidate detection frames whose total number of pixels is less than that of the detection frame to be matched is determined, and it is judged whether the candidate detection frames whose degree of overlap with the detection frame to be matched meets the preset conditions are included. If included, the corresponding candidate detection frames are eliminated, thereby updating the candidate detection frame set. If not included, the candidate detection frame set remains unchanged, thereby improving the matching efficiency by sorting and determining the detection frames to be matched.

[0064] It can be understood that all candidate detection frames in the candidate detection frame set are traversed to obtain the final retained candidate detection frame, and the candidate detection frame in the final candidate detection frame set is used as the target detection frame.

[0065] It should be noted that based on the degree of overlap between the detection frame to be matched and other unmatched candidate detection frames, the candidate detection frame set is updated, including: obtaining the number of overlapping pixels corresponding to the overlapping area between the detection frame to be matched and other unmatched candidate detection frames, and determining the ratio of the number of overlapping pixels to the total number of pixels in the corresponding candidate detection frames; eliminating the candidate detection frames whose ratios exceed the ratio threshold, and updating the candidate detection frame set.

[0066] Specifically, determine whether the detection frame to be matched includes an overlapping area with other unmatched candidate detection frames. If an overlapping area is included, obtain the number of overlapping pixels corresponding to the overlapping area, determine the ratio of the number of overlapping pixels to the total number of pixels in the candidate detection frames with a smaller number, and remove the candidate detection frames whose ratio exceeds the ratio threshold, thereby updating the candidate detection frames, so that when the candidate detection frame is small and mostly overlaps with other larger candidate detection frames, only the larger candidate detection frame is retained, so that the target detection frame finally identified is more representative, and most of the areas in the removed candidate detection frames can also be identified in the target detection frame. The above process is expressed by the formula as follows:

[0067]

[0068] Among them, the numerator represents the number of overlapping pixels corresponding to the overlapping area of ​​the two candidate detection frames, and the denominator represents the total number of pixels included in the smaller candidate detection frame of the two candidate detection frames.

[0069] Understandably, see Figure 3 and Figure 4 , Figure 4 This is a schematic diagram of an application scenario of an embodiment of the target detection frame of the present application. Taking the ratio threshold as 50% as an example, Figure 4 Comparison Figure 3 Among the candidate detection frames, the smallest candidate detection frame is eliminated.

[0070] In this embodiment, the image to be detected is input into the segmentation model so that the segmentation model can segment the mixture in the image to be detected, wherein the segmentation model is obtained by supervised training using training samples with impurity labels, and the impurity labels include polygonal labeled areas set for impurities, so that the segmentation model can segment the impurities in the mixture according to the polygons, effectively improving the efficiency and accuracy of image segmentation, and the segmentation model is configured with a mask matrix for the image to be detected, and the mask image is generated based on the mask matrix, and the mask image and the image to be detected are fused to obtain a segmented image. After traversing the mask value corresponding to the mask area in the segmented image, the pixel coordinates corresponding to the current mask value are determined, and other mask values ​​are searched one by one in a preset direction with the current mask value as the center until the mask value cannot be obtained, thereby ensuring that the mask value found in a single time has continuity, when the total number of mask values ​​exceeds the number threshold, the circumscribed rectangle corresponding to all the mask values ​​found this time is obtained to obtain a candidate detection frame, and when the total number of mask values ​​does not exceed the number threshold, the corresponding area is eliminated, thereby avoiding the identification of too small areas, and reducing the probability of false alarms caused by other interference factors. All candidate detection frames are sorted according to the total number of pixels covered by the detection frames to obtain a set of candidate detection frames. The candidate detection frames that have not been matched and have the largest number of pixels in the candidate detection frame set are used as the detection frames to be matched, so as to improve the matching efficiency by sorting and determining the detection frames to be matched. The overlap between the detection frame to be matched and other unmatched candidate detection frames is determined, that is, the overlap between the detection frame to be matched and other candidate detection frames with a total number of pixels less than itself is determined, and the candidate detection frames whose overlap with the detection frame to be matched meets the preset conditions are eliminated, so that the target detection frame finally identified is more representative.

[0071] See also Figure 5 , Figure 5 It is a structural diagram of an embodiment of an electronic device of the present application, wherein the electronic device 30 includes a memory 301 and a processor 302 coupled to each other, wherein the memory 301 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0072] See also Figure 6 , Figure 6It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0073] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0074] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0076] The above description is only an implementation method of the present application, and does not limit the protection scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the protection scope of the present application.

Claims

1. A mixture detection method, characterized in that: The method comprises: Acquire an image to be detected, and segment the mixture in the image to be detected according to polygons using a segmentation model to obtain a segmented image; wherein the mixture includes impurities and target objects, the segmented image includes a mask area of ​​the impurities, the target object corresponds to a coal mine, and the impurities include water; Obtain all candidate detection frames matched by mask regions in which pixels in the segmented image are continuously distributed and whose number exceeds a number threshold; Based on the overlap between all the candidate detection frames, a target detection frame of the foreign object is determined from all the candidate detection frames.

2. The mixture detection method according to claim 1, characterized in that: The step of acquiring an image to be detected and segmenting the mixture in the image to be detected according to polygons using a segmentation model to obtain a segmented image includes: Acquire an image to be detected, segment the mixture in the image to be detected using a segmentation model, and determine a mask image matched by the mixture in the image to be detected; wherein the segmentation model is obtained by training using training samples with impurity labels, and the impurity labels include polygonal annotated areas set for impurities; The mask image and the image to be detected are fused to obtain the segmented image; wherein the segmentation model is configured with a mask matrix for the image to be detected, and the mask image is generated based on the mask matrix.

3. The mixture detection method according to claim 2, characterized in that: The image to be detected is obtained based on the following steps: An initial image captured in a target scene is obtained, and an image to be detected corresponding to a target area is extracted from the initial image using a detection model; wherein the detection model is trained using training samples with area labels, and the area labels include rectangular annotated areas set for the target area.

4. The mixture detection method according to claim 1, characterized in that: The step of obtaining all candidate detection frames matched by mask regions in which pixels in the segmented image are continuously distributed and the number of pixels exceeds a threshold value includes: Traversing pixels in the segmented image until a mask value corresponding to the mask area is traversed, and searching for other mask values ​​in a preset direction with the current mask value as the center; Determine the total number of mask values ​​found this time, and when the total number of mask values ​​exceeds a quantity threshold, obtain the circumscribed rectangles corresponding to all mask values ​​found this time to obtain the candidate detection box; Return to the step of traversing the pixels in the segmented image until the mask value corresponding to the mask area is traversed, and searching for other mask values ​​in a preset direction with the current mask value as the center until all the pixels in the segmented image are traversed.

5. The mixture detection method according to claim 4, characterized in that: The traversing of pixels in the segmented image until a mask value corresponding to the mask area is reached, and searching for other mask values ​​in a preset direction with the current mask value as the center, includes: Obtaining an access matrix corresponding to the segmented image, and initializing the access matrix; wherein the elements in the access matrix match the pixels in the segmented image, and the elements in the initialized access matrix correspond to initial values; Traverse the pixels whose elements in the access matrix are initial values, and when the elements in the access matrix are initial values ​​and the mask value is obtained, adjust the corresponding initial value in the access matrix to the indication value, search for other mask values ​​in a preset direction with the current indication value as the center, and adjust the initial value of the corresponding position to the indication value.

6. The mixture detection method according to claim 1, characterized in that: The determining the target detection frame of the impurity from all the candidate detection frames based on the overlap between all the candidate detection frames includes: Sort all the candidate detection frames according to the total number of pixels in the detection frames to obtain a set of candidate detection frames; wherein the candidate detection frame in the set of candidate detection frames that has not been matched and has the largest total number of pixels is the detection frame to be matched; Based on the overlap between the detection frame to be matched and other unmatched candidate detection frames, the candidate detection frame set is updated; wherein the candidate detection frames whose overlap with the detection frame to be matched meets a preset condition are eliminated; Return to the step of sorting all the candidate detection frames according to the total number of pixels in the detection frames to obtain a candidate detection frame set, until all the candidate detection frames in the candidate detection frame set are traversed, and the final candidate detection frame in the candidate detection frame set is used as the target detection frame.

7. The mixture detection method according to claim 6, characterized in that: The updating of the candidate detection frame set based on the overlap between the to-be-matched detection frame and other unmatched candidate detection frames includes: Obtaining the number of overlapping pixels corresponding to the overlapping area between the to-be-matched detection frame and other unmatched candidate detection frames, and determining the ratio of the number of overlapping pixels to the total number of pixels in the corresponding candidate detection frame; The candidate detection frames whose ratio exceeds the ratio threshold are eliminated, and the candidate detection frame set is updated.

8. The mixture detection method according to any one of claims 1 to 7, characterized in that: The image to be detected corresponds to an image corresponding to a conveyor belt area, and the image to be detected corresponds to a preset size.

9. An electronic device, characterized in that: include: A memory and a processor coupled to each other, wherein the memory stores program data, and the processor calls the program data to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program data stored thereon, characterized in that: When the program data is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images