Computer vision-based detection method and system for uncovered muck trucks

Abstract

The invention relates to the field of image processing technology, and in particular to a method and system for detecting uncovered muck trucks based on computer vision. The method comprises: dividing a grayscale image of a compartment surface of a muck truck into blocks, preliminarily evaluating the possibility that the blocks belong to uncovered areas, calculating a first probability that the pixel point belongs to the uncovered area according to the difference between adjacent pixel points of any pixel point and other pixel points in a gradient direction, sorting the first probabilities corresponding to the pixel point and its neighboring pixel points, calculating a second probability that the pixel point belongs to the uncovered area according to the difference between the first probabilities and the spatial distance between the corresponding pixel points, determining the uncovered pixel point according to the size of the second probability and updating the grayscale value, segmenting the grayscale image using an Otsu method, judging whether there is a covering anomaly according to the segmentation result, thereby improving the segmentation accuracy of the Otsu method and the detection accuracy of the muck truck covering state.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for detecting uncovered muck trucks based on computer vision. Background Art

[0002] In modern urban management, computer vision technology has been applied to the supervision of muck truck transportation to achieve real-time detection of environmental compliance. It is mainly through high-definition cameras installed at key traffic nodes to continuously capture and analyze images of muck truck transportation, segment the images, and judge the condition of the muck truck's tarpaulin based on the segmented areas. It can promptly discover the situation where the tarpaulin is not covered as required, and promptly warn and notify relevant management personnel to prevent the spillage of muck and dust pollution. Compared with traditional manual inspections, the automated visual monitoring system has all-weather operation capabilities, greatly improving the efficiency and scope of monitoring, reducing manpower requirements, and avoiding the uncertainty of subjective judgment. It can effectively identify the condition of the muck truck's tarpaulin.

[0003] For example, the patent application document with the existing publication number CN116665146A provides a vehicle tarpaulin detection method, device, electronic device and storage medium, and the method includes: obtaining an original vehicle tarpaulin image, and preprocessing the original vehicle tarpaulin image; performing regional calibration on the preprocessed original vehicle tarpaulin image and using it as the calibration result; performing target feature extraction on the calibrated original vehicle tarpaulin image to obtain a feature image containing the target feature; performing semantic segmentation on the defective part in the feature image, and constructing a defect detection data set based on the segmentation result; obtaining a vehicle tarpaulin image to be detected, detecting the vehicle tarpaulin image to be detected based on the constructed target detection model, and obtaining a vehicle tarpaulin defect detection result.

[0004] The above method identifies the vehicle covering status through semantic segmentation, but the training and inference process of its deep learning model requires a lot of computing resources and labeled data to adapt to the semantic understanding of different scenarios, which brings higher computational complexity. In view of the resource requirements and computational complexity of semantic segmentation, the Otsu threshold segmentation method is usually used to handle the image segmentation task in the process of muck truck covering recognition and detection. As a classic segmentation technology, the Otsu method has the characteristics of small computational amount, simplicity and high efficiency.

[0005] However, due to the uneven distribution of the slag under the cover, the surface of the cover appears uneven. In addition, the uncovered slag area shows highly random grayscale changes. This change is caused by the diversity of the slag itself and the fluctuation of lighting conditions, which causes significant differences in the grayscale values ​​of corresponding pixels in the image. Under the combined effect of these factors, the traditional Otsu threshold segmentation method has low accuracy in image segmentation and cannot accurately distinguish between covered and uncovered areas, which in turn affects the accurate judgment of the covering status of the dump truck and reduces the recognition rate of violations. Summary of the invention

[0006] In order to solve the problem that in the process of muck truck cover recognition, the traditional Otsu threshold segmentation method has low accuracy in image segmentation and cannot accurately distinguish covered and uncovered areas, which affects the accurate judgment of the muck truck cover status, the present invention proposes a muck truck uncovered detection method and system based on computer vision, which specifically adopts the following scheme:

[0007] On the one hand, the present invention provides a method for detecting uncovered muck trucks based on computer vision, comprising:

[0008] A grayscale image of the surface of the compartment of the muck truck is collected and divided into blocks. The possibility that the block belongs to an uncovered area is evaluated by using the fluctuation degree of the grayscale value, the fluctuation degree of the gradient value, and the span of the grayscale value within the block. The possibility is positively correlated with the fluctuation degree of the grayscale value, the fluctuation degree of the gradient value, and the span of the grayscale value.

[0009] According to the difference between any pixel point in the block and the adjacent pixel points of the other pixel points in the gradient direction, and in combination with the possibility and the gradient direction angle between the pixel point and the other pixel points, a first probability that the pixel point belongs to the uncovered area is calculated, wherein the first probability is positively correlated with the gradient direction angle, the possibility, and the difference between the adjacent pixel points;

[0010] Sorting the first probabilities that the pixel point and all its neighboring pixel points belong to the uncovered area, and after the sorting, calculating the second probability that the pixel point belongs to the uncovered area according to the difference between adjacent first probabilities and the spatial distance between pixels corresponding to adjacent first probabilities, wherein the second probability is positively correlated with the spatial distance and the difference between the first probability;

[0011] Uncovered pixels are determined by the magnitude of the second probability, and gray values ​​of the uncovered pixels are updated. The grayscale image is segmented using the Otsu method, and whether there is a covering anomaly is determined based on the segmentation result.

[0012] The above technical solution comprehensively considers the gray value fluctuation degree, the gradient value fluctuation degree and the gray value span to evaluate the possibility that the block belongs to the uncovered area. This evaluation process utilizes the statistical characteristics of the grayscale and gradient changes in the uncovered area, which is contrasted with the covered area, laying the foundation for the subsequent pixel point analysis. Then, the first probability that each pixel point in the block belongs to the uncovered area is further calculated. The calculation of this probability comprehensively considers the difference between each independent pixel point and other pixel points. In order to make the first probability more accurate, the neighborhood pixels of each pixel point are analyzed. Based on the difference between the neighborhood pixels and the corresponding image, the probability of the first pixel point belonging to the uncovered area is calculated. The spatial distance of the pixels is calculated, and the second probability is calculated. The calculation of the second probability further integrates the spatial relationship between the pixels, enhances the recognition ability of the uncovered area, and accurately screens out the uncovered pixels according to the second probability, and updates the grayscale values ​​of the uncovered pixels, which strengthens the characteristics of the uncovered area. The updated grayscale image is segmented using the Otsu method to obtain a more accurate segmentation result, which improves the segmentation accuracy of the Otsu method and can effectively distinguish between covered and uncovered areas. Through the preliminary pixel-level analysis and grayscale value update, the segmentation effect of the Otsu method is optimized, and the detection accuracy of the covered state of the dump truck is improved.

[0013] Furthermore, the calculation formula for the second probability that the pixel point belongs to the uncovered area is:

[0014] ;

[0015] In the formula, For the The first The second probability that the pixel belongs to the uncovered area, For the The first The mean of the first probabilities that all neighboring pixels of a pixel belong to the uncovered area, For the said The pixel Neighborhood pixels and The distance between neighboring pixels, For the said The pixel The first probability that the neighboring pixel belongs to the uncovered area, For the said The pixel The first probability that the neighboring pixel belongs to the uncovered area, To find the absolute value sign.

[0016] The above technical solution analyzes the characteristics of the neighborhood pixels based on the first probability that the pixel point belongs to the uncovered area. By quantifying the fluctuation characteristics and spatial position relationship of the first probability of the pixels in the neighborhood, a more accurate second probability that the pixel point belongs to the uncovered area can be obtained.

[0017] Furthermore, the first probability that the pixel point belongs to the uncovered area satisfies the following relationship:

[0018] ;

[0019] In the formula, For the The first The first probability that the pixel belongs to the uncovered area, For the The probability that a block belongs to the uncovered area, For the In the block The serial numbers of other pixels besides pixels, For the The number of other pixels in the blocks, For the The first The gradient direction of the pixel is The angle between the gradient directions of pixels, For the The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction, For the The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction, To find the absolute value sign, is a sine function.

[0020] The above technical solution analyzes each pixel independently, and based on the gradient direction difference and grayscale difference between each pixel and other pixels, it can preliminarily evaluate the difference between each pixel and other pixels in the block. The greater the difference, the more likely the pixel is to be an uncovered area.

[0021] Furthermore, the method for determining uncovered pixels according to the magnitude of the second probability is:

[0022] A second probability threshold is preset. If the second probability that a certain pixel point in the grayscale image belongs to an uncovered area is greater than or equal to the second probability threshold, the pixel point is an uncovered pixel point; otherwise, the pixel point is a covered pixel point.

[0023] Furthermore, the sorting is based on an ascending order.

[0024] Furthermore, the possibility that the block belongs to the uncovered area satisfies the following relationship:

[0025] ;

[0026] In the formula, For the The probability that a block belongs to the uncovered area, is the span of the grayscale value, is the fluctuation degree of the gray value, is the fluctuation degree of the gradient value, is the normalization function;

[0027] The span of the grayscale value is the absolute value of the difference between the maximum grayscale value and the minimum grayscale value of all pixels in the block; the fluctuation degree of the grayscale value is the variance of the grayscale values ​​of all pixels in the block; the fluctuation degree of the gradient value is the variance of the gradient values ​​of all pixels in the block.

[0028] In the above technical solution, the grayscale value span represents the absolute value of the difference between the maximum grayscale value and the minimum grayscale value of all pixels in the block. A larger grayscale value span usually means that the block contains rich detail information, which may be one of the characteristics of the uncovered area; a higher grayscale value fluctuation degree indicates that the grayscale distribution in the block is uneven, and there may be edges or textures, which is also a sign of the uncovered area; the gradient value reflects the grayscale change rate between pixels, and a higher gradient value fluctuation degree means that there are more edges or details in the block, which is usually consistent with the characteristics of the uncovered area; by comprehensively considering the grayscale value span, the grayscale value fluctuation degree and the gradient value fluctuation degree, the formula can more comprehensively evaluate the possibility that the block belongs to the uncovered area. This comprehensive consideration enables the algorithm to more accurately distinguish between the uncovered area and the covered area.

[0029] Furthermore, the size of the block is Pixel size, is the default value.

[0030] Further, the updating of the grayscale value of the uncovered pixel is to multiply the grayscale value of the uncovered pixel by the second probability to obtain a new grayscale value of the uncovered pixel.

[0031] The above technical solution enhances the contrast between the uncovered area and the covered area by multiplying the grayscale value of the uncovered pixel with the second probability as the new grayscale value. The grayscale value of the pixel with a higher second probability will be amplified, making it easier to be identified as an uncovered area in subsequent image segmentation, thereby improving the accuracy of detection.

[0032] Furthermore, the method for judging whether there is a covering anomaly according to the segmentation result is:

[0033] If the segmentation result does not contain uncovered areas, then the muck truck does not have a covering anomaly; if the segmentation result includes uncovered areas and covered areas;

[0034] If the area ratio of the uncovered area in the grayscale image is greater than or equal to the preset area ratio threshold, the dump truck has a covering abnormality; if the area ratio of the uncovered area in the grayscale image is less than the preset area ratio threshold, the dump truck does not have a covering abnormality.

[0035] The above technical solution provides an effective and flexible way to determine whether there is any abnormal covering on the dump truck. By flexibly adjusting the area ratio threshold, the best judgment effect can be achieved in different environments. For example, in some strict regulatory scenarios, a lower threshold can be set to increase the sensitivity of detection; in other scenarios, a higher threshold can be set to reduce false alarms.

[0036] On the other hand, the present invention also provides a computer vision-based uncovered dump truck detection system, the defect detection system comprising a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any step of the detection method.

[0037] The present invention has the following effects:

[0038] The present invention comprehensively analyzes grayscale fluctuations, gradient changes and grayscale spans, accurately evaluates the uncovered possibility of image blocks, lays a foundation for pixel-level analysis, effectively distinguishes covered and uncovered areas, calculates the first probability that pixels are uncovered, and calculates the second probability through neighborhood analysis, strengthens spatial relationship integration, improves the recognition accuracy of uncovered areas, updates the grayscale values ​​of uncovered pixels, enhances feature contrast, optimizes the segmentation effect of the Otsu method, can effectively distinguish covered / uncovered areas, and improves the accuracy of cover detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0040] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0042] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0043] Reference Figure 1 A method for detecting uncovered muck trucks based on computer vision includes steps S1-S4, which are as follows:

[0044] S1: Collect the surface image of the muck truck compartment.

[0045] In the process of implementing the supervision of muck truck transportation, the first step is to capture the real-time image of the muck truck through the high-definition cameras deployed at key traffic nodes. After collecting the image of the car body surface, it is necessary to analyze the covering status of the muck truck during transportation to determine whether the muck truck is covered in accordance with regulations to prevent the spillage of muck and dust pollution. According to the actual covering status of the muck truck, the car body surface image can be summarized into the following three typical situations:

[0046] The first type is complete covering: in the complete covering state, the entire surface of the muck truck is tightly covered by the covering material, forming a relatively smooth and uniform surface. In the image, the covered area shows consistent texture and color, and the grayscale value distribution is relatively concentrated with a small variation. In this state, the image features low grayscale difference, low texture complexity and high uniformity. When processing this type of image, it is necessary to be able to detect whether there are large uniform areas in the image, which can be effectively identified by analyzing grayscale consistency, texture features and edge strength.

[0047] The second type is partial covering: In the partial covering state, only a part of the surface of the muck truck compartment is covered by the covering material, and the rest is exposed. In this case, the image will have both covered areas and uncovered areas with different smoothness and grayscale distributions. The uncovered areas may have significant changes in grayscale value and texture due to the irregular distribution of the muck and the uneven surface. When processing partially covered images, it is necessary to be able to distinguish the boundaries between covered and uncovered areas.

[0048] The third type is completely uncovered. When the surface of the muck truck compartment is completely uncovered, the image will show the full picture of the exposed muck. The uncovered muck area usually has a high degree of grayscale and texture changes. Due to the uneven accumulation and natural form of the muck, the image will present complex shadows and high-contrast texture features. It mainly depends on whether there are large areas of irregular texture areas and high grayscale differences in the detection image, and combined with the overall structure of the image to judge whether the muck truck is in a completely uncovered state.

[0049] In order to facilitate subsequent analysis, after the image of the surface of the muck truck compartment is collected, it is converted into a grayscale image. This is because the grayscale image can simplify the image processing process and reduce the computational burden. The grayscale image is composed of the grayscale level of pixels, and the value of each pixel represents a brightness level from 0 (black) to 255 (white).

[0050] S2: Divide the image into blocks and calculate the possibility that each block belongs to the uncovered area.

[0051] In order to accurately analyze the image of the car surface, the image segmentation method is used to divide the entire image into multiple blocks of fixed size.

[0052] Since there are significant differences in the image features between covered areas and uncovered areas, the grayscale value distribution of pixels in covered areas is relatively uniform and the grayscale changes slowly due to the relatively smooth and flat surface material; while in uncovered areas, due to the irregularity and surface roughness of the slag itself, the grayscale value of pixels changes dramatically and the grayscale distribution is highly random and inconsistent.

[0053] Based on this observation, the probability of the existence of debris in the block, that is, the possibility of each block belonging to an uncovered area, can be inferred by analyzing the grayscale value and gradient value of the pixels in each block. Specifically, the possibility of the block belonging to an uncovered area is evaluated by utilizing the fluctuation degree of the grayscale value and the gradient value as well as the span of the grayscale value. The possibility is positively correlated with the fluctuation degree of the grayscale value, the fluctuation degree of the gradient value and the span of the grayscale value. The span of the grayscale value is the absolute value of the difference between the maximum grayscale value and the minimum grayscale value of all pixels in the block. The fluctuation degree of the grayscale value is the variance of the grayscale values ​​of all pixels in the block. The fluctuation degree of the gradient value is the variance of the gradient values ​​of all pixels in the block.

[0054] To quantify this difference, Taking a block as an example, first obtain the grayscale value and gradient value of all pixels in the block, and calculate the possibility that the block belongs to the uncovered area:

[0055]

[0056] In the formula, Indicates The probability that a block belongs to the uncovered area, Indicates The absolute value of the difference between the maximum grayscale value and the minimum grayscale value of all pixels in a block, It reflects the grayscale span of all pixels in the block. The smaller the grayscale span, the more likely it is to be a covered area, and the larger the grayscale span, the more likely it is to be an uncovered area. The grayscale difference of pixels in the uncovered area is large, and the grayscale value changes greatly, so it is used Indicates The grayscale variance of the pixels in the first block is used to reflect the The smaller the variance, the more concentrated the grayscale distribution is, and it is more likely to be a smooth covered surface, that is, it is more likely to be a covered area. The larger the variance, the more likely it is an uncovered area. Indicates The larger the variance of the gradient value of the pixel points in the block, the greater the change in the grayscale value of the pixel points in the block, and the more likely it is that there is an uncovered area containing slag. is the normalization function.

[0057] This formula is converted into and Add and pass By normalizing the function, we can get an index that combines grayscale consistency and grayscale variation. Then, we can compare this index with Multiply them together to get The possibility that each block is a slag block, The larger the value, the The blocks are more likely to be uncovered areas.

[0058] This formula takes into account the span, consistency and variation of the grayscale value, and can make a preliminary overall assessment of the possibility that each block is a muck block (uncovered area). Block processing allows the algorithm to analyze in a local area, improving sensitivity to details, while the combination of grayscale and gradient values ​​ensures that the algorithm can distinguish the different features of covered and uncovered areas, thereby improving the accuracy and reliability of muck truck cover detection.

[0059] S3: Evaluate the probability that each pixel in the block belongs to the uncovered area.

[0060] S31: Calculate a first probability that the pixel point belongs to an uncovered area based on the difference between adjacent pixels of any pixel point and other pixel points in the gradient direction in the block and in combination with the possibility and the angle between the pixel point and other pixel points in the gradient direction.

[0061] In the supervision of muck truck transportation, covered areas and uncovered areas show significantly different characteristics in the image. Even if the covered area is affected by the muck below, its surface still maintains a certain degree of smoothness and consistency, which means that in the image, the grayscale change amplitude and gradient direction of continuous pixels show a certain regularity and consistency. On the contrary, in the uncovered area, that is, the exposed muck surface, the grayscale value of the pixel changes irregularly and the gradient direction changes disorderly.

[0062] In order to further improve the accuracy of the detection of the covering status of dump trucks, each block is analyzed in detail. Specifically, by analyzing the first probability that each pixel in each block belongs to the uncovered area, it is possible to more accurately identify which areas composed of pixels in the block are truly uncovered areas, thereby improving the accuracy of violation detection. By carefully analyzing each pixel, the misjudgment of the covering detection process can be reduced, ensuring that neither excessive warning prompts are generated due to misjudgment nor important information is missed.

[0063] This step is to preliminarily evaluate the first probability that each pixel in the block belongs to the uncovered area based on the possibility that each block belongs to the uncovered area and the grayscale difference of the pixels in the block.

[0064] First The first For example, the first The first The first probability that a pixel belongs to the uncovered area satisfies the following formula:

[0065]

[0066] In the formula, Indicates The first The first probability that a pixel point belongs to an uncovered area. The larger its value is, the greater the possibility that the pixel point belongs to an uncovered area. Indicates The probability that a block belongs to the uncovered area, Indicates The first The gradient direction of the pixel is related to the The angle between the gradient directions of the two pixels. The smaller the angle, the closer the gradient directions of the two pixels are. Indicates The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction is Indicates The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction is Indicates The total number of pixels in each block, express The value of the sine function, To find the absolute value sign.

[0067] In this formula, Reflects the The first pixels and The grayscale value difference of the pixel points in the gradient direction of the two pixels. The larger the value, the greater the difference in the grayscale change of the two pixels in the gradient direction. Reflects the The first pixels and The difference in the gradient direction of the two pixels, the larger the value, the greater the difference in the gradient direction of the two pixels. Comprehensive reflection The first pixels and The difference of pixels is Reflect the The first Pixels and The difference between other pixels in the block, the larger the value, the The more likely a pixel is to be in an uncovered position, the more likely it is that the pixel belongs to an uncovered area.

[0068] This step further refines the analysis to each pixel in each block to more accurately evaluate the second probability that each pixel in the block belongs to the uncovered area.

[0069] S32: Sort the first probabilities that the pixel point and all its neighboring pixel points belong to the uncovered area, and after sorting, calculate the second probability that the pixel point belongs to the uncovered area based on the difference between adjacent first probabilities and the spatial distance between the pixels corresponding to the adjacent first probabilities.

[0070] Step S31 preliminarily evaluates the first probability that the pixel belongs to the uncovered area by calculating the first probability that each pixel in the block belongs to the uncovered area. This evaluation method has a high evaluation accuracy when the block only contains covered areas or the block only contains uncovered areas. However, when the block contains both covered areas and uncovered areas, according to the evaluation method in S31, the first probabilities that the pixels located in the covered area and the pixels located in the uncovered area belong to the uncovered area are basically the same, and the evaluation method is inaccurate. Step S31 only analyzes the grayscale and gradient differences of the pixels from the overall perspective of the block, but fails to fully consider the local context information of the pixels, especially the feature consistency of the pixel neighborhood. When there are complex mixed areas in the block, the evaluation is not accurate enough.

[0071] Therefore, in order to overcome the limitation of step S31, step S32 carefully analyzes the neighborhood features of the pixel points to obtain the second probability that each pixel point in the block belongs to an uncovered pixel point. The specific operations include:

[0072] The eight neighboring pixels of each pixel in the block are set as the neighboring pixels of the pixel, and the first probability that each pixel and its neighboring pixels belong to the uncovered area is obtained, and the pixels are sorted in ascending order according to the first probability.

[0073] If the pixel is located in a covered area, the first probability that the pixel belongs to an uncovered area and the first probability that the neighboring pixel belongs to an uncovered area are relatively close and change steadily. Pixels that are adjacent in the first probability should also be adjacent in spatial position, that is, the spatial distance is also close. On the contrary, if the pixel is located in an uncovered area, pixels that are adjacent in the first probability have large differences in spatial position, that is, the spatial distance is large.

[0074] Therefore, according to the first probability difference between adjacent pixels after sorting (in the first probability) and the spatial distance difference, the second probability that the pixel belongs to the uncovered area is calculated, and the formula is:

[0075]

[0076] In the formula, Indicates The first The second probability that a pixel belongs to an uncovered area is larger, The more pixels there are, the more likely they are uncovered areas.

[0077] In the formula, Indicates The mean of the first probability of all pixels in the neighborhood of a pixel belonging to the uncovered area. The size of the mean reflects the first The characteristics of the environment where the pixel is located, if The larger the The pixel point may be in an area with a large fluctuation in the soil slag rate, that is, it may be an uncovered area.

[0078] In the formula, Indicates The first The pixel in the neighborhood of pixels and The distance between pixels, that is, within the neighborhood, the first probability of the pixel belonging to the uncovered area is sorted from small to large. pixels and The distance between neighboring pixels. Indicates The first The pixel in the neighborhood of The first probability that the pixel belongs to the uncovered area, Indicates The first The pixel in the neighborhood of The first probability that the pixel belongs to the uncovered area, The difference in the first probabilities that these two pixels belong to the uncovered area reflects the volatility of the first probability. In the covered area, due to the smooth grayscale change of the pixel, the difference between the first probability that the pixel belongs to the uncovered area and the first probability that its neighboring pixel belongs to the uncovered area is small. In the uncovered area, due to the irregularity of the slag surface, the difference between the first probability that the pixel belongs to the uncovered area and the first probability that its neighboring pixel belongs to the uncovered area is large.

[0079] In the formula, through Whether the spatial positions corresponding to two adjacent pixel points in the first probability within the neighborhood are close. If the spatial positions corresponding to two adjacent pixel points in the first probability are very different, it means that the pixel point is more likely to belong to the uncovered area. Conversely, if the spatial positions corresponding to two adjacent pixel points in the first probability are close, it means that the pixel point is more likely to belong to the covered area.

[0080] S4: Determine uncovered pixels by the size of the probability, update the grayscale values ​​of the uncovered pixels, segment the grayscale image using the Otsu method, and determine whether there is a covering anomaly based on the segmentation result.

[0081] Specifically, determining the uncovered pixel point according to the magnitude of the second probability includes:

[0082] A second probability threshold of 0.9 is preset. If the second probability that a pixel point in the grayscale image belongs to an uncovered area is greater than or equal to the second probability threshold, the pixel point is an uncovered pixel point, that is, the pixel point is located in the uncovered area; otherwise, the pixel point is a covered pixel point, and the pixel point is located in the uncovered area.

[0083] Then, the grayscale value of the uncovered pixel is multiplied by the second probability as the new grayscale value of the uncovered pixel to complete the grayscale value update of the pixel and obtain a grayscale image with stronger features of the uncovered area. The updated image is then subjected to Otsu threshold segmentation to accurately identify and segment the uncovered area.

[0084] Finally, determine whether the dump truck has a covering abnormality based on the segmentation result: if the segmentation result does not include an uncovered area, then the dump truck does not have a covering abnormality; if the segmentation result includes an uncovered area and a covered area, the preset area ratio threshold is 10%. If the area ratio of the uncovered area in the grayscale image is greater than or equal to the preset area ratio threshold, the dump truck has a covering abnormality, and the relevant personnel are notified in time; if the area ratio of the uncovered area in the grayscale image is less than the preset area ratio threshold, the dump truck does not have a covering abnormality.

[0085] The present invention also provides a computer vision-based uncovered muck truck detection system, the defect detection system includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement any step of the detection method.

[0086] In the description of this specification, "plurality" or "several" means at least two, such as two, three or more, etc., unless otherwise clearly and specifically defined.

[0087] Although this specification has shown and described a number of embodiments of the present invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Those skilled in the art will conceive of many modifications, changes and alternatives without departing from the ideas and spirit of the present invention. It should be understood that in the practice of the present invention, various alternatives to the embodiments of the present invention described herein may be employed.

Claims

1. A method for detecting uncovered muck trucks based on computer vision, characterized in that: include: A grayscale image of the surface of the compartment of the muck truck is collected and divided into blocks. The possibility that the block belongs to an uncovered area is evaluated by using the fluctuation degree of the grayscale value, the fluctuation degree of the gradient value, and the span of the grayscale value within the block. The possibility is positively correlated with the fluctuation degree of the grayscale value, the fluctuation degree of the gradient value, and the span of the grayscale value. According to the difference between any pixel point in the block and the adjacent pixel points of the other pixel points in the gradient direction, and in combination with the possibility and the gradient direction angle between the pixel point and the other pixel points, a first probability that the pixel point belongs to the uncovered area is calculated, wherein the first probability is positively correlated with the gradient direction angle, the possibility, and the difference between the adjacent pixel points; Sorting the first probabilities that the pixel point and all its neighboring pixel points belong to the uncovered area, and after the sorting, calculating the second probability that the pixel point belongs to the uncovered area according to the difference between adjacent first probabilities and the spatial distance between pixels corresponding to adjacent first probabilities, wherein the second probability is positively correlated with the spatial distance and the difference between the first probability; Determine uncovered pixels according to the magnitude of the second probability, update the grayscale values ​​of the uncovered pixels, segment the grayscale image using the Otsu method, and determine whether there is a covering anomaly according to the segmentation result; The calculation formula for the second probability that a pixel belongs to an uncovered area is: ; In the formula, For the The first The second probability that the pixel belongs to the uncovered area, For the The first The mean of the first probabilities that all neighboring pixels of a pixel belong to the uncovered area, For the The pixel Neighborhood pixels and The distance between neighboring pixels, For the The pixel The first probability that the neighboring pixel belongs to the uncovered area, For the The pixel The first probability that the neighboring pixel belongs to the uncovered area, To find the absolute value sign; The first probability that a pixel belongs to an uncovered area satisfies the following relationship: ; In the formula, For the The first The first probability that the pixel belongs to the uncovered area, For the The probability that a block belongs to the uncovered area, For the In the block The serial numbers of other pixels besides pixels. For the The number of other pixels in the blocks, For the The first The gradient direction of the pixel is The angle between the gradient directions of pixels, For the The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction, For the The first The grayscale difference between a pixel and its adjacent pixel in the gradient direction, To find the absolute value sign, is a sine function.

2. The computer vision-based uncovered muck truck detection method according to claim 1, characterized in that: The method for determining uncovered pixels by the magnitude of the second probability is: A second probability threshold is preset. If the second probability that a certain pixel point in the grayscale image belongs to an uncovered area is greater than or equal to the second probability threshold, the pixel point is an uncovered pixel point; otherwise, the pixel point is a covered pixel point.

3. The method for detecting uncovered muck trucks based on computer vision according to claim 1, characterized in that: The sorting is based on the order from smallest to largest.

4. The computer vision-based method for detecting uncovered muck trucks according to claim 1, characterized in that: The possibility that the block belongs to the uncovered area satisfies the following relationship: ； In the formula, For the The probability that a block belongs to the uncovered area, is the span of the grayscale value, is the fluctuation degree of the gray value, is the fluctuation degree of the gradient value, is the normalization function; The span of the grayscale value is the absolute value of the difference between the maximum grayscale value and the minimum grayscale value of all pixels in the block; the fluctuation degree of the grayscale value is the variance of the grayscale values ​​of all pixels in the block; the fluctuation degree of the gradient value is the variance of the gradient values ​​of all pixels in the block.

5. The computer vision-based method for detecting uncovered muck trucks according to claim 1, characterized in that: The size of the block is Pixel size, is the default value.

6. The computer vision-based method for detecting uncovered muck trucks according to claim 2, characterized in that: The updating of the grayscale value of the uncovered pixel is to multiply the grayscale value of the uncovered pixel by the second probability to obtain a new grayscale value of the uncovered pixel.

7. The computer vision-based method for detecting uncovered muck trucks according to claim 2, characterized in that: The method to determine whether there is a covering anomaly based on the segmentation results is: If the segmentation result does not contain uncovered areas, then the muck truck does not have a covering anomaly; if the segmentation result includes uncovered areas and covered areas; If the uncovered area accounts for a larger than or equal to a preset area ratio threshold in the grayscale image, the muck truck has a covering anomaly; If the area ratio of the uncovered area in the grayscale image is less than a preset area ratio threshold, there is no covering abnormality in the muck truck.

8. A computer vision-based uncovered muck truck detection system, characterized in that: The detection system comprises a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the computer program to implement the steps of the detection method according to any one of claims 1 to 7.

Images