An image detection-based road garbage identification method for unmanned cleaning vehicles

Abstract

The present application relates to the technical field of image processing, in particular to a garbage recognition method for unmanned sweeper road based on image detection. The method comprises: obtaining target clipLimit parameters of each current sub-block based on a basic clipLimit parameter adjustment factor, a garbage separability adjustment factor and an environmental interference adjustment factor; finally, based on the target clipLimit parameters of each current sub-block, using CLAHE algorithm to perform image enhancement on the current frame road image to obtain an enhanced image, and performing garbage recognition for the unmanned sweeper road based on the enhanced image. Moreover, the present application dynamically configures clipLimit parameters for each sub-block based on the characteristics of each sub-block, and performs image enhancement based on the reconfigured clipLimit parameters, which can improve the garbage recognition ability of the model, and thus can improve the accuracy and reliability of the garbage recognition for the unmanned sweeper road.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method for identifying road debris from unmanned sweepers based on image detection. Background Technology

[0002] With the acceleration of urbanization, the problems of urban road cleaning and garbage disposal have become increasingly prominent, thus giving rise to unmanned cleaning vehicles. Unmanned cleaning vehicles utilize technologies such as image recognition and automatic cleaning to significantly improve the cleaning efficiency and management level of urban roads. In general, unmanned cleaning vehicles clean up garbage by detecting and identifying garbage in collected road images. In order to ensure the accuracy of garbage detection and identification, the collected images are usually enhanced first, and then garbage is identified based on the enhanced images. Currently, the CLAHE algorithm is usually used to enhance the collected images.

[0003] However, in the traditional CLAHE (Contrast-Limited Adaptive Histogram Equalization) algorithm, the clipLimit parameter (limiting threshold) for each sub-block in the image is fixed or an empirical value. This image enhancement method with a fixed clipLimit parameter is prone to problems such as low detection accuracy or low reliability of garbage recognition, which in turn affects the working efficiency and effect of the unmanned sweeper. If the clipLimit parameter is too large, it will make the contrast enhancement of some sub-blocks too aggressive. While highlighting the garbage features, it will also over-amplify the noise in the image, producing obvious artifacts, interfering with subsequent garbage recognition, and reducing the recognition accuracy. If the clipLimit parameter is too small, the effect on improving the image contrast is limited. When facing uneven lighting or complex road surface textures, it cannot effectively enhance the contrast between garbage and the road surface, resulting in indistinct garbage features and easy missed detection. Therefore, how to dynamically configure the clipLimit parameter for each sub-block based on the local statistical characteristics of each sub-block to improve the garbage recognition accuracy has become an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for identifying road debris from unmanned street sweepers based on image detection. The specific technical solution adopted is as follows: One embodiment of the present invention provides a method for identifying road litter in an unmanned sweeper based on image detection, comprising the following steps: Get each current sub-block on the current frame road image; Based on the grayscale standard deviation and grayscale mean of each current sub-block, the basic clipLimit parameter adjustment factor of each current sub-block is obtained; based on the HSV space histogram, grayscale co-occurrence matrix eigenvalues ​​and connected component count of each current sub-block, and the garbage identification results of sub-blocks in the previous frame road image of the current frame road image that are at the same position as each current sub-block, the garbage separability adjustment factor of each current sub-block is obtained; based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block, the environmental interference adjustment factor of each current sub-block is obtained, where the grayscale value of shadow pixels is less than a preset shadow threshold; based on the basic clipLimit parameter adjustment factor, the garbage separability adjustment factor, and the environmental interference adjustment factor, the target clipLimit parameter of each current sub-block is obtained; Based on the target clipLimit parameter of each current sub-block, the CLAHE algorithm is used to enhance the road image of the current frame to obtain an enhanced image, and the unmanned sweeper is used to identify road garbage based on the enhanced image.

[0005] Beneficial effects: This invention first acquires each current sub-block on the current frame road image; then, based on the grayscale standard deviation and grayscale mean of each current sub-block, it obtains the basic clipLimit parameter adjustment factor for each current sub-block; based on the HSV space histogram, grayscale co-occurrence matrix eigenvalues, and number of connected components of each current sub-block, and the garbage identification results of sub-blocks in the previous frame road image that are at the same position as each current sub-block, it obtains the garbage separability adjustment factor for each current sub-block; based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block, it obtains the environmental interference adjustment factor for each current sub-block; based on the basic clipLimit parameter adjustment factor, the garbage separability adjustment factor, and the environmental interference adjustment factor, it obtains the target clipLimit parameter for each current sub-block; finally, based on the target clipLimit parameter of each current sub-block, it uses the CLAHE algorithm to enhance the current frame road image to obtain an enhanced image, and performs road garbage identification by an unmanned sweeper based on the enhanced image. Furthermore, this invention dynamically configures the clipLimit parameters for each sub-block based on the basic clipLimit parameter adjustment factor, the waste separability adjustment factor, and the environmental interference adjustment factor, and performs image enhancement based on the reconfigured clipLimit parameters, which can improve the model's ability to identify waste, thereby improving the accuracy and reliability of road waste identification by unmanned sweepers. Attached Figure Description

[0006] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0007] Figure 1 This is a flowchart of a method for identifying road debris using an unmanned sweeper based on image detection, according to the present invention. Detailed Implementation

[0008] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the protection scope of the embodiments of the present invention.

[0009] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art.

[0010] This embodiment provides a method for identifying road debris from an unmanned street sweeper based on image detection, detailed as follows: like Figure 1 As shown, the method for identifying road litter using an unmanned sweeper includes the following steps: Step S001: Obtain each current sub-block on the current frame road image.

[0011] The main purpose of this embodiment is to dynamically configure the clipLimit parameter for each sub-block based on the characteristics of each sub-block in the road image collected by the unmanned sweeper. Based on the reconfigured clipLimit parameter, the CLAHE algorithm is used for image enhancement. The enhanced image is then input into the garbage recognition model to recognize garbage, thereby improving the accuracy and reliability of garbage recognition on the road by the unmanned sweeper.

[0012] This embodiment first acquires any unmanned sweeping vehicle performing a sweeping operation, and obtains the images captured in real time by the vehicle's onboard camera during the sweeping operation. These images are recorded as road images captured by the unmanned sweeping vehicle. The camera on the unmanned sweeping vehicle is a high-definition camera that can cover the road area in front of and on both sides of the vehicle. During the sweeping vehicle's movement, the camera continuously captures road images in real time and transmits the captured image data to the onboard processing system. In addition, to ensure image quality, an image stabilization device can be equipped to reduce image shaking caused by vehicle bumps, providing a clear, complete, and stable image foundation for subsequent accurate road debris identification.

[0013] Since the unmanned sweeper uses a continuous data acquisition mechanism when performing sweeping operations, and the image enhancement process of each frame of image acquired by the unmanned sweeper during the sweeping operation is consistent in this embodiment, for ease of description and understanding, this embodiment will subsequently describe the enhancement process of the road image acquired by the unmanned sweeper at the current operation moment as an example, and the road image acquired by the unmanned sweeper at the current operation moment will be recorded as the current frame road image. That is, this embodiment will subsequently describe the enhancement process of the current frame road image and the garbage recognition process as an example.

[0014] Before applying the CLAHE algorithm to enhance the current frame road image, it is generally necessary to first divide the current frame road image. Therefore, in this embodiment, the current frame road image is first divided into non-overlapping rectangular blocks, and each sub-block obtained from the division is recorded as the current sub-block on the current frame road image. Furthermore, this embodiment divides the current frame road image according to empirical dimensions, and requires that the length and width of the road image be divisible by the empirical dimensions. If the length and width of the road image are both divisible by 8, then this embodiment can choose to divide the current frame road image into 8×8 blocks. If the current frame road image is divided into 8×8 blocks, then the number of sub-blocks obtained is... Where L1 is the length of the current frame road image and C1 is the width of the current frame road image; and the rule for dividing any road image in this embodiment is consistent with that of the current frame road image.

[0015] Therefore, this embodiment obtains each current sub-block on the current frame road image through the above process.

[0016] Step S002: Based on the grayscale standard deviation and grayscale mean of each current sub-block, obtain the basic clipLimit parameter adjustment factor for each current sub-block; based on the HSV spatial histogram, grayscale co-occurrence matrix eigenvalues, and number of connected components of each current sub-block, and the garbage identification results of sub-blocks in the previous frame road image of the current frame road image that are at the same position as each current sub-block, obtain the garbage separability adjustment factor for each current sub-block; based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block, obtain the environmental interference adjustment factor for each current sub-block, where the grayscale value of shadow pixels is less than a preset shadow threshold; based on the basic clipLimit parameter adjustment factor, the garbage separability adjustment factor, and the environmental interference adjustment factor, obtain the target clipLimit parameter for each current sub-block.

[0017] In this embodiment, after dividing the current frame road image into segments, feature analysis is performed on each current sub-block. Based on the analysis results, a clipLimit parameter, also known as the target clipLimit parameter, is configured for each sub-block. Specifically, this embodiment will next perform feature analysis on each current sub-block to obtain the target clipLimit parameter for each current sub-block. The specific process is as follows: In road litter recognition, image contrast and brightness are key indicators reflecting the overall characteristics of an image. Contrast reflects the degree of difference between the litter and the road surface. Locally low contrast often indicates severe shadows, requiring stronger enhancement to improve detail visibility. Locally high contrast requires conservative enhancement to avoid noise amplification. Locally low brightness indicates low litter visibility, requiring stronger enhancement to improve litter visibility. Locally overly bright areas require suppressed enhancement to avoid overexposure. This embodiment then obtains the basic clipLimit parameter adjustment factor for each current sub-block based on its contrast and brightness. The factor reflects the degree of adjustment of the final clipLimit parameter of the corresponding sub-block based on the basic clipLimit parameter. Since the grayscale standard deviation of a sub-block characterizes its contrast and the grayscale mean characterizes its brightness, this embodiment will subsequently obtain the basic clipLimit parameter adjustment factor for each current sub-block based on its grayscale standard deviation and grayscale mean. For ease of understanding, this embodiment will subsequently describe the specific process of obtaining the basic clipLimit parameter adjustment factor for the a-th current sub-block in the current frame road image as an example. The specific process of obtaining the basic clipLimit parameter adjustment factor for the a-th current sub-block is as follows: The current frame road image is converted to grayscale to obtain the grayscale value of each pixel in the current frame road image. The standard deviation of the grayscale values ​​of all pixels in the current frame road image is calculated and recorded as the overall contrast. The mean of the grayscale values ​​of all pixels in the current frame road image is calculated and recorded as the overall brightness. The standard deviation of the grayscale values ​​of all pixels in the a-th current sub-block is calculated and recorded as the contrast of the a-th current sub-block. The mean of the grayscale values ​​of all pixels in the a-th current sub-block is calculated and recorded as the brightness of the a-th current sub-block. The ratio of the contrast of the a-th current sub-block to the overall contrast is recorded as the relative contrast of the a-th current sub-block. The ratio of the brightness of the a-th current sub-block to the overall brightness is recorded as the relative brightness of the a-th current sub-block. The constant 1 is subtracted from the relative contrast of the a-th current sub-block and recorded as the contrast quantization value of the a-th current sub-block. The constant 1 is subtracted from the relative brightness of the a-th current sub-block and recorded as the brightness quantization value of the a-th current sub-block. This is the contrast quantization value for the a-th current sub-block. This represents the luminance quantization value of the current sub-block (a-th). The contrast of the current sub-block a is given by the given information. For overall contrast, The brightness of the current sub-block a is... The overall brightness; when the sum of the contrast quantization value and the brightness quantization value of the a-th current sub-block is less than 0, that is... When the value is less than 0, it indicates that the contrast or brightness of the a-th current sub-block is higher than the overall contrast and brightness. Therefore, the a-th current sub-block is more likely to be a region with excessively high local contrast or brightness. In this case, to suppress noise and avoid overexposure, conservative enhancement is required; that is, no further enhancement is needed, and the basic clipLimit parameter can be used directly. When the sum of the contrast quantization value and the luminance quantization value of the a-th current sub-block is less than 0, this embodiment directly uses 0 as the adjustment factor for the basic clipLimit parameter of the a-th current sub-block; when the sum of the contrast quantization value and the luminance quantization value of the a-th current sub-block is not less than 0, that is... A value greater than or equal to 0 and the larger it is, the more the contrast or brightness of the current sub-block (a-th sub-block) is lower than the overall contrast and brightness. Therefore, the current sub-block (a-th sub-block) is more likely to be a region with excessively low local contrast and brightness. In this case, the clipLimit parameter needs to be increased more significantly to improve detail and visibility of junk. When the sum of the contrast quantization value and the luminance quantization value of the a-th current sub-block is not less than 0, this embodiment normalizes the result of the sum of the contrast quantization value and the luminance quantization value of the a-th current sub-block, that is... As the adjustment factor for the base clipLimit parameter of the a-th current sub-block; the adjustment factor for the base clipLimit parameter of the a-th current sub-block is used This indicates that Norm() is the normalization function. and These represent the contrast and brightness of the current sub-block relative to the overall image, respectively. A larger value results in a stronger enhancement for the a-th current sub-block, meaning the clipLimit parameter for the a-th current sub-block should be larger for subsequent enhancements. The smaller the value, the smaller the enhancement intensity of the current sub-block a, which means the smaller the clipLimit parameter of the current sub-block a during subsequent enhancements; the larger the clipLimit parameter, the stronger the local contrast enhancement intensity of CLAHE.

[0018] Therefore, this embodiment can obtain the basic clipLimit parameter adjustment factor for each current sub-block through the above process. Although the basic clipLimit parameter adjustment factor can combine the overall characteristics, different regions still have significant differences in many aspects. Contrast and brightness alone cannot meet the requirements of refined enhancement. For example, in different sub-blocks, garbage and road surface may differ in color, texture, edge density, number of connected components, and historical detection stability. These features can reflect the separability of garbage in different sub-blocks, and the strength of garbage separability can reflect the difficulty of garbage being identified before the corresponding sub-block is enhanced. The environmental interference of different sub-blocks will also affect the identification of garbage. For example, the uniformity of illumination, the degree of image blur, and the shadow coverage of the sub-block will all affect the identification of garbage. That is, the environmental interference of different sub-blocks can reflect the difficulty of garbage identification in the corresponding sub-block. The difficulty of identifying garbage before block enhancement is considered. Therefore, to ensure the accuracy and stability of subsequent garbage identification, this embodiment cannot determine the final clipLimit parameter of the sub-block solely based on the basic clipLimit parameter adjustment factor obtained above. It is also necessary to analyze the garbage separability and environmental interference of the current sub-block. Finally, the final clipLimit parameter of the sub-block is determined by combining the analysis results of the garbage separability and environmental interference of the current sub-block with the basic clipLimit parameter adjustment factor. That is, this embodiment further needs to analyze and obtain the garbage separability adjustment factor and environmental interference adjustment factor for each current sub-block. The garbage separability adjustment factor and environmental interference adjustment factor are also key parameters for subsequently determining the target clipLimit parameter of each current sub-block. The specific acquisition process is as follows: Trash differs from the background in color, texture, edge density, number of connected components, and stability in historical detection environments. These differences reflect the degree of separability between trash and the background. When separability is low, stronger enhancement is needed to highlight trash details and improve subsequent trash recognition. Specifically, in road scenes, when there are many colors within a sub-block but the differences between colors are insignificant, it indicates that the colors of trash and road surface are likely similar, which can lead to insufficient contrast in the trash area. In this case, it is necessary to increase the enhancement intensity to improve the recognizability of trash. Road surface (such as asphalt) has regular, periodic textures (such as the uniform arrangement of asphalt particles), a continuous overall structure, and blurred edges, while trash... Items like plastic bottles and paper scraps have irregular, non-periodic textures and usually present clear outlines in images. Therefore, the worse the texture regularity and the clearer the edges of a sub-block, the higher the probability of it containing litter. In this case, stronger enhancement should be applied to highlight its features and improve the accuracy of subsequent detection and recognition. Small pieces of litter on the road (such as cigarette butts and bottle caps) occupy a small proportion in the image and are easily overlooked during recognition. Therefore, the more connected components in a sub-block, the higher the probability that the corresponding sub-block contains small pieces of litter, and the stronger the image enhancement should be applied to highlight its features and improve the accuracy of subsequent detection and recognition. Unmanned sweepers generally move relatively slowly. If several consecutive frames detect the same area... If trash is detected, it indicates a stable environment suitable for identification. In this case, the enhancement intensity needs to be reduced to prevent over-enhancement. That is, when an unmanned sweeper is moving, if trash is present at a certain location in front of it and is detected in multiple consecutive frames, it indicates a stable environment and strong trash detection capability. The stronger the trash separability in the current environment, the more necessary it is to reduce the enhancement intensity to prevent over-enhancement. Based on this, this embodiment will next measure the trash separability of the current sub-block based on its environmental stability, color characteristics, texture characteristics, and the number of connected components. Since the trash identification results of sub-blocks in the previous frame of the road image that are at the same position as each current sub-block can reflect environmental stability... Qualitatively, the eigenvalues ​​on the HSV space histogram of the current sub-block reflect color characteristics, and the eigenvalues ​​on the gray-level co-occurrence matrix of the current sub-block reflect texture characteristics. Therefore, this embodiment will next obtain the garbage separability adjustment factor of each current sub-block based on the HSV space histogram, gray-level co-occurrence matrix eigenvalues, number of connected components, and garbage identification results of sub-blocks in the previous frame road image that are at the same position as each current sub-block in the current frame road image. Furthermore, for ease of understanding, this embodiment will subsequently describe the specific process of obtaining the garbage separability adjustment factor of the a-th current sub-block in the current frame road image as an example. The specific process of obtaining the garbage separability adjustment factor of the a-th current sub-block is as follows: First, a preset number of road images preceding the current frame are acquired and recorded as the nearest neighbor road images of the current frame. It is required that the acquired preset number of road images are not only temporally continuous themselves but also temporally continuous with the current frame road image; that is, the nearest neighbor road images of the current frame and the current frame road image belong to the same cleaning operation. In specific applications, the implementer needs to set the preset number value according to actual conditions such as the moving speed of the unmanned sweeper. The faster the unmanned sweeper moves, the smaller the preset number value should be. For example, in this embodiment, it can be set to 3, meaning the nearest neighbor road image of the current frame is the current... The first three frames of the image are used; then, each neighboring road image is divided into non-overlapping sub-blocks. Based on the sub-blocks in the neighboring road images of the current frame that share the same coordinates as the current sub-block, the environmental stability characterization value of the current sub-block is obtained. The division process of the neighboring road images is consistent with the division process of the current frame road images. The specific process for obtaining the environmental stability characterization value of the current sub-block based on the sub-blocks in the neighboring road images of the current frame that share the same coordinates is as follows: All sub-blocks in the neighboring road images that share the same coordinates as the current sub-block are denoted as... The nearest neighbor blocks corresponding to the a-th current sub-block (i.e., the nearest neighbor blocks whose positions in the image are the same as those of the a-th current sub-block) are counted. The number of nearest neighbor blocks identified as garbage is calculated, or the number of nearest neighbor blocks identified as garbage among all the nearest neighbor blocks corresponding to the a-th current sub-block is counted, and this is recorded as the nearest neighbor count value corresponding to the a-th current sub-block. The inverse normalization result of the ratio of the nearest neighbor count value corresponding to the a-th current sub-block to the total number of nearest neighbor blocks corresponding to the a-th current sub-block is recorded as the environmental stability characterization value of the a-th current sub-block. Here, normalization is implemented based on the Sigmoid() function. The `id()` function is a non-linear normalization function that maps real numbers to the interval (0,1). If a neighboring road image is input into the garbage identification model, and garbage is detected in a neighboring sub-block of that neighboring road image, then that neighboring sub-block is considered a garbage-containing neighboring sub-block, and the number of garbage-containing neighboring sub-blocks is the neighbor count value corresponding to the `a`th current sub-block. The garbage identification model is the garbage identification model trained in this embodiment. The smaller the environmental stability representation value of the `a`th current sub-block, the more stable the environment, and thus, identification can be performed. In this case, the enhancement intensity needs to be reduced to a certain extent to prevent over-enhancement.

[0019] Next, the HSV histogram of the current sub-block (HV) is obtained. The HV histogram contains H, S, and V values. H represents the hue of the color, S represents saturation, and V represents brightness. Based on the HSV histogram of the current sub-block, the color representation value of the current sub-block is obtained. Specifically, the color representation value of the current sub-block is obtained by multiplying the standard deviations of all S values ​​and all V values ​​in the HSV histogram of the current sub-block, adding a preset constant, and then performing inverse normalization. This result is then linearly normalized to the entropy of the information of all H values ​​in the HSV histogram of the current sub-block. The product of the normalization results is denoted as the color representation value of the a-th current sub-block. The reverse normalization here is to first take the reciprocal and then perform linear normalization. The smaller the standard deviation of the S value and the standard deviation of the V value, the less obvious the color differences in the a-th current sub-block. The larger the entropy value of the H value, the more color types there are in the a-th current sub-block. When the standard deviation of the S value and the standard deviation of the V value are smaller and the entropy value of the H value is larger, that is, when the color representation value of the a-th current sub-block is larger, it indicates that there are more color types in the a-th current sub-block and the color differences are less obvious. In this case, it is necessary to increase the enhancement intensity to improve the identifiability of the garbage.

[0020] Next, based on the eigenvalues ​​of the gray-level co-occurrence matrix of the a-th current sub-block, the texture representation value of the a-th current sub-block is obtained. The process of obtaining the eigenvalues ​​of the gray-level co-occurrence matrix is ​​a known technique. The specific process of obtaining the texture representation value of the a-th current sub-block is as follows: the product of the energy and correlation of the gray-level co-occurrence matrix of the a-th current sub-block is recorded as the fusion eigenvalue of the a-th current sub-block. The product of the linear normalization result of the ratio of the entropy value of the gray-level co-occurrence matrix of the a-th current sub-block to the fusion eigenvalue of the a-th current sub-block plus a preset constant and the linear normalization result of the contrast of the gray-level co-occurrence matrix of the a-th current sub-block is recorded as the texture representation value of the a-th current sub-block. The higher the contrast of the gray-level co-occurrence matrix of the a-th current sub-block, the clearer the texture of the corresponding sub-block, the more likely there are garbage edges, and the more the intensity needs to be enhanced to improve the recognizability of garbage. When the energy and correlation of the gray-level co-occurrence matrix of the a-th current sub-block are smaller and the entropy value is larger, it is more in line with the irregular texture characteristics of the garbage region, or it indicates that the texture regularity of the corresponding sub-block region is worse and the edges are clearer, and it needs to be further enhanced. That is, when the texture representation value of the a-th current sub-block is larger, it is more in line with the irregular texture characteristics of the garbage region, and at this time, the enhancement intensity needs to be increased to improve the recognizability of garbage.

[0021] Then, continue to perform connected component analysis on the a-th current sub-block to obtain the connected components in the a-th current sub-block, and count the number of connected components in the a-th current sub-block. The normalized result of the number of connected components in the a-th current sub-block is recorded as the connected component count representation value of the a-th current sub-block. The more connected components in the a-th current sub-block or the larger the connected component count representation value, the greater the possibility of small-sized garbage in the a-th current sub-block. In this case, it is necessary to increase the enhancement strength to improve the garbage's identifiability.

[0022] Finally, the sum of the connected component quantity, texture, and color representation values ​​of the a-th current sub-block is calculated and denoted as the comprehensive representation value of the a-th current sub-block. The linearly normalized result of multiplying the comprehensive representation value of the a-th current sub-block by its environmental stability representation value is denoted as the garbage separability adjustment factor of the a-th current sub-block. The expression for the garbage separability adjustment factor of the a-th current sub-block is: , , in, is the garbage separability adjustment factor for the a-th current sub-block, and Norm() is the linear normalization function; This is the ratio of the number of nearest neighbors corresponding to the current a-th sub-block to the total number of nearest neighbor sub-blocks corresponding to the current a-th sub-block. To use a variation of the Sigmoid() function to The expression to be normalized Let be the environmental stability characterization value of the a-th current sub-block. The coefficient 10 in this formula controls the steepness of the curve, ensuring the enhancement intensity is within a certain range. It can quickly adjust when deviating from 0.5, achieving a sensitive response. In the formula, 0.5 is the offset, and represents the inflection point, making... When the value is less than 0.5 (unstable), the enhancement intensity approaches 1; when it is not less than 0.5, a symmetrical adjustment is formed, and e is the natural constant. The smaller the value, the more stable the environment is, indicating that identification is possible. In this case, the enhancement intensity needs to be reduced to a certain extent to prevent over-enhancement. The larger the value, the more unstable the environment. In this case, it is necessary to increase the reinforcement strength to improve the identifiability of the waste. The larger, The smaller; This is the comprehensive representation value of the current sub-block (a-th). Let H be the information entropy of all H values ​​in the HSV space histogram of the current sub-block a. Let S be the standard deviation of all S values ​​in the HSV space histogram of the current sub-block a. The standard deviation of all V values ​​in the HSV space histogram of the current sub-block; The color representation value of the a-th current sub-block is such that the larger the value, the more color types there are in the a-th current sub-block and the less obvious the differences between the colors. In this case, it is necessary to increase the enhancement intensity to improve the identifiability of the garbage. Let be the contrast of the gray-level co-occurrence matrix of the current sub-block (a-th sub-block). Let be the entropy value of the gray-level co-occurrence matrix of the current sub-block (a-th sub-block). Let be the correlation of the gray-level co-occurrence matrix of the current sub-block . Let be the energy of the gray-level co-occurrence matrix of the current sub-block (a-th). This is the texture representation value of the a-th current sub-block. The larger the value, the more the corresponding sub-block conforms to the texture characteristics of the garbage area, and the more the intensity needs to be enhanced to improve the recognizability of the garbage. The value represents the number of connected components in the current a-th sub-block. The larger the value, the greater the possibility that there is small-sized garbage in the current a-th sub-block. In this case, the enhancement strength needs to be increased to improve the identifiability of the garbage. In the formula, c0 is a preset constant, which is taken as 0.01 in this embodiment. Its purpose is to prevent the denominator from being 0. The purpose and function of linear normalization in the formula is to eliminate differences in magnitude and dimensions. The bigger, The bigger, The larger and When it is larger, The larger, therefore A larger value indicates weaker separability or identifiability of the waste within that sub-block, requiring increased reinforcement strength to improve its identifiability. When the value is larger, the clipLimit parameter of the a-th current sub-block should be larger, and vice versa. The smaller the value, the smaller the clipLimit parameter of the current sub-block a should be. Additionally, if the current frame road image is the first frame image collected during the cleaning operation performed by the unmanned sweeper at that moment—that is, if the current frame road image has no neighboring road images or no previous frame image—then the environmental stability characterization value is not considered when calculating the waste separability adjustment factor for the current sub-block; in other words, the environmental stability characterization value is not included in the calculation of the waste separability adjustment factor for the current sub-block.

[0023] Furthermore, the degree of environmental interference affecting different sub-blocks can reflect the difficulty of identifying trash before enhancement. For example, the uniformity of illumination, image blurriness, and shadow coverage of a sub-block can reflect the difficulty of identifying trash before enhancement. Specifically, road scenes often have uneven illumination (tree shadows, building shadows, vehicle headlights). The more uneven the illumination of a sub-block, the greater the environmental interference, and the more necessary it is to reduce the enhancement intensity to avoid further amplifying noise or preventing further interference with subsequent trash identification, thereby improving the ability to identify trash. Camera defocus, unmanned sweeping vehicle movement, or atmospheric conditions (such as fog or rain) can easily cause image blurriness. Blurriness reduces image clarity, which in turn affects the edge detection and texture features of trash. Therefore, further enhancement is needed to restore or strengthen these features. To improve the reliability of subsequent garbage identification; when garbage falls into the shadow area, the image contrast is extremely low due to insufficient lighting, making it difficult to distinguish the garbage from the background and easily missed by the detection system. Therefore, the larger the proportion of shadow coverage in the sub-block, the more necessary it is to increase the intensity of image enhancement to improve the visibility of details in the dark areas and ensure that the garbage target is not missed; the number of peak points on the gray-level histogram of the sub-block and the gray-level fluctuation range in the sub-block can reflect the uniformity of lighting in the sub-block. The more peak points and the larger the gray-level fluctuation range in the sub-block, the more uneven the lighting; the gradient mean of the edge pixels in the sub-block can reflect the blur of the sub-block. The smaller the gradient mean, the greater the image blur of the sub-block may be; the more pixels in the sub-block with gray values ​​less than the preset shadow threshold, the larger the proportion of shadow coverage in the sub-block. Based on the above analysis, this embodiment will next obtain the environmental interference adjustment factor for each current sub-block based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block. The grayscale value of the shadow pixels is less than the preset shadow threshold. Furthermore, for ease of understanding, this embodiment will subsequently describe the specific process of obtaining the environmental interference adjustment factor for the a-th current sub-block in the current frame road image as an example. The specific process of obtaining the environmental interference adjustment factor for the a-th current sub-block is as follows: First, the Canny edge detection algorithm is used to obtain all edge pixels on the current frame road image, and the gradient magnitude of each edge pixel in the a-th current sub-block is calculated. The average gradient magnitude of all edge pixels in the a-th current sub-block is added to the preset constant and then reverse normalized. This result is denoted as the blur characterization value of the a-th current sub-block. Here, the reverse normalization is to first take the reciprocal and then perform linear normalization. The smaller the average gradient magnitude of all edge pixels in the a-th current sub-block or the larger the blur characterization value of the a-th current sub-block, the greater the blur of the image in that sub-block, and the more the enhancement intensity needs to be increased. Then, the grayscale histogram of the a-th current sub-block is obtained. Based on the grayscale histogram of the a-th current sub-block, the illumination uniformity characterization value of the a-th current sub-block is obtained. The specific process for obtaining the illumination uniformity characterization value of the a-th current sub-block is as follows: after smoothing the grayscale histogram of the a-th current sub-block, the number of peak points on the smoothed grayscale histogram is counted and recorded as the peak point count value. The grayscale range in the a-th current sub-block is calculated, and the grayscale range in the a-th current sub-block is multiplied by the peak point count value and then inversely normalized by adding a preset constant. This result is recorded as the illumination uniformity characterization value of the a-th current sub-block. Here, the inverse normalization is also to first take the reciprocal and then perform linear normalization. In this embodiment, Gaussian filtering or moving average can be used for smoothing. The grayscale range in the a-th current sub-block is... The result is the maximum gray value minus the minimum gray value in the sub-block. The more peak points and the larger the gray value range on the smoothed gray-level histogram, the smaller the illumination uniformity characterization value of the a-th current sub-block. This indicates that the gray-level fluctuation of the sub-block is stronger, the gray-level range is wider, and it is more in line with the characteristics of uneven illumination. Correspondingly, the interference it receives is greater, and the enhancement effect needs to be weakened. Then, the linear normalized result of the proportion of shadow pixels in the a-th current sub-block is obtained and recorded as the shadow characterization value of the a-th current sub-block. The proportion of shadow pixels in the a-th current sub-block is the ratio of the number of shadow pixels in the a-th current sub-block to the total number of pixels in the a-th current sub-block. The larger the shadow characterization value of the a-th current sub-block, the greater the shadow coverage in the a-th current sub-block, and the greater the need to increase its enhancement intensity. Finally, the linearly normalized result of adding the illumination uniformity value and the shadow value of the a-th current sub-block, and then multiplying it by the fuzziness value of the a-th current sub-block, is denoted as the environmental disturbance adjustment factor of the a-th current sub-block. The expression for the environmental disturbance adjustment factor of the a-th current sub-block is: , in, This is the environmental interference adjustment factor for the a-th current sub-block. Let be the average gradient magnitude of all edge pixels in the current sub-block (a-th sub-block). Let be the number of peak points in the current sub-block (a-th sub-block). The grayscale range in the current sub-block a is the grayscale value range. Let a be the number of shadow pixels in the current sub-block. The total number of pixels in the current sub-block (a); The smaller, The larger the value, the greater the blurriness of the image in that sub-block, and the more the enhancement intensity needs to be increased; The greater the sum The larger the value, the stronger the grayscale fluctuation of the sub-block and the wider the grayscale range it contains, which better matches the characteristics of uneven illumination. This corresponds to greater interference and a greater need to reduce its enhancement effect. and The smaller the product or The larger the value, the greater its reinforcing strength; The larger the value, the greater the shadow coverage in the current sub-block a, and the more it needs to be enhanced. The bigger, The larger and When it is larger, The larger, therefore A larger value indicates a stronger enhancement for the a-th current sub-block, and a larger clipLimit parameter for the a-th current sub-block; conversely, a smaller value indicates a weaker clipLimit parameter. The smaller the value, the smaller the clipLimit parameter of the a-th current sub-block; c0 in the formula is a preset constant, its purpose being to prevent the denominator from being 0. The purpose and function of linear normalization in the formula is also to eliminate differences in magnitude and dimension.

[0024] In practical applications, implementers can set a preset shadow threshold according to the actual situation. The preset shadow threshold acquisition method in this embodiment is as follows: First, obtain the maximum gray value of the current frame road image. In this embodiment, one-quarter of it is selected as a candidate value. Determine whether the candidate value is greater than the gray value with the largest proportion in the current frame road image. If it is not greater, select the candidate value as the preset shadow threshold. If it is greater, use half of the candidate value as the preset shadow threshold to prevent the road surface area from being identified as a shadow. The gray value with the largest proportion in the image is the road surface area.

[0025] Therefore, this embodiment can obtain the basic clipLimit parameter adjustment factor, waste separability adjustment factor, and environmental interference adjustment factor for each current sub-block through the above process. After obtaining the basic clipLimit parameter adjustment factor, waste separability adjustment factor, and environmental interference adjustment factor, this embodiment obtains the target clipLimit parameter for each current sub-block based on the basic clipLimit parameter adjustment factor, waste separability adjustment factor, and environmental interference adjustment factor for each current sub-block. The specific process of obtaining the target clipLimit parameter for each current sub-block based on the basic clipLimit parameter adjustment factor, waste separability adjustment factor, and environmental interference adjustment factor for each current sub-block is as follows: For the current sub-block a: The normalized result of the geometric mean of the waste separability adjustment factor and the environmental disturbance adjustment factor of the current sub-block a is multiplied by the preset adjustment limit value and the basic clipLimit parameter adjustment factor of the current sub-block a, and this result is denoted as the parameter adjustment value of the current sub-block a. The geometric mean is also the square root of the product of the waste separability adjustment factor and the environmental disturbance adjustment factor of the current sub-block a. The purpose of taking the square root is to avoid the excessive dominance of a single feature. The sum of the basic clipLimit parameter and the parameter adjustment value of the current sub-block a is denoted as the target clipLimit parameter of the current sub-block a. The expression for the target clipLimit parameter of the current sub-block a is: , in, The target clipLimit parameter for the a-th current sub-block. The base clipLimit parameter, CL is the preset adjustment limit value. This is the adjustment factor for the base clipLimit parameter of the a-th current sub-block. This is the garbage separability adjustment factor for the a-th current sub-block. This is the environmental interference adjustment factor for the a-th current sub-block; The bigger, The larger and When it is larger, The larger, A larger value indicates a larger target clipLimit parameter for the a-th current sub-block, and a greater degree of enhancement for the a-th current sub-block, and vice versa. A smaller value indicates a smaller target clipLimit parameter for the a-th current sub-block, resulting in a greater degree of enhancement. CL represents the minimum clipLimit parameter and the maximum adjustable value of the parameter. In specific applications, implementers need to set the values ​​of CL0 and CL according to the actual situation. Since the image in this embodiment is an outdoor image, it is necessary to highlight the details and sense of layering of the image to a certain extent. Therefore, in this embodiment, the value of CL0 is set to 3.0 and the value of CL is set to 2.0, that is, the target clipLimit parameter is 3.0 to 5.0. A higher clipLimit can enhance local contrast and make road debris and object textures clearer.

[0026] Therefore, this embodiment can obtain the target clipLimit parameter of each current sub-block through the above process; the linear normalization in this embodiment is global min-max normalization.

[0027] Step S003: Based on the target clipLimit parameter of each current sub-block, the CLAHE algorithm is used to enhance the road image of the current frame to obtain an enhanced image, and the unmanned sweeper is used to identify road garbage based on the enhanced image.

[0028] In this embodiment, after obtaining the target clipLimit parameters for each current sub-block, the CLAHE algorithm is used to enhance the current frame road image, resulting in an enhanced image. Since the clipLimit parameters for each current sub-block in the current frame road image are known, the process of enhancing the current frame road image using the CLAHE algorithm is well-known; that is, this embodiment only changes the process of obtaining the clipLimit parameters, without altering other processes during enhancement. After obtaining the enhanced image, it is input into the garbage identification model, and the model outputs the garbage identification result for the enhanced image. The waste identification model in this embodiment is a convolutional neural network, such as YOLOv8 or Faster R-CNN. This embodiment uses a convolutional neural network to identify waste. The network training data covers common waste types such as plastic bottles, paper scraps, and fallen leaves, and data augmentation is used to improve the model's generalization ability. The trained lightweight model is deployed on an edge computing device, and combined with sensor fusion technology (such as cameras and radar), dynamic interference factors (such as pedestrians and vehicles) are filtered out, thereby ensuring that the unmanned sweeper's intelligent waste identification system has a low response latency. Finally, the system outputs the location coordinates and category information of the waste to guide the sweeper in planning the optimal sweeping path, thereby improving the automation level of sanitation operations. The training process of the waste identification model is publicly known.

[0029] Thus, this embodiment completes the identification of road litter by unmanned sweepers. Furthermore, this embodiment dynamically configures the clipLimit parameter for each sub-block based on the characteristics of each sub-block on the road image, and performs image enhancement based on the reconfigured clipLimit parameter, which can improve the model's ability to identify litter, thereby improving the accuracy and reliability of road litter identification by unmanned sweepers.

[0030] In summary, this embodiment first obtains each current sub-block on the current frame road image; then, based on the grayscale standard deviation and grayscale mean of each current sub-block, it obtains the basic clipLimit parameter adjustment factor for each current sub-block; based on the HSV space histogram, gray-level co-occurrence matrix eigenvalues, and number of connected components of each current sub-block, and the garbage identification results of sub-blocks in the previous frame road image that are at the same position as each current sub-block, it obtains the garbage separability adjustment factor for each current sub-block; based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block, it obtains the environmental interference adjustment factor for each current sub-block; based on the basic clipLimit parameter adjustment factor, the garbage separability adjustment factor, and the environmental interference adjustment factor, it obtains the target clipLimit parameter for each current sub-block; finally, based on the target clipLimit parameter of each current sub-block, it uses the CLAHE algorithm to enhance the current frame road image to obtain an enhanced image, and performs road garbage identification by the unmanned sweeper based on the enhanced image. Furthermore, this embodiment dynamically configures the clipLimit parameters for each sub-block based on the basic clipLimit parameter adjustment factor, the waste separability adjustment factor, and the environmental interference adjustment factor, and performs image enhancement based on the reconfigured clipLimit parameters, which can improve the model's ability to identify waste, thereby improving the accuracy and reliability of road waste identification by unmanned sweepers.

[0031] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for identifying road debris from an unmanned street sweeper based on image detection, characterized in that, The method includes the following steps: Get each current sub-block on the current frame road image; Based on the grayscale standard deviation and grayscale mean of each current sub-block, the basic clipLimit parameter adjustment factor of each current sub-block is obtained; Based on the HSV space histogram, gray-level co-occurrence matrix eigenvalues ​​and number of connected components of each current sub-block, and the garbage identification results of sub-blocks with the same position as each current sub-block on the previous frame road image of the current frame road image, the garbage separability adjustment factor of each current sub-block is obtained. Based on the grayscale histogram of each current sub-block and the gradient of edge pixels and the proportion of shadow pixels in the corresponding sub-block, the environmental interference adjustment factor of each current sub-block is obtained, and the grayscale value of the shadow pixels is less than the preset shadow threshold; based on the basic clipLimit parameter adjustment factor, the garbage separability adjustment factor and the environmental interference adjustment factor, the target clipLimit parameter of each current sub-block is obtained. Based on the target clipLimit parameter of each current sub-block, the CLAHE algorithm is used to enhance the road image of the current frame to obtain an enhanced image, and the unmanned sweeper is used to identify road garbage based on the enhanced image.

2. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 1, characterized in that, The current sub-block is obtained by dividing the current frame road image into non-overlapping segments.

3. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 1, characterized in that, Methods for obtaining the adjustment factor of the basic clipLimit parameter include: The ratio of the grayscale standard deviation of each current sub-block to the overall contrast is denoted as the relative contrast of the corresponding current sub-block, and the ratio of the brightness of each current sub-block to the overall brightness is denoted as the relative brightness of the corresponding current sub-block. The overall contrast and overall brightness are the grayscale standard deviation and grayscale mean of the current frame road image, respectively. The result of subtracting the relative contrast of each current sub-block from the constant 1 is denoted as the contrast quantization value of the corresponding current sub-block. The result of subtracting the relative brightness of each current sub-block from the constant 1 is denoted as the brightness quantization value of the corresponding current sub-block. If the sum of the contrast quantization value and the brightness quantization value of each current sub-block is less than 0, then 0 is used as the adjustment factor of the basic clipLimit parameter of the corresponding current sub-block. If the sum of the contrast quantization value and the brightness quantization value of each current sub-block is not less than 0, then the normalized result of adding the contrast quantization value and the brightness quantization value of each current sub-block is denoted as the adjustment factor of the basic clipLimit parameter of the corresponding current sub-block.

4. The method for identifying road litter from unmanned sweeping vehicles based on image detection as described in claim 1, characterized in that, The method for obtaining the garbage separability adjustment factor for any current sub-block includes: The road images of the previous preset number of frames of the current frame are all recorded as the neighboring road images of the current frame; each neighboring road image is divided into non-overlapping sub-blocks on each neighboring road image, and the environmental stability characterization value of the current sub-block is obtained based on the sub-blocks on the neighboring road images that have the same coordinate position as the current sub-block. The color representation value of the current sub-block is obtained based on the HSV space histogram of the current sub-block; The texture representation value of the current sub-block is obtained based on the feature value of the gray-level co-occurrence matrix of the current sub-block; Obtain the number of connected components in the current sub-block, and record the normalized result of the number of connected components as the connected component count representation value of the current sub-block; The normalized result of summing the connected component quantity, texture, and color representation values ​​of the current sub-block and multiplying them by the environmental stability representation value is denoted as the garbage separability adjustment factor of the current sub-block.

5. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 4, characterized in that, The method for obtaining the environmental stability characterization value of the current sub-block includes: All sub-blocks on the neighboring road image that have the same coordinate position as the current sub-block are recorded as the neighboring sub-blocks corresponding to the current sub-block. The number of neighboring sub-blocks where garbage is identified is counted and recorded as the neighboring quantity value corresponding to the current sub-block. The reverse normalized result of the ratio of the neighboring quantity value corresponding to the current sub-block to the total number of neighboring sub-blocks corresponding to the current sub-block is recorded as the environmental stability characterization value of the current sub-block.

6. The method for identifying road litter in an unmanned sweeper based on image detection as described in claim 4, characterized in that, The method for obtaining the texture representation value of the current sub-block includes: The product of the energy and correlation of the gray-level co-occurrence matrix of the current sub-block is recorded as the fusion feature value of the corresponding current sub-block. The product of the normalized result of the ratio of the entropy value of the gray-level co-occurrence matrix of the current sub-block to the fusion feature value plus a preset constant, and the normalized result of the contrast of the gray-level co-occurrence matrix of the current sub-block, is recorded as the texture representation value of the current sub-block.

7. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 4, characterized in that, The method for obtaining the color representation value of the current sub-block includes: The color representation value of the current sub-block is the product of the inverse normalization result obtained by multiplying the standard deviation of the S value and the standard deviation of the V value of the HSV space histogram of the current sub-block and adding a preset constant, and then multiplying the product of the normalized result of the information entropy of all H values ​​in the HSV space histogram of the current sub-block.

8. The method for identifying road litter in an unmanned sweeper based on image detection as described in claim 1, characterized in that, The method for obtaining the environmental interference adjustment factor for any current sub-block includes: The inverse normalization result of adding a preset constant to the average gradient of all edge pixels in the current sub-block is denoted as the fuzziness characterization value of the current sub-block. The illumination uniformity characterization value of the current sub-block is obtained based on the gray-level histogram of the current sub-block; The normalized result of the proportion of shadow pixels in the current sub-block is recorded as the shadow representation value of the current sub-block. The normalized result of adding the illumination uniformity characterization value and the shadow characterization value and then multiplying it by the fuzziness characterization value is denoted as the environmental interference adjustment factor of the current sub-block.

9. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 8, characterized in that, The method for obtaining the illumination uniformity characterization value of the current sub-block includes: After smoothing the grayscale histogram of the current sub-block, the number of peak points is counted and recorded as the peak point count value. The grayscale range of the current sub-block is multiplied by the peak point count value and then the result of inverse normalization after adding a preset constant is recorded as the illumination uniformity characterization value of the current sub-block.

10. The method for identifying road debris from an unmanned sweeper based on image detection as described in claim 1, characterized in that, The methods for obtaining the target clipLimit parameter for any current sub-block include: The normalized result of the geometric mean of the waste separability adjustment factor and the environmental disturbance adjustment factor of the current sub-block is multiplied by the preset adjustment limit value and the basic clipLimit parameter adjustment factor of the current sub-block, and the result is recorded as the parameter adjustment value of the current sub-block. The sum of the basic clipLimit parameter and the parameter adjustment value is recorded as the target clipLimit parameter of the current sub-block.

Images