Road surface cleanliness degree detection method based on deep learning

Abstract

The invention relates to the field of intelligent cleaning, and provides a road surface cleanliness detection method based on deep learning, which adopts an image processing technology based on deep learning, performs feature analysis on a road surface image of a to-be-detected area to obtain road surface garbage position data, and responds to the road surface garbage position data to detect the road surface cleanliness degree of the to-be-detected area. And sending a cleaning instruction to the unmanned cleaning robot. Therefore, the road surface image is automatically analyzed, the garbage position data is generated, and the cleaning instruction is sent to the unmanned cleaning robot according to the garbage position data, so that the garbage cleaning efficiency and accuracy can be remarkably improved, and the resource waste is reduced.

Description

Technical Field

[0001] The present application relates to the field of intelligent cleaning, and more specifically, to a road surface cleanliness detection method based on deep learning. Background Art

[0002] In urban environmental management, there is often a lack of coordination between road cleanliness detection and cleaning operations, which results in garbage not being cleaned up in time, thus affecting the cleanliness of the road. Accurately determining the location of road garbage is crucial for efficient cleaning. In practice, it is difficult to accurately generate garbage location information due to the complexity of the road environment, which results in wasted time and resources searching for the exact location of the garbage during the cleaning process.

[0003] Therefore, an optimized road cleanliness detection solution is needed. Summary of the invention

[0004] In view of the shortcomings of the prior art, this application provides a road cleanliness detection method based on deep learning, which adopts image processing technology based on deep learning, performs feature analysis on the road image of the detection area to obtain road garbage location data, and sends a cleaning instruction to the unmanned cleaning robot in response to the road garbage location data. In this way, by automatically analyzing the road image and generating garbage location data, and then sending a cleaning instruction to the unmanned cleaning robot based on the data, the efficiency and accuracy of garbage cleaning can be significantly improved, and resource waste can be reduced.

[0005] According to one aspect of the present application, a road surface cleanliness detection method based on deep learning is provided, which includes:

[0006] Acquire a road surface image of the area to be detected;

[0007] Performing noise reduction processing on the road surface image of the area to be detected to obtain a road surface detection enhanced image;

[0008] Performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector;

[0009] Based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a cleaning instruction is generated.

[0010] In the above-mentioned road cleanliness detection method based on deep learning, the road surface image of the area to be detected is subjected to denoising processing to obtain a road surface detection enhanced image, including: passing the road surface image of the area to be detected through a road surface detection image denoiser to obtain the road surface detection enhanced image.

[0011] In the above-mentioned road cleanliness detection method based on deep learning, multi-scale feature encoding is performed on the road detection enhanced image to obtain a multi-scale road detection feature vector, including: passing the road detection enhanced image through a first hole road image encoder to obtain a first-scale road detection feature vector; passing the road detection enhanced image through a second hole road image encoder to obtain a second-scale road detection feature vector; and fusing the first-scale road detection feature vector and the second-scale road detection feature vector to obtain the multi-scale road detection feature vector.

[0012] In the above-mentioned road cleanliness detection method based on deep learning, the road detection image denoiser is a denoiser based on an automatic codec, the first hole road image encoder is a convolutional neural network model using a convolution kernel of a first size and a first void ratio, and the second hole road image encoder is a convolutional neural network model using a convolution kernel of a first size and a second void ratio.

[0013] In the above-mentioned road cleanliness detection method based on deep learning, based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a cleaning instruction is generated, including: the multi-scale road surface detection feature vector is modulated and selected based on the chaos management feature distribution to obtain an optimized multi-scale road surface detection feature vector; the optimized multi-scale road surface detection feature vector is passed through a decoder-based garbage location information generator to obtain a decoded value, and the decoded value is the road surface garbage location data of the area to be detected; based on the decoded road surface garbage location data of the area to be detected, the cleaning instruction is generated, and the cleaning instruction is sent to the unmanned cleaning robot.

[0014] In the above-mentioned road cleanliness detection method based on deep learning, the multi-scale road surface detection feature vector is subjected to feature distribution modulation selection based on chaos management to obtain an optimized multi-scale road surface detection feature vector, including: extracting each row vector of the garbage location prediction parameter space to obtain a set of garbage location prediction parameter row vectors; calculating the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of garbage location prediction parameter row vectors to obtain a set of garbage location class probability entropy degradation simulation values; calculating the square root of the sum of squares of the garbage location class probability entropy degradation simulation values ​​to obtain a garbage location entropy degradation modulation value; based on the garbage location entropy degradation modulation value, the multi-scale road surface detection feature vector is subjected to feature distribution modulation to obtain the optimized multi-scale road surface detection feature vector.

[0015] In the above-mentioned road cleanliness detection method based on deep learning, the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of the garbage location prediction parameter row vector is calculated to obtain a set of garbage location class probability entropy degradation simulation values, including: using Spring Initializr to create a new Spring Boot project; creating a service class in the new Spring Boot project, and implementing the logic of calculating the garbage location class probability entropy degradation simulation value in the service class; creating a controller class in the new Spring Boot project, and the controller class is used to define routing and process external requests; packaging the Spring Boot project containing the function implementation operation, and deploying the Spring Boot project to the server.

[0016] This application has significant technical effects due to the adoption of the above technical solutions:

[0017] The road cleanliness detection method based on deep learning provided by the present application adopts image processing technology based on deep learning, performs feature analysis on the road image of the detection area to obtain road garbage location data, and sends a cleaning instruction to the unmanned cleaning robot in response to the road garbage location data. In this way, by automatically analyzing the road image and generating garbage location data, and then sending a cleaning instruction to the unmanned cleaning robot based on the data, the efficiency and accuracy of garbage cleaning can be significantly improved, and resource waste can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0019] Figure 1 This is a flow chart of a road cleanliness detection method based on deep learning according to an embodiment of the present application.

[0020] Figure 2 Schematic diagram of data flow of a road cleanliness detection method based on deep learning according to an embodiment of the present application.

[0021] Figure 3 The present invention is a flowchart of performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector in a road surface cleanliness detection method based on deep learning according to an embodiment of the present application.

[0022] Figure 4 In the road cleanliness detection method based on deep learning according to an embodiment of the present application, based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a flowchart of cleaning instructions is generated. DETAILED DESCRIPTION

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] In modern urban environmental management, the detection of road cleanliness and the efficient connection of cleaning operations are key factors in ensuring a clean city and improving the quality of life of residents. However, due to the complexity and variability of urban road environments, it is still a challenge to accurately locate and clean up garbage in a timely manner. In order to effectively detect and clean up road garbage and ensure the cleanliness of the road, a road cleanliness detection solution is expected.

[0025] In recent years, deep learning and neural networks have been widely used in computer vision, natural language processing, text signal processing and other fields. In addition, deep learning and neural networks have also demonstrated a level close to or even beyond that of humans in image classification, object detection, semantic segmentation, text translation and other fields. The development of deep learning and neural networks has provided new solutions and solutions for road cleanliness detection.

[0026] Figure 1 This is a flow chart of a road cleanliness detection method based on deep learning according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of a road surface cleanliness detection method based on deep learning according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the deep learning-based road cleanliness detection method of the embodiment of the present application, the method includes: S110, acquiring a road surface image of the area to be detected; S120, performing denoising processing on the road surface image of the area to be detected to obtain a road surface detection enhanced image; S130, performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector; S140, based on the multi-scale road surface detection feature vector, obtaining road surface garbage location data of the area to be detected, and generating a cleaning instruction.

[0027] In step S110, a road surface image of the area to be detected is obtained. It should be understood that the road surface image of the area to be detected contains garbage information, which is the most direct and critical information, including the type of garbage (such as paper scraps, plastic bags, cigarette butts, etc.), size, shape, color and its specific location on the road surface. This information is the basis for generating cleaning instructions. In addition to garbage information, road surface background information is also included, which may be different types of road materials and road markings. In general, by processing and analyzing road surface images through deep learning, garbage can be accurately identified and its location can be determined, and then effective cleaning instructions can be generated to guide efficient cleaning operations.

[0028] In step S120, the road surface image of the area to be detected is subjected to noise reduction processing to obtain a road surface detection enhanced image. Specifically, in an embodiment of the present application, the road surface image of the area to be detected is subjected to noise reduction processing to obtain a road surface detection enhanced image, including: passing the road surface image of the area to be detected through a road surface detection image denoiser to obtain the road surface detection enhanced image. It should be understood that the road surface image of the area to be detected is often affected by various factors during the acquisition process, such as changes in illumination, weather conditions, equipment noise, etc. These factors will cause the quality of the acquired road surface image to deteriorate, and problems such as blur and noise will occur. In order to effectively remove these noises, improve the clarity and signal-to-noise ratio of the image, and provide higher quality input for subsequent feature extraction and recognition, it is necessary to pass the road surface image of the area to be detected through a road surface detection image denoiser to obtain the road surface detection enhanced image. The presence of noise will interfere with the deep learning model's learning of image features and affect the recognition accuracy of the model. By performing noise reduction processing on the road surface image, the interference of noise on the model training process can be reduced, and the model's recognition ability for targets such as road garbage can be improved, thereby improving the accuracy of the entire garbage detection. The road detection image denoiser in this application is a denoiser based on an automatic codec. The automatic codec structure has a powerful feature learning capability and can learn deep features in the image. In the denoising process, it can extract the main features of the original road image through the encoding process and restore a clearer image with less noise in the decoding process. This structure has a certain robustness to noise and can still maintain good performance under noise interference.

[0029] In an embodiment of the present application, a possible implementation method of passing the road surface image of the area to be detected through a road surface detection image denoiser to obtain the road surface detection enhanced image may be: inputting the road surface image of the area to be detected into an encoder of the road surface detection image denoiser, wherein the encoder uses a convolutional layer to perform explicit spatial encoding on the road surface image of the area to be detected to obtain a road surface feature map to be detected; inputting the road surface feature map to be detected into a decoder of the road surface detection image denoiser, wherein the decoder uses a deconvolution layer to perform deconvolution processing on the road surface feature map to be detected to obtain the road surface detection enhanced image.

[0030] In step S130, multi-scale feature encoding is performed on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector. Specifically, Figure 3 The present invention is a flowchart of performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector in the road surface cleanliness detection method based on deep learning according to an embodiment of the present application. Figure 3 As shown, multi-scale feature encoding is performed on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector, including: S131, passing the road surface detection enhanced image through a first hole road surface image encoder to obtain a first-scale road surface detection feature vector; S132, passing the road surface detection enhanced image through a second hole road surface image encoder to obtain a second-scale road surface detection feature vector; S133, fusing the first-scale road surface detection feature vector and the second-scale road surface detection feature vector to obtain the multi-scale road surface detection feature vector.

[0031] Next, the road surface detection enhanced image is passed through the first hollow road surface image encoder to obtain a first-scale road surface detection feature vector. It should be understood that after the road surface detection enhanced image is pre-processed by denoising, enhancement, etc., although the image quality is improved, it still contains a large amount of redundant information and noise. In order to automatically extract features useful for road cleanliness detection from the denoised road surface detection enhanced image, while suppressing the interference of irrelevant information, it is necessary to pass the road surface detection enhanced image through the first hollow road surface image encoder to obtain a first-scale road surface detection feature vector. In this application, the first hollow road surface image encoder is a convolutional neural network model using a convolution kernel of a first size and a first hollow rate. Hollow convolution is a special convolution method that increases the receptive field by filling zeros (i.e., holes) in the convolution kernel, so that context information in a larger range can be captured without increasing the amount of calculation too much. This is particularly important for road detection, because garbage, stains, etc. on the road may be distributed in different locations and have different shapes and sizes. Using hollow convolution can ensure that the encoder can take into account this wide range of context information when extracting features.

[0032] In an embodiment of the present application, a possible implementation method of passing a road surface detection enhanced image through a first hole road surface image encoder to obtain a first-scale road surface detection feature vector may be as follows: each layer of the first hole road surface image encoder performs the following on the input data during the forward transmission process of the layer: convolving the input data based on a convolution kernel of a first size and a first hole rate to obtain a convolution feature map; performing mean pooling processing on the convolution feature map along the channel dimension of each feature matrix to obtain a pooling feature map; performing nonlinear activation on the pooling feature map to obtain an activation feature map; wherein the output of the last layer of the first hole road surface image encoder is the first-scale road surface detection feature vector, and the input of the first layer of the first hole road surface image encoder is the road surface detection enhanced image.

[0033] At the same time, the road surface detection enhanced image is passed through a second hole road surface image encoder to obtain a second-scale road surface detection feature vector. The second hole road surface image encoder in the present application is a convolutional neural network model using a convolution kernel of a first size and a second hole ratio. It should be understood that the hole convolution kernel can capture contextual information of different scales by using different hole ratios. The second hole ratio is usually larger than the first hole ratio, so it can capture a wider range of road surface features and help identify a wider range of road surface features. Although the hole ratio increases, the physical size of the convolution kernel remains unchanged, so the amount of calculation is still lower than that of the model using a larger size convolution kernel, which helps to maintain processing speed and meet road surface detection tasks with higher real-time requirements.

[0034] In an embodiment of the present application, a possible implementation method of passing the road surface detection enhanced image through a second hole road surface image encoder to obtain a second-scale road surface detection feature vector may be as follows: each layer of the second hole road surface image encoder performs the following on the input data during the forward transmission process of the layer: convolving the input data with a convolution kernel of a first size and a second hole rate to obtain a convolution feature map; performing mean pooling processing on the convolution feature map along the channel dimension of each feature matrix to obtain a pooling feature map; performing nonlinear activation on the pooling feature map to obtain an activation feature map; wherein the output of the last layer of the second hole road surface image encoder is the second-scale road surface detection feature vector, and the input of the first layer of the second hole road surface image encoder is the road surface detection enhanced image.

[0035] The first-scale road surface detection feature vector focuses more on details and local features, such as small stains, garbage, etc., while the second-scale road surface detection feature vector focuses on the cleanliness or pollution of a large area of ​​the road surface. In order to comprehensively utilize information of different scales and improve the accuracy and comprehensiveness of road surface cleanliness detection, the technical solution of this application needs to fuse the first-scale road surface detection feature vector and the second-scale road surface detection feature vector to obtain a multi-scale road surface detection feature vector. It should be understood that the pollution or cleanliness on the road surface appears in the form of different scales, such as small stains, large pieces of garbage, etc. By fusing multi-scale feature vectors, it is possible to better adapt to these targets of different scales and improve the flexibility of detection.

[0036] In an embodiment of the present application, a possible implementation method of fusing the first-scale road surface detection feature vector and the second-scale road surface detection feature vector to obtain a multi-scale road surface detection feature vector may be: fusing the first-scale road surface detection feature vector and the second-scale road surface detection feature vector using the following fusion formula to obtain the multi-scale road surface detection feature vector; wherein the fusion formula is:

[0037]

[0038] Among them, V c is the multi-scale road surface detection feature vector, V a is the first scale road surface detection feature vector, V b is the second scale road surface detection feature vector, represents the addition of the elements at corresponding positions of the first-scale road surface detection feature vector and the second-scale road surface detection feature vector, and α and β are weighting parameters for controlling the balance between the first-scale road surface detection feature vector and the second-scale road surface detection feature vector in the multi-scale road surface detection feature vector.

[0039] In step S140, based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a cleaning instruction is generated. Specifically, Figure 4 In the road cleanliness detection method based on deep learning according to the embodiment of the present application, based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a flow chart of a cleaning instruction is generated. Figure 4As shown, based on the multi-scale road surface detection feature vector, the road surface garbage position data of the area to be detected is obtained, and a cleaning instruction is generated, including: S141, the multi-scale road surface detection feature vector is subjected to feature distribution modulation selection based on chaos management to obtain an optimized multi-scale road surface detection feature vector; S142, the optimized multi-scale road surface detection feature vector is passed through a decoder-based garbage position information generator to obtain a decoded value, and the decoded value is the road surface garbage position data of the area to be detected; S143, based on the decoded road surface garbage position data of the area to be detected, the cleaning instruction is generated, and the cleaning instruction is sent to the unmanned cleaning robot.

[0040] In particular, considering that when extracting features at different scales from the road detection enhanced image, although the representation richness of the road detection features is increased, there is also repeated or similar information in the road detection features of different scales. This information is not completely necessary when performing road garbage detection, that is, it does not increase the effectiveness of the final garbage location generation task, which will lead to feature redundancy in the multi-scale road detection feature vector finally formed by fusion. In the process of decoding the multi-scale road detection feature vector, the decoder needs to map the encoded multi-scale feature vector back to the spatial dimension of the original road image in order to generate accurate garbage location data. When there is redundant information in the multi-scale road detection feature vector, the decoder will encounter ambiguity when processing this information, that is, multiple redundant features correspond to the same position features in the road image, which makes it difficult for the decoder to distinguish which features are important and which are not important, thereby affecting the iterative effect of decoding and the final garbage location data generation accuracy. Therefore, in the technical solution of the present application, it is necessary to perform feature distribution modulation selection based on chaos management on the multi-scale road detection feature vector to obtain an optimized multi-scale road detection feature vector.

[0041] Specifically, in an embodiment of the present application, the multi-scale road surface detection feature vector is subjected to feature distribution modulation selection based on chaos management to obtain an optimized multi-scale road surface detection feature vector, including: extracting each row vector of the garbage location prediction parameter space to obtain a set of garbage location prediction parameter row vectors; calculating the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of garbage location prediction parameter row vectors to obtain a set of garbage location class probability entropy degradation simulation values; calculating the square root of the sum of squares of the garbage location class probability entropy degradation simulation values ​​to obtain a garbage location entropy degradation modulation value; based on the garbage location entropy degradation modulation value, performing feature distribution modulation on the multi-scale road surface detection feature vector to obtain the optimized multi-scale road surface detection feature vector.

[0042] In an embodiment of the present application, another expression of performing feature distribution modulation selection based on chaos management on the multi-scale road surface detection feature vector to obtain an optimized multi-scale road surface detection feature vector may be: processing the multi-scale road surface detection feature vector using the following formula to obtain an optimized multi-scale road surface detection feature vector; wherein the formula is:

[0043] P={p1,p2,...,p j ,...,p n}

[0044] D j =Softmax(M m v c +H m p j +b m )

[0045]

[0046] Among them, P represents the set of garbage location prediction parameter row vectors, p1, p2, p j 、p n represents the first, second, jth, and nth garbage location prediction parameter row vectors in the set of garbage location prediction parameter row vectors, v c represents the multi-scale road surface detection feature vector, Softmax represents the normalized exponential function, M m represents the first weight matrix, H m represents the second weight matrix, b m represents the bias vector, D j represents the jth garbage location class probability entropy degradation simulation value of the set of garbage location class probability entropy degradation simulation values, t represents the garbage location entropy degradation modulation value, v c ⊙t represents the calculation of multi-scale road surface detection feature vector v c Each eigenvalue of the t-th power, represents the i-th eigenvalue of the multi-scale road surface detection feature vector, α represents a predetermined hyperparameter, represents addition by position, and V' represents the optimized multi-scale road surface detection feature vector.

[0047] More specifically, in an embodiment of the present application, the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of the garbage location prediction parameter row vector is calculated to obtain a set of garbage location class probability entropy degradation simulation values, including: using Spring Initializr to create a new Spring Boot project; creating a service class in the new Spring Boot project, and implementing the logic of calculating the garbage location class probability entropy degradation simulation value in the service class; creating a controller class in the new Spring Boot project, and the controller class is used to define routing and process external requests; packaging the Spring Boot project containing the function implementation operation, and deploying the Spring Boot project to the server.

[0048] In an embodiment of the present application, the code for calculating the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of garbage location prediction parameter row vectors to obtain a set of garbage location class probability entropy degradation simulation values ​​can be implemented as follows:

[0049]

[0050]

[0051]

[0052] It should be understood that using the Spring Web mechanism to deploy and implement specific computing services has the following benefits: a. The Spring framework provides rich infrastructure support, such as dependency injection and aspect-oriented programming, which can simplify the development process and improve development efficiency; b. Spring Web supports multiple configuration methods, and you can choose annotation configuration or XML configuration as needed to flexibly respond to different development needs; c. Spring Boot projects are easy to integrate with CI / CD tools, such as Jenkins, GitLab CI, etc., to facilitate automated construction and deployment; d. Spring's layered architecture and modular design make business logic, data access, and presentation layers clearly separated, which is easy to maintain and test; e. The Spring framework itself has excellent performance and provides a variety of optimization methods, such as asynchronous processing, caching, etc., which help improve service performance; f. Spring has a large community and rich documentation resources, and it is easy to find solutions when encountering problems. By using the Spring Web mechanism, you can build a reliable, scalable, and easy-to-maintain Web service to meet complex computing and data processing needs.

[0053] In the technical solution of the present application, the multi-scale road surface detection feature vector has feature redundancy, which leads to the problem that when it is decoded and generated by the decoder, there will be a run on the decoding labels relative to the decoding labels of the decoder, which affects the decoding iteration effect. Based on this, in the technical solution of the present application, the multi-scale road surface detection feature vector is subjected to feature distribution modulation selection based on chaos management, which first extracts each row vector from the garbage position prediction parameter space of the decoder to form a row vector set. Then, the garbage position class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each row vector in the above row vector set is calculated to measure the uncertainty or information entropy change of the multi-scale road surface detection feature vector under given parameters. Then, all garbage position class probability entropy degradation simulation values ​​are squared and summed, and then the square root is taken to obtain a comprehensive garbage position entropy degradation modulation value, which reflects the degree of entropy change of the entire system when processing feature vectors. Finally, the multi-scale road surface detection feature vector is adjusted according to the entropy degradation modulation value of the garbage location obtained above. Specifically, the attribute set of the multi-scale road surface detection feature vector is represented in a power series based on the entropy degradation modulation value, and a set aggregated bounded deviation represented by an aggregation radius based on each attribute value of the multi-scale road surface detection attribute vector is added, so as to perform attribute selection and sparsification optimization on the multi-scale road surface detection attribute vector under a unified entropy degradation modulation value restriction standard, so as to ensure that the attribute vector contains sufficient data but is not too cumbersome to introduce unnecessary interference or redundancy.

[0054] The optimized multi-scale road surface detection feature vector contains road surface information of different scales. In order to generate garbage location information based on the information in the optimized multi-scale feature vector, it is necessary to pass the optimized multi-scale road surface detection feature vector through a decoder-based garbage location information generator to obtain a decoded value, and the decoded value is the road surface garbage location data of the area to be detected. Based on the decoded road surface garbage location data of the area to be detected, a cleaning instruction is generated and sent to the unmanned cleaning robot. The cleaning instruction should contain a clear cleaning location. The cleaning instruction is sent to the unmanned cleaning robot through wireless communication technology. After receiving the cleaning instruction, the unmanned cleaning robot will perform the corresponding cleaning task according to the instruction content. The road surface cleanliness detection technology based on deep learning can realize the accurate detection and positioning of road surface garbage, thereby guiding the unmanned cleaning robot to perform targeted cleaning operations, which greatly improves the cleaning efficiency and reduces the waste of manpower and material resources.

[0055] In summary, the road cleanliness detection method based on deep learning in the embodiment of the present application is explained, which adopts deep learning-based image processing technology, performs feature analysis on the road image of the detection area to obtain road garbage location data, and sends a cleaning instruction to the unmanned cleaning robot in response to the road garbage location data. In this way, by automatically analyzing the road image and generating garbage location data, and then sending a cleaning instruction to the unmanned cleaning robot based on it, the efficiency and accuracy of garbage cleaning can be significantly improved, and resource waste can be reduced.

[0056] In other embodiments of the present application, a method for detecting the cleanliness of a road surface is also provided. It mainly detects garbage on the road surface through personnel inspections, and uses positioning intelligent devices to report the garbage location data to the platform. The platform will issue instructions to the unmanned cleaning robot based on the garbage location data. After receiving the instructions, the unmanned cleaning robot will find the optimal path to reach the designated location for cleaning operations.

[0057] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0058] Those of ordinary skill in the art will appreciate that the algorithmic steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0059] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A road cleanliness detection method based on deep learning, characterized in that: include: Acquire a road surface image of the area to be detected; Performing noise reduction processing on the road surface image of the area to be detected to obtain a road surface detection enhanced image; Performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector; Based on the multi-scale road surface detection feature vector, the road surface garbage location data of the area to be detected is obtained, and a cleaning instruction is generated.

2. The road cleanliness detection method based on deep learning according to claim 1 is characterized in that: The road surface image of the area to be detected is subjected to noise reduction processing to obtain a road surface detection enhanced image, comprising: passing the road surface image of the area to be detected through a road surface detection image noise reducer to obtain the road surface detection enhanced image.

3. The road cleanliness detection method based on deep learning according to claim 2 is characterized in that: Performing multi-scale feature encoding on the road surface detection enhanced image to obtain a multi-scale road surface detection feature vector includes: Passing the road surface detection enhanced image through a first hole road surface image encoder to obtain a first-scale road surface detection feature vector; Passing the road surface detection enhanced image through a second hole road surface image encoder to obtain a second-scale road surface detection feature vector; The first-scale road surface detection feature vector and the second-scale road surface detection feature vector are fused to obtain the multi-scale road surface detection feature vector.

4. The road cleanliness detection method based on deep learning according to claim 3 is characterized in that: The road surface detection image denoiser is a denoiser based on an automatic codec, the first hole road surface image encoder is a convolutional neural network model using a convolution kernel of a first size and a first hole rate, and the second hole road surface image encoder is a convolutional neural network model using a convolution kernel of a first size and a second hole rate.

5. The road cleanliness detection method based on deep learning according to claim 4 is characterized in that: Based on the multi-scale road surface detection feature vector, the road surface garbage position data of the area to be detected is obtained, and a cleaning instruction is generated, including: Performing feature distribution modulation selection based on chaos management on the multi-scale road surface detection feature vector to obtain an optimized multi-scale road surface detection feature vector; The optimized multi-scale road surface detection feature vector is passed through a decoder-based garbage location information generator to obtain a decoded value, wherein the decoded value is the road surface garbage location data of the area to be detected; Based on the decoded road garbage location data of the area to be detected, the cleaning instruction is generated and sent to the unmanned cleaning robot.

6. The road cleanliness detection method based on deep learning according to claim 5 is characterized in that: Performing feature distribution modulation selection based on chaos management on the multi-scale road surface detection feature vector to obtain an optimized multi-scale road surface detection feature vector includes: Extracting each row vector of the garbage location prediction parameter space to obtain a set of garbage location prediction parameter row vectors; Calculating garbage location class probability entropy degradation simulation values ​​between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of garbage location prediction parameter row vectors to obtain a set of garbage location class probability entropy degradation simulation values; Calculating the square root of the sum of squares of the garbage location class probability entropy degradation simulation values ​​to obtain a garbage location entropy degradation modulation value; Based on the garbage position entropy degradation modulation value, feature distribution modulation is performed on the multi-scale road surface detection feature vector to obtain the optimized multi-scale road surface detection feature vector.

7. The road cleanliness detection method based on deep learning according to claim 6 is characterized in that: Calculating the garbage location class probability entropy degradation simulation value between the multi-scale road surface detection feature vector and each garbage location prediction parameter row vector in the set of garbage location prediction parameter row vectors to obtain a set of garbage location class probability entropy degradation simulation values, including: Use Spring Initializr to create a new Spring Boot project; Create a service class in the new Spring Boot project, and implement the logic of calculating the probability entropy degradation simulation value of the garbage location class in the service class; Create a controller class in the new Spring Boot project, which is used to define routes and handle external requests; Package the Spring Boot project containing the functional implementation operation and deploy the Spring Boot project to the server.

Images