A road cleanliness detection method and intelligent cleaning method

Abstract

The present invention discloses a road cleanliness detection method and an intelligent cleaning method, the detection method comprises the following steps: obtaining an image of a road to be detected and identifying road garbage in the image, and calculating a first cleanliness index according to the identified road garbage; cleaning dust in a designated target area of ​​the road to be detected, obtaining images of the target area before and after cleaning and comparing them, and calculating a second cleanliness index according to the comparison result; calculating a comprehensive cleanliness index according to the first cleanliness index and the second cleanliness index, and obtaining a cleanliness detection result of the road to be detected according to the comprehensive cleanliness index. The intelligent cleaning method comprises: a cleaning vehicle collects road images in a designated range ahead in real time during the cleaning process, performs road cleanliness detection, and controls the cleaning vehicle to clean according to the detection result. The present invention has the advantages of simple implementation method, high detection accuracy and efficiency, and strong anti-interference.

Description

Technical Field

[0001] The present invention relates to the field of intelligent cleaning technology, and in particular to a road cleanliness detection method and an intelligent cleaning method. Background Art

[0002] Smart cleaning is to intelligently control the road sweeper to perform appropriate cleaning in real time according to the cleanliness of the road, so as to achieve intelligent and efficient road cleaning and road environment maintenance. The performance of smart cleaning directly depends on the detection accuracy of road cleanliness. Regarding the detection of road cleanliness, it is currently usually achieved by directly detecting garbage on the road, such as identifying specific objects in the street through visual analysis to identify garbage, and then determining the cleanliness of the road based on the distribution of garbage on the road. For example, Chinese patent application CN201910896277.X discloses a road cleanliness quantification method based on deep learning. The scheme is to use a deep network model to identify garbage on the road by collecting road images. The deep network model is pre-trained and generated using a large number of image data sets for classification and calibration. Chinese patent application CN202010663905.2 discloses a road surface cleanliness assessment method for intelligent road cleaning. The scheme is to extract the road surface area through the Yo l act road surface segmentation network, and then use the Yo lov3 target detection network and the MCNN density estimation network to obtain the location and category of garbage, and obtain the road surface cleanliness index. That is, the scheme also directly realizes road cleanliness detection by detecting garbage in the road.

[0003] However, the above road cleanliness detection method has the following problems:

[0004] 1. In addition to garbage, there is also dust on the road. When there is a lot of dust on the road, it needs to be cleaned. However, dust is different from garbage. It has no specific shape and is distributed irregularly. It cannot be directly identified by the method of garbage identification. Therefore, a single road garbage identification is difficult to accurately characterize the cleanliness status of the road.

[0005] 2. The shapes and other features of garbage targets are diverse, and it is actually difficult to extract the category features of many different types of garbage. In particular, it is difficult to accurately identify garbage of special shapes and categories using deep learning methods. Therefore, the above-mentioned method of identifying garbage based on visual analysis has a very low actual detection accuracy, resulting in inaccurate assessment of road cleanliness. 3. Due to the large number of garbage types, identifying garbage based on deep learning requires a large amount of model training, which will lead to reduced recognition efficiency. Therefore, the detection speed of the road cleanliness detection method in the above-mentioned prior art is difficult to meet the current demand for real-time detection. Summary of the invention

[0006] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides a road cleanliness detection method with simple implementation method, high detection accuracy and efficiency, and strong anti-interference ability, as well as an intelligent cleaning method with high intelligence, high cleaning efficiency, good cleaning effect and low power consumption.

[0007] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0008] A road cleanliness detection method, comprising the steps of:

[0009] Acquire an image of the road to be inspected and segment the boundary of road garbage from the image, and calculate a first cleanliness index according to the segmented boundary of the road garbage;

[0010] Cleaning dust in a designated target area of ​​the road to be inspected, obtaining images of the target area before and after cleaning and comparing them, and calculating a second cleanliness index according to the comparison result;

[0011] A comprehensive cleanliness index is calculated according to the first cleanliness index and the second cleanliness index, and a cleanliness detection result of the road to be detected is obtained according to the comprehensive cleanliness index.

[0012] Furthermore, the step of calculating the first cleanliness index according to the area size of the segmented road garbage boundary in the image includes:

[0013] In the acquired image of the road to be detected, the boundary of the garbage on the road is segmented;

[0014] Calculating the pixel area ratio of road garbage according to the road garbage boundary;

[0015] The first cleanliness index is calculated according to the pixel area ratio.

[0016] Furthermore, the steps of respectively acquiring images of the target area before and after cleaning and comparing them, and calculating the second cleanliness index according to the comparison results include:

[0017] Calculate the standard deviation T of the cleaned image of the target area. If the standard deviation exceeds the preset standard threshold T max , the second cleanliness index is configured as a preset value; otherwise, the images of the target area before and after cleaning are respectively subjected to grayscale binarization processing, and a difference matrix between the images of the target area before and after cleaning after the binarization processing is calculated, and each row of the difference matrix is ​​subjected to Fourier transformation and modulus calculation to obtain a Fourier transform sequence modulus value matrix, and the second cleanliness index is calculated according to the Fourier transform sequence modulus value matrix.

[0018] Furthermore, the second cleanliness index is calculated according to the Fourier transform sequence modulus matrix according to the following formula:

[0019]

[0020] Among them, I d is the second cleanliness index, δ is a preset coefficient, F(n,m) is the Fourier transform sequence modulus matrix, N×M is the number of pixels of the image of the target area after cleaning, and N and M are both even numbers.

[0021] Furthermore, the preset standard threshold T max It is calculated based on the road surface material and the real-time lighting status. The specific calculation formula is:

[0022]

[0023] Among them, a is the road surface material coefficient and b is the illumination coefficient.

[0024] Furthermore, the calculation expression for calculating the comprehensive cleanliness index according to the first cleanliness index and the second cleanliness index is:

[0025] R=W r I r +W d I d

[0026] Among them, W r and W d The first cleanliness index I r The corresponding weight factor and the second cleanliness index I d The corresponding weight factor, R, is the comprehensive cleanliness index.

[0027] An intelligent cleaning method, comprising the steps of:

[0028] The road sweeper collects images of the road in the specified range ahead in real time during the cleaning process;

[0029] Performing a road cleanliness detection on the road image collected in real time according to the road cleanliness detection method to obtain a detection result;

[0030] Determine whether the cleaning path of the road sweeper needs to be adjusted based on the road cleanliness detected. If adjustment is necessary, obtain the identification result of road garbage, and re-plan the cleaning path and cleaning mode of the road sweeper based on the location distribution status of road garbage and dust areas.

[0031] Furthermore, when the road sweeper collects road images within a specified range ahead in real time during the cleaning process, the image acquisition frame rate F and the driving speed V of the road sweeper are dynamically adjusted according to the road cleanliness detected, and the sweeping disc rotation torque N and suction force X of the road sweeper are adjusted according to the driving speed of the road sweeper.

[0032] Furthermore, the image acquisition frame rate F is adjusted according to the formula F=A×(C+1), wherein A represents the maximum frame rate of the image acquisition device, and C represents the road cleanliness index obtained by detection; the driving speed V of the sweeper is adjusted according to V=B×(10-C) / 10, wherein B represents the maximum operating speed of the sweeper, and the sweeping disc rotation torque N and suction force X of the sweeper are specifically adjusted according to the following formulas: N=a0*(a1-V), X=a0*(a1-V), and a0 and a1 are preset fixed values, respectively.

[0033] Furthermore, the method of judging whether it is necessary to adjust the cleaning path of the sweeper according to the road cleanliness obtained by the detection includes: re-detecting the road cleanliness of the cleaned road, and comparing the newly detected detection result with the road cleanliness obtained by the previous detection; if it is determined that the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is higher than a preset difference threshold, it is determined that there is no need to adjust the cleaning path; if the newly detected road cleanliness value is greater than a preset cleaning judgment threshold and the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is less than a preset threshold, it is determined that the cleaning path needs to be adjusted, and the sweeper's sweeping disc rotation torque and suction force are controlled to increase.

[0034] Compared with the prior art, the advantages of the present invention are:

[0035] 1. The present invention obtains an image of a road to be detected and then segments the boundary of road garbage, and then calculates a first cleanliness index to characterize the distribution of garbage in the road. At the same time, the second cleanliness index is calculated by comparing the images of the target area in the road to be detected before and after cleaning to characterize the distribution of dust in the road. Different detection methods are used according to the characteristics of garbage and dust, respectively, so as to accurately evaluate the state of garbage and dust in the road. Then, the first cleanliness index and the second cleanliness index are combined to calculate a comprehensive cleanliness index. The distribution of garbage and dust in the road can be combined to obtain an accurate road cleanliness detection result, effectively improving the detection accuracy and anti-interference performance. Moreover, since the first cleanliness index is calculated based on the boundary of road garbage, it does not need to rely on complex model training, and thus can greatly reduce the complexity and efficiency of ensuring the detection.

[0036] 2. The present invention performs real-time road cleanliness detection during the cleaning process of the road sweeper, and feeds back the cleaning effect based on the result of the road cleanliness detection to determine whether it is necessary to adjust the cleaning path of the road sweeper. When the cleaning effect of the road sweeper is not good, the cleaning path and cleaning mode can be adjusted in time according to the location distribution status of road garbage and dust areas, thereby effectively improving the cleaning efficiency and cleaning effect and reducing the power consumption required for cleaning. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the implementation flow of the road cleanliness detection method according to Example 1 of the present invention.

[0038] Figure 2 It is a schematic diagram of the implementation flow of calculating the second road cleaning index according to Example 1 of the present invention.

[0039] Figure 3 It is a schematic diagram of the implementation flow of the intelligent cleaning method of embodiment 2 of the present invention.

[0040] Figure 4 It is a schematic diagram of the implementation flow of intelligent cleaning control in a specific application embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0042] Embodiment 1:

[0043] like Figure 1 As shown, the steps of the road cleanliness detection method in this embodiment include:

[0044] S1. Acquire an image of the road to be detected and segment the boundary of the road garbage from the image, and calculate the first cleanliness index according to the segmented road garbage boundary;

[0045] S2. Clean the dust in the designated target area of ​​the road to be detected, obtain images of the target area before and after cleaning and compare them, and calculate a second cleanliness index based on the comparison results;

[0046] S3. Calculate a comprehensive cleanliness index according to the first cleanliness index and the second cleanliness index, and obtain a cleanliness detection result of the road to be detected according to the comprehensive cleanliness index.

[0047] This embodiment acquires an image of the road to be detected and then segments the boundaries of road garbage, and then calculates a first cleanliness index to characterize the distribution of garbage on the road. At the same time, the second cleanliness index is calculated by comparing the images of the target area in the road to be detected before and after cleaning to characterize the distribution of dust on the road. Different detection methods are used according to the characteristics of garbage and dust, respectively, so as to accurately evaluate the status of garbage and dust on the road. Then, the first cleanliness index and the second cleanliness index are combined to calculate a comprehensive cleanliness index. The distribution of garbage and dust on the road can be combined to obtain an accurate road cleanliness detection result, effectively improving the detection accuracy and anti-interference performance. Moreover, since the first cleanliness index is calculated based on the boundary of road garbage, it does not need to rely on complex model training, and thus can greatly reduce the complexity and efficiency of ensuring the detection.

[0048] In step S1 of this embodiment, the first cleanliness index is calculated according to the area size of the segmented road garbage boundary in the image, and the steps include:

[0049] S101. Segment the road garbage boundary in the acquired image of the road to be detected;

[0050] S102. Calculate the pixel area ratio of road garbage according to the road garbage boundary;

[0051] S103. Calculate a first cleanliness index according to the pixel area ratio.

[0052] In the above step S101, an image segmentation algorithm may be used to identify road garbage and perform boundary segmentation. The specific image segmentation algorithm may be selected according to actual needs.

[0053] In the above step S102, the pixel area ratio occupied by the road garbage boundary is calculated to obtain the distribution state of the road garbage in the road. In step S103, the first cleanliness index is determined based on the pixel area ratio. Specifically, the reciprocal of the pixel area ratio can be directly used as the first cleanliness index, or a specified ratio or multiple of the reciprocal of the pixel area ratio can be used as the first cleanliness index. The specific configuration can be based on actual needs. If the area ratio of the road garbage boundary exceeds a certain threshold, the corresponding first cleanliness index is small, which means that there are more road garbage and the road cleanliness judged from the road garbage is low. On the contrary, if the area ratio of the road garbage boundary is less than a certain threshold, the corresponding first cleanliness index is high, which means that there are less road garbage and the road cleanliness judged from the road garbage is high.

[0054] Since the present embodiment determines the corresponding first cleanliness index by identifying the boundary of road garbage, when the area of ​​dust is large and the accumulated thickness reaches a certain level, it may be mistakenly segmented as garbage. In order to avoid misidentifying this type of special dust area as garbage, in step S101 of the present embodiment, when segmenting the boundary of road garbage, it is further determined whether it may be a dust area. If it may be a dust area, the boundary area is deleted. Considering that the colors of garbage and dust areas are usually quite different, when determining whether the boundary of road garbage may be a dust area, each pixel of each identified road garbage can be obtained for comparison, and the area where the pixel difference is greater than the preset threshold can be screened out. The screened area can be further reconfirmed with the help of a pre-constructed dust area recognition model, so as to finally accurately identify whether there is a dust area mistakenly identified as road garbage.

[0055] like Figure 2 As shown, in step S2 of this embodiment, the images of the target area before and after cleaning are respectively obtained and compared, and the second cleanliness index is calculated according to the comparison result, including:

[0056] S201. Calculate the standard deviation T of the cleaned image of the target area. If the standard deviation exceeds the preset standard threshold T max , then the second cleanliness index is configured as a preset value; otherwise, the process goes to step S202;

[0057] S202. Perform grayscale binarization processing on the images before and after cleaning of the target area respectively, and calculate the difference matrix between the images before and after cleaning of the target area after the binarization processing, perform Fourier transform on each row of the difference matrix and calculate the modulus to obtain the Fourier transform sequence modulus value matrix, and calculate the second cleanliness index based on the Fourier transform sequence modulus value matrix.

[0058] In a specific application embodiment, a camera and a high-pressure air gun are installed on a sweeper, and the camera is perpendicular to the road surface. When the sweeper is driving, the camera first takes a road surface picture P1 before cleaning at a selected shooting point, and then starts the high-pressure air gun to clean the shooting point. After the cleaning process is completed, another road surface picture P2 is taken. Both P1 and P2 have N×M pixels, and N and M are both even numbers. The two pictures are sent to the processing module for calculation. The standard deviation T of P2 is calculated first. When T is greater than or equal to the preset standard threshold T max When it is , it indicates that there is a lot of dust on the road that needs to be cleaned, and the second cleanliness index I is directly configured d =1, to indicate the start of cleaning; when T is less than T max, then the two images P1 and P2 are processed by grayscale binarization, and the corresponding points in P1 and P2 are subtracted to obtain the N×M difference matrix P, and then each row of the matrix P is Fourier transformed and modulo is calculated to obtain the Fourier transform sequence modulus matrix F(n,m), and finally the second cleanliness index I is calculated based on F(n,m) d , which can accurately characterize the cleanliness of the road in terms of dust.

[0059] The second cleanliness index is calculated according to the Fourier transform sequence modulus matrix in the following formula:

[0060]

[0061] Among them, I d is the second cleanliness index, δ is a preset coefficient (specifically, it can be taken as 4), F(n,m) is the Fourier transform sequence modulus matrix, and M and N are the number of rows and columns of the F(n,m) matrix respectively.

[0062] The above preset standard threshold T max It is calculated based on the road surface material and the real-time lighting status, that is, the preset standard threshold T max It is adjusted in real time according to the road surface material and the real-time lighting conditions to adapt to different road environments and weather conditions, thereby further improving the calculation accuracy of the second cleanliness index to more accurately evaluate the state of dust on the road in different environments. max The calculation formula is:

[0063]

[0064] Among them, a is the road surface material coefficient and b is the illumination coefficient.

[0065] In this embodiment, the calculation expression for calculating the comprehensive cleanliness index R according to the first cleanliness index and the second cleanliness index is specifically:

[0066] R=W r I r +W d I d (3)

[0067] Among them, W r and W d The first cleanliness index I r The corresponding weight factor and the second cleanliness index I d The corresponding weight factor, W r It can be set according to the type, quantity and road grade of road garbage, that is, it is related to the type, quantity and road grade of road garbage, and can be obtained based on neural network calculation.d The specific setting can be based on the relative position of the road sweeper. For example, if the probability of dust being located on both sides of the road is higher, a higher weight value can be set for the areas on both sides of the road, and the W corresponding to the area close to the center of the road is d It can be set to a smaller value to further improve the calculation accuracy of the comprehensive cleanliness index R.

[0068] This embodiment further determines the road cleanliness level based on the calculated comprehensive cleanliness index R, and pre-configures a corresponding cleaning mode for each road cleanliness level. After the road cleanliness level is determined, it can be provided to the sweeper to control the sweeper to clean according to the cleaning mode suitable for the current road cleanliness level, so as to achieve intelligent and precise cleaning, greatly improve the degree of intelligence and cleaning effect, and avoid unnecessary cleaning operations and reduce cleaning power consumption.

[0069] In a specific application embodiment, the road cleanliness level can be divided into 10 levels according to Table 1. The smaller the level, the cleaner the road, and the larger the level, the dirtier the road and more cleaning is required.

[0070] Table 1: Classification of road cleanliness levels

[0071] grade R-value range grade R-value range 0 0≤R<0.01 5 0.1≤R<0.15 1 0.01≤R<0.03 6 0.15≤R<0.2 2 0.03≤R<0.05 7 0.2≤R<0.3 3 0.05≤R<0.08 8 0.3≤R<0.5 4 0.08≤R<0.1 9 R≥0.5

[0072] In a specific application embodiment, during the road cleaning process of a road sweeper, road images are collected in real time through a high-resolution camera installed on the vehicle, and the image data is transmitted to a cloud center through a vehicle-side wireless network. A cloud server is used to store and process road image information, segment road garbage boundaries, and count the amount of garbage, thereby calculating a first road cleaning index. At the same time, a road dust evaluation module is used in combination with a dust index evaluation model based on two-dimensional image similarity to calculate a second road cleaning index by comparing images of the target area before and after cleaning. Finally, the first road cleaning index and the second road cleaning index are combined to obtain a final comprehensive road cleaning index, and then the road cleanliness level is estimated according to a preset division rule to evaluate the urban road cleaning efficiency, and the road cleanliness level is further fed back to the vehicle to control the reasonable planning of road cleaning paths.

[0073] Embodiment 2:

[0074] like Figure 3 As shown, the steps of the intelligent cleaning method of this embodiment include:

[0075] Step S01. The road sweeper collects images of the road in a specified range ahead in real time during the cleaning process;

[0076] Step S02. For the road image collected in real time, a road cleanliness detection is performed according to the road cleanliness detection method in Example 1 to obtain a detection result;

[0077] Step S03. Determine whether the cleaning path of the road sweeper needs to be adjusted based on the road cleanliness detected. If adjustment is required, re-plan the cleaning path and cleaning mode of the road sweeper based on the location distribution of road garbage and dust areas.

[0078] This embodiment performs real-time road cleanliness detection during the cleaning process of the road sweeper, and feeds back the cleaning effect based on the result of the road cleanliness detection to determine whether the cleaning path of the road sweeper needs to be adjusted. When the cleaning effect of the road sweeper is not good, such as the road cleanliness cannot reach a certain value for a long time, indicating that there may be errors in the cleaning path and the area with more garbage and dust cannot be accurately cleaned, the cleaning path and cleaning mode can be adjusted in time according to the location distribution status of the road garbage and dust areas, thereby effectively improving the cleaning efficiency and cleaning effect and reducing the power consumption required for cleaning.

[0079] In this embodiment, when the road sweeper collects the road image in the specified range ahead in real time during the cleaning process, the image acquisition frame rate F and the driving speed V of the sweeper are dynamically adjusted according to the road cleanliness detected, and the sweeping disc rotation torque N and suction force X of the sweeper are adjusted according to the driving speed of the sweeper. That is, the cleaner the road, the slower the road image processing speed is controlled, and the faster the operating speed of the sweeper is; the dirtier the road, the faster the road image processing speed is controlled, and the slower the operating speed of the sweeper is. The sweeping control parameters of the sweeper can be dynamically adjusted according to the road cleaning state, so that the cleaning process matches the road cleaning state, thereby further reducing the power consumption during the cleaning process.

[0080] In a specific application embodiment, the image acquisition frame rate F can be adjusted according to the formula F=A×(C+1), where A represents the maximum frame rate of the image acquisition device, and C represents the road cleanliness index obtained by detection, which can be obtained by looking up a table (such as Table 1); the driving speed V of the sweeper is adjusted according to V=B×(10-C) / 10, where B represents the maximum operating speed of the sweeper, and the sweeping disc rotation torque N and suction force X of the sweeper are adjusted according to the following formulas: N=a0*(a1-V), X=a0*(a1-V), a0 and a1 are preset fixed values, which can be set to a0=300 and a1=40 respectively.

[0081] In this embodiment, judging whether it is necessary to adjust the cleaning path of the road sweeper according to the road cleanliness obtained by the detection includes: re-detecting the road cleanliness of the cleaned road, and comparing the newly detected detection result with the road cleanliness obtained by the previous detection. If the newly detected road cleanliness is less than the preset cleaning judgment threshold, it is judged that no cleaning is required; if it is judged that the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is higher than the preset difference threshold, that is, the front and back cleanings make a significant difference in road cleanliness, and the current cleaning effect is good, then it is judged that there is no need to adjust the cleaning path; if the newly detected road cleanliness is less than the preset cleaning judgment threshold, it is judged that the road cleanliness is less than the preset cleaning judgment ... the cleaning path is not required to be adjusted. If the detected road cleanliness is higher than the preset cleaning threshold, it is judged that cleaning is required, and the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is less than the preset difference threshold, that is, the front and back cleaning has not made a significant difference in road cleanliness, and the current cleaning effect is poor, then it is judged that the cleaning path needs to be adjusted, and the cleaning path and cleaning mode of the sweeper are re-planned according to the location distribution status of road garbage and dust areas, so that the sweeper can travel to the location of road garbage and dust areas for cleaning, and the sweeper's sweeping disc rotation torque and suction force are increased, thereby effectively improving the cleaning effect.

[0082] In a specific application embodiment, Figure 4 As shown, the detailed steps of the present invention to realize intelligent cleaning control are:

[0083] Step 1: After the cleaning vehicle is started, it receives the cleaning task sent by the cloud and goes to the task starting point according to the task instructions;

[0084] Step 2: After arriving at the mission starting point, the on-board visual device starts to collect road garbage images and sends them to the on-board image processing device for image processing to obtain the road cleanliness level C. The calculation method of the road cleanliness level C is as follows:

[0085] S1, the front vision module of the road sweeper captures a high-definition image of the road ahead and transmits the image to the cloud center via a wireless network;

[0086] S2: After receiving the road image data, the cloud center sends it to the deep learning module to process the image and identify the road garbage and segment the road garbage boundary. The first road cleanliness index I is calculated by the percentage of garbage pixel area. r and transmit it back to the sweeper;

[0087] S3, start the road dust index evaluation module of the road sweeper to calculate the second road cleanliness index I dSpecifically, the camera first takes a picture of the road surface before cleaning P1 at the selected shooting point, and then starts the high-pressure air gun to clean the shooting point. After the cleaning process is completed, another road surface picture P2 is taken. Both P1 and P2 are N×M pixels, and N and M are both even numbers. The two pictures are sent to the road dust index evaluation module for calculation; the road dust index evaluation module first calculates the standard deviation T of P2, and when T is greater than or equal to the preset standard threshold T max When it is , it indicates that there is a lot of dust on the road that needs to be cleaned, and the second cleanliness index I is directly configured d =1, to indicate the start of cleaning; when T is less than T max , then the two images P1 and P2 are processed by grayscale binarization, and the corresponding points in P1 and P2 are subtracted to obtain the N×M difference matrix P, and then each row of the matrix P is Fourier transformed and modulo is calculated to obtain the Fourier transform sequence modulus matrix F(n,m), and finally the second cleanliness index I is calculated based on F(n,m) d ;

[0088] S4, based on the first road cleanliness index I r and the second road cleanliness index I d The comprehensive road cleanliness index R is calculated.

[0089] S5, looking up a table according to the comprehensive road cleanliness index R to obtain the road cleanliness grade C.

[0090] Step 4: The road sweeper starts working, and the on-board visual equipment continuously collects road images, processes the image data on the vehicle side, the road side or the cloud side, and updates the road cleanliness level C and road garbage identification results in real time;

[0091] Step 5: The road cleanliness level C determines whether the cleaning path of the sweeper needs to be adjusted. If adjustment is required, the rotation torque of the sweeper's sweeping disc and the suction force of the suction cup are dynamically adjusted according to the distribution of road garbage and dust areas.

[0092] In the above step 4, the image processing end can choose to process road images based on the operating vehicle speed and the network link delay from the vehicle to the road and cloud. For example, when the delay meets the processing requirements, the road and cloud devices are given priority for road image processing.

[0093] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A road cleanliness detection method, characterized in that the steps include: Acquire an image of the road to be inspected and segment the boundary of road garbage from the image, and calculate a first cleanliness index according to the segmented boundary of the road garbage; Cleaning dust in a designated target area of ​​the road to be inspected, obtaining images of the target area before and after cleaning and comparing them, and calculating a second cleanliness index according to the comparison result; Calculating a comprehensive cleanliness index according to the first cleanliness index and the second cleanliness index, and obtaining a cleanliness detection result of the road to be detected according to the comprehensive cleanliness index; Calculating the first cleanliness index according to the area size of the segmented road garbage boundary in the image, the steps include: In the acquired image of the road to be detected, the boundary of the road garbage is segmented; Calculating the pixel area ratio of road garbage according to the road garbage boundary; Calculating the first cleanliness index according to the pixel area ratio; The steps of respectively acquiring images of the target area before and after cleaning and comparing them, and calculating the second cleanliness index according to the comparison results comprises: Calculate the standard deviation T of the cleaned image of the target area. If the standard deviation exceeds the preset standard threshold T max , then the second cleanliness index is configured as a preset value; otherwise, the images of the target area before and after cleaning are respectively subjected to grayscale binarization processing, and a difference matrix between the images of the target area before and after cleaning after the binarization processing is calculated, and each row of the difference matrix is ​​subjected to Fourier transformation and modulus calculation to obtain a Fourier transform sequence modulus value matrix, and the second cleanliness index is calculated according to the Fourier transform sequence modulus value matrix; The second cleanliness index is calculated according to the Fourier transform sequence modulus matrix according to the following formula: Among them, I d is the second cleanliness index, δ is a preset coefficient, F(n,m) is the Fourier transform sequence modulus matrix, N×M is the number of pixels of the image of the target area after cleaning, and N and M are both even numbers.

2. The road cleanliness detection method according to claim 1, characterized in that: The preset standard threshold T max It is calculated based on the road surface material and the real-time lighting status. The specific calculation formula is: Among them, a is the road surface material coefficient and b is the illumination coefficient.

3. The road cleanliness detection method according to any one of claims 1 to 2, characterized in that: The calculation expression for calculating the comprehensive cleanliness index according to the first cleanliness index and the second cleanliness index is: R=W r ·I r +W d ·I d Among them, W r and W d The first cleanliness index I r The corresponding weight factor and the second cleanliness index I d The corresponding weight factor, R, is the comprehensive cleanliness index.

4. An intelligent cleaning method, characterized in that the steps include: The road sweeper collects images of the road in the specified range ahead in real time during the cleaning process; Performing a road cleanliness detection on the road image collected in real time according to the road cleanliness detection method described in any one of claims 1 to 3 to obtain a detection result; According to the road cleanliness test results obtained, it is determined whether the cleaning path of the road sweeper needs to be adjusted. If adjustment is required, the identification results of road garbage are obtained, and the cleaning path and cleaning mode of the road sweeper are re-planned according to the location distribution status of the road garbage and dust areas; The calculating the first cleanliness index according to the area size of the segmented road garbage boundary in the image includes: In the acquired image of the road to be detected, the boundary of the road garbage is segmented; Calculating the pixel area ratio of road garbage according to the road garbage boundary; Calculating the first cleanliness index according to the pixel area ratio; The steps of respectively acquiring images of the target area before and after cleaning and comparing them, and calculating the second cleanliness index according to the comparison results comprises: Calculate the standard deviation T of the cleaned image of the target area. If the standard deviation exceeds the preset standard threshold T max , the second cleanliness index is configured as a preset value; otherwise, the images of the target area before and after cleaning are respectively subjected to grayscale binarization processing, and a difference matrix between the images of the target area before and after cleaning after the binarization processing is calculated, and each row of the difference matrix is ​​subjected to Fourier transformation and modulus calculation to obtain a Fourier transform sequence modulus value matrix, and the second cleanliness index is calculated according to the Fourier transform sequence modulus value matrix.

5. The intelligent cleaning method according to claim 4, characterized in that: When the road sweeper collects road images in a specified range ahead in real time during the cleaning process, the image acquisition frame rate F and the driving speed V of the road sweeper are dynamically adjusted according to the road cleanliness detection results obtained by the detection, and the sweeping disc rotation torque N and suction force X of the road sweeper are adjusted according to the driving speed of the road sweeper.

6. The intelligent cleaning method according to claim 5, characterized in that: The image acquisition frame rate F is adjusted according to the formula F=A×(C+1), where A represents the maximum frame rate of the image acquisition device, and C represents the road cleanliness level obtained according to the comprehensive cleanliness index R; the driving speed V of the sweeper is adjusted according to V=B×(10-C) / 10, where B represents the maximum operating speed of the sweeper, and the sweeping disc rotation torque N and suction force X of the sweeper are adjusted according to the following formulas: N=a0*(a1-V), X=a0*(a1-V), and a0 and a1 are preset fixed values ​​respectively.

7. The intelligent cleaning method according to any one of claims 4 to 6, characterized in that: The method of judging whether it is necessary to adjust the cleaning path of the sweeper according to the road cleanliness detection result obtained by the detection includes: re-detecting the road cleanliness of the cleaned road, and comparing the newly detected detection result with the road cleanliness obtained by the previous detection. If it is determined that the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is higher than the preset difference threshold, it is determined that there is no need to adjust the cleaning path; if the newly detected road cleanliness value is greater than the preset cleaning judgment threshold and the difference between the newly detected road cleanliness and the road cleanliness obtained by the previous detection is less than the preset threshold, it is determined that the cleaning path needs to be adjusted, and the sweeping disc rotation torque and suction of the sweeper are controlled to increase.

Images