Intelligent road sweeper and control method and device thereof

Abstract

The application provides an intelligent road sweeper and a control method and device thereof, and relates to the technical field of sweepers. The control method comprises steps S01 to S05 and step S10. S01, a picture captured by a front camera is acquired and preprocessed. S02, according to the preprocessed picture captured by the front camera, a to-be-cleaned area is acquired through an image segmentation algorithm. S03, according to the to-be-cleaned area, the category and area of to-be-cleaned garbage are acquired through a target detection algorithm. S04, according to the category and area of the to-be-cleaned garbage, the road cleanliness is acquired. S05, the power of an air blower, a second driving assembly and a water pump is acquired according to the road cleanliness. S10, when the to-be-cleaned area enters the bottom of the vehicle body, the air blower, the second driving assembly and the water pump are controlled to operate at the power. The intelligent road sweeper automatically identifies the dirt condition in front of the vehicle, automatically adjusts the suction force of a suction cup, the water pressure of a nozzle and the rotating speed of a disc brush, and uses the lowest energy consumption to clean the ground.

Description

Technical Field

[0001] This invention relates to the field of sweeper technology, and more specifically, to an intelligent road sweeper and its control method and device. Background Technology

[0002] Road sweepers are equipped with brushes, suction nozzles, and spray heads on a vehicle chassis to wash and sweep the road surface, and suck up garbage into the garbage bin, thus completing the tasks of road sweeping, garbage collection, and transportation to ensure the cleanliness of the road surface.

[0003] Traditional road sweepers do not allow for adjustments to the brush speed, brush pressure on the ground, suction power of the nozzles, or water pressure from the spray nozzles; they are all set by default based on the dirtiest road surface. Therefore, even when cleaning relatively clean surfaces, they operate at maximum power, resulting in significant resource waste.

[0004] Some road sweepers with adjustable sweeping power allow for manual control of the power of their various components via the dashboard in the cab. However, manual control requires the operator in the cab to constantly monitor road conditions and make accurate judgments, which not only increases the difficulty of controlling the road sweeper but also poses a challenge to the driver's safe driving.

[0005] In view of this, the applicant hereby submits this application after studying the existing technology. Summary of the Invention

[0006] The present invention provides an intelligent road sweeper and its control method and apparatus to improve at least one of the above-mentioned technical problems.

[0007] First aspect

[0008] This invention provides an intelligent road sweeper, which includes a vehicle body, a dust collection device, a sweeping device, a washing device, a detection device, and a control device.

[0009] The vacuum system includes a trash can, a suction nozzle, and a blower, all mounted on the vehicle body. The suction nozzle is located below the vehicle body. The blower is connected to the suction nozzle and the trash can, configured to suck objects in front of the suction nozzle into the trash can.

[0010] The cleaning device includes a disc brush mounted on the vehicle body, a first drive assembly, and a second drive assembly. The disc brush is located on both sides of the bottom of the vehicle body. The first drive assembly is configured to drive the disc brush to rise and fall. The second drive assembly is configured to drive the disc brush to rotate.

[0011] The washing device includes a water tank, nozzles, and a water pump disposed on the vehicle body. The nozzles are located below the vehicle body. The water pump is connected to the water tank and the nozzles and is configured to spray water from the water tank onto the road surface through the nozzles.

[0012] The detection device includes a front camera mounted on the vehicle body. The front camera is configured to capture images of the road surface in front of the vehicle body.

[0013] The control device is electrically connected to the front camera, the exhaust fan, the first drive assembly, the second drive assembly, and the water pump. The control device includes a processor and a memory. The processor is configured to execute a computer program stored in the memory to perform steps S01 to S05, and step S10.

[0014] S01. Acquire the image captured by the front camera and perform preprocessing.

[0015] S02. Based on the pre-processed image captured by the front camera, the area to be cleaned is obtained through an image segmentation algorithm.

[0016] S03. Based on the area to be cleaned, obtain the type and area of ​​the garbage to be cleaned through a target detection algorithm.

[0017] S04. Obtain the road surface cleanliness level based on the type and area of ​​the garbage to be cleaned. The calculation model for road surface cleanliness level AC is as follows: In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0018] S05. Obtain the power of the exhaust fan, the second drive assembly, and the water pump based on the road surface cleanliness. The power is a value preset based on the road surface cleanliness.

[0019] S10. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, second drive assembly and water pump are operated at power according to the control.

[0020] The second aspect

[0021] This invention provides a control method for an intelligent road sweeper, which includes steps S01 to S05 and step S10.

[0022] S01. Acquire the image captured by the front camera and perform preprocessing.

[0023] S02. Based on the pre-processed image captured by the front camera, the area to be cleaned is obtained through an image segmentation algorithm.

[0024] S03. Based on the area to be cleaned, obtain the type and area of ​​the garbage to be cleaned through a target detection algorithm.

[0025] S04. Obtain the road surface cleanliness level based on the type and area of ​​the garbage to be cleaned. The calculation model for road surface cleanliness level AC is as follows: In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0026] S05. Obtain the power of the exhaust fan, the second drive assembly, and the water pump based on the road surface cleanliness. The power is a value preset based on the road surface cleanliness.

[0027] S10. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, second drive assembly and water pump are operated at power according to the control.

[0028] Third aspect

[0029] This invention provides a control device for an intelligent road sweeper, comprising:

[0030] The front view acquisition module is used to acquire and preprocess the images captured by the front camera.

[0031] The front view segmentation module is used to obtain the area to be cleaned based on the pre-processed image captured by the front camera using an image segmentation algorithm.

[0032] The vehicle front image recognition module is used to obtain the type and area of ​​the garbage to be cleaned based on the area to be cleaned through a target detection algorithm.

[0033] The cleanliness calculation module is used to obtain the road surface cleanliness based on the type and area of ​​the debris to be cleaned. The calculation model for road surface cleanliness AC is as follows: In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0034] The cleaning power acquisition module is used to acquire the power of the exhaust fan, the second drive assembly, and the water pump based on the road surface cleanliness. The power is a pre-set value based on the road surface cleanliness.

[0035] The execution module is used to control the exhaust fan, the second drive assembly, and the water pump to operate at power when the area to be cleaned enters the bottom of the vehicle body.

[0036] By adopting the above technical solution, the present invention can achieve the following technical effects:

[0037] The intelligent road sweeper of this invention can automatically identify the dirt in front of the vehicle, automatically adjust the suction of the suction cup, the water pressure of the nozzle, and the rotation speed of the brush, and clean the ground with the lowest energy consumption, which has great practical significance. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 A schematic diagram of the structure of an intelligent road sweeper.

[0040] Figure 2 This is a flowchart illustrating the control method.

[0041] Figure 3 It is a logic control diagram of the control method.

[0042] Figure 4 This is a schematic diagram of an image segmentation algorithm.

[0043] Figure 5 This is a schematic diagram of the target detection algorithm.

[0044] The markings in the diagram are: 1-sweeping device, 2-front camera, 3-rear camera, 4-radar, 5-vacuuming device, 6-trash can, 7-control device. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0046] Example 1

[0047] Please see Figures 1 to 5 The first embodiment of the present invention provides an intelligent road sweeper, which includes a vehicle body, a dust collection device 5, a sweeping device 1, a washing device, a detection device and a control device 7.

[0048] like Figure 1 As shown, the vacuuming device 5 includes a trash can 6 disposed on the vehicle body, a suction nozzle, and an exhaust fan. The suction nozzle is located below the vehicle body. The exhaust fan is connected to the suction nozzle and the trash can 6 and is configured to suck objects in front of the suction nozzle into the trash can 6.

[0049] like Figure 1As shown, the sweeping device 1 includes a disc brush, a first drive assembly, and a second drive assembly disposed on the vehicle body. The disc brush is located on both sides of the bottom of the vehicle body. The first drive assembly is configured to drive the disc brush to rise and fall. The second drive assembly is configured to drive the disc brush to rotate.

[0050] like Figure 1 As shown, the washing device includes a water tank, nozzles, and a water pump disposed on the vehicle body. The nozzles are located below the vehicle body. The water pump is connected to the water tank and the nozzles and is configured to spray water from the water tank onto the road surface through the nozzles.

[0051] like Figure 1 As shown, the detection device includes a front camera 2 disposed on the vehicle body. The front camera 2 is configured to capture images of the road surface in front of the vehicle body.

[0052] The control device 7 is electrically connected to the front camera 2, the exhaust fan, the first drive assembly, the second drive assembly, and the water pump. The control device 7 includes a processor and a memory. The processor is configured to execute a computer program stored in the memory to perform steps S01 to S05, and step S10.

[0053] S01. Acquire the image captured by the front camera 2 and perform preprocessing.

[0054] Specifically, the front camera 2 is installed directly in front of the sweeper truck to capture images of the road ahead as the sweeper truck travels. The images undergo preprocessing, the main purpose of which is to remove noise and irrelevant information, recover useful information, and enhance the detectability of relevant information, thereby improving the reliability of road segmentation and object detection.

[0055] Based on the above embodiments, in an optional embodiment of the present invention, step S01 specifically includes:

[0056] S011. Acquire the image captured by the front camera 2.

[0057] S012. Perform grayscale conversion, geometric transformation, and image enhancement on the image captured by the front camera 2 to obtain the pre-processed image captured by the front camera 2.

[0058] Specifically, the image preprocessing workflow includes: grayscale conversion, geometric transformation, and image enhancement. Grayscale conversion converts a color image into a black and white image, requiring only one byte to store the grayscale value for each pixel, thus reducing the amount of data to be processed. Geometric transformation processes the acquired image through translation, rotation, scaling, mirroring, etc., primarily correcting systematic errors in image acquisition and random errors in instrument positioning. Image enhancement enhances useful information in the image, purposefully enhancing overall or local features, making previously unclear images clearer or emphasizing certain features of interest, widening the differences between features of different objects in the image, suppressing features of little interest, and improving image quality to meet the needs of road segmentation and waste identification.

[0059] S02. Based on the pre-processed image captured by the front camera 2, the area to be cleaned is obtained through an image segmentation algorithm.

[0060] Specifically, the image segmentation network is first used to extract and fit the road edges from the footage captured by the camera, thereby identifying the areas to be cleaned. This effectively prevents subsequent steps from identifying trash in areas that the sweeper cannot clean, such as trash in green belts.

[0061] Based on the above embodiments, in an optional embodiment of the present invention, step S02 specifically includes: obtaining the area to be cleaned by using an image semantic segmentation algorithm based on the FCN-8S network structure according to the preprocessed image captured by the front camera 2.

[0062] In this embodiment, the road surface segmentation neural network for the area to be cleaned uses the FCN-8S framework structure. In other embodiments, other existing image segmentation models can be used, and this invention does not specifically limit them.

[0063] like Figure 4 As shown, the FCN network structure is mainly divided into two parts: a fully convolutional network and a deconvolutional network. The fully convolutional network uses VGG to extract image features, while the deconvolutional network obtains a semantic segmentation image of the original image size through upsampling.

[0064] The input to FCN is an image of the road surface, and the output is the area of ​​the road surface to be cleaned and other background information. The principle of FCN is as follows: A road surface image is input into FCN. First, it passes through convolutional and pooling layers to obtain a feature map. Then, deconvolution and skipping steps are used to reconstruct a feature map of the same size as the original input image. This feature map is compared with the original image, enabling prediction of each pixel while preserving the spatial information in the original input image.

[0065] S03. Based on the area to be cleaned, obtain the type and area of ​​the garbage to be cleaned through a target detection algorithm.

[0066] Specifically, the system identifies and locates objects in the area to be cleaned. It employs the one-stage object detection algorithm YOLOv7, which uses pre-defined bounding boxes on the entire area to be cleaned and classifies and regresses the object's center point, length, width, and type to achieve object identification and localization.

[0067] Based on the above embodiments, in an optional embodiment of the present invention, step S03 specifically includes: obtaining the category and area of ​​the debris to be cleaned according to the area to be cleaned using a target detection algorithm based on the YOLOv7 network structure. The area of ​​the debris to be cleaned is the width multiplied by the height of the target bounding box.

[0068] Specifically, by combining image segmentation and object detection algorithms, this method identifies litter after segmenting and extracting the road cleaning area, avoiding misjudgments and significantly improving the accuracy of litter identification, which has great practical significance. This embodiment uses the one-stage object detection algorithm YOLOv7. In other embodiments, those skilled in the art can use other existing object recognition algorithms, and this invention does not specifically limit them.

[0069] The YOLOv7 framework network consists of three parts: input, backbone, and head. Preprocessing and data augmentation methods are used in the input layer to preprocess the input data, which is beneficial for network training. The backbone is used as the main feature extraction network, and the head layer is used for target object prediction.

[0070] The backbone layer of YOLOv7 consists of several BConv layers, ELAN-E layers, and MBConv layers. The BConv layer is composed of convolutional layers, BN layers, and activation functions. The ELAN-E layer is also composed of different convolutional layers. The MBConv layer is composed of max pooling layers and BConv layers. The entire backbone layer alternately halves the width and height and doubles the channels to extract features.

[0071] The entire head layer of Yolov7 consists of SPPCPC layers, several BConv layers, several MBConv layers, several CtConv layers, and the RepVGG block layers of the subsequent three heads. The head layer of Yolov7 mainly utilizes the FPN feature pyramid to enable the network to predict object bounding boxes at multiple scales. At the first scale, it is suitable for detecting large objects; at the second scale, it is suitable for detecting medium-sized objects; and at the third scale, it is suitable for detecting small objects. By training and predicting at three different scales, the network can predict the category and bounding box of objects at different scales.

[0072] In the initial training of the object detection algorithm, samples need to be manually created. Beforehand, image data of road debris is collected using a camera, and the targets in each image are labeled using LabelImg. The label information is (class, x, y, w, h). Here, class represents the type of object being labeled, x and y represent the x and y coordinates of the normalized target center point, and w and h represent the width and height of the normalized target bounding box, respectively.

[0073] S04. Obtain the road surface cleanliness level based on the type and area of ​​the garbage to be cleaned. The calculation model for road surface cleanliness level AC is as follows:

[0074]

[0075] In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0076] Preferably, α = 0.7 and β = 0.6.

[0077] Specifically, by combining the weight and area ratio of waste, the cleanliness of the road surface can be accurately determined, thus providing an accurate information basis for the selection of cleaning power, which has great practical significance.

[0078] Based on the above embodiments, in an optional embodiment of the present invention, the initial weight values ​​of each waste category are as follows: the initial weight value of leaves and cigarette butts is 1, the initial weight value of paper towels is 2, the initial weight value of garbage bags is 3, the initial weight value of soil is 4, and the initial weight value of stones is 5.

[0079] Preferably, the calculation model for the weights of all garbage in the area to be cleaned and C1 is as follows:

[0080]

[0081] In the formula, n is the number of pieces of trash detected in the area to be cleaned, and Q is the number of pieces of trash detected in the area to be cleaned. n Let be the weight of the nth garbage.

[0082] Specifically, the weights are the sum of the weights of each piece of garbage identified.

[0083] Preferably, the calculation model for the ratio C2 of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned is as follows:

[0084]

[0085] In the formula, n is the number of pieces of trash detected in the area to be cleaned, and w n The width of the target detection box representing the object, h n S represents the height of the object detection box and S represents the area of ​​the cleaning area.

[0086] In this embodiment, the ratio of the garbage area to the area to be cleaned is directly multiplied by 100. In other embodiments, the ratio can be converted according to the actual ratio, and this invention does not specifically limit this. For example: when the proportion of the total identified garbage area exceeds 70% of the identified area, the area ratio value is set to 25. When the proportion of the total identified garbage area exceeds 40% of the identified area, the area ratio value is set to 15. When the proportion of the total identified garbage area is more than or less than 40% of the area, the area ratio value is set to 10.

[0087] S05. Obtain the power of the exhaust fan, the second drive component, and the water pump based on the road surface cleanliness; wherein the power is a value preset based on the road surface cleanliness.

[0088] As shown in Table 1, based on the above embodiments, in an optional embodiment of the present invention, the correspondence between power and road surface cleanliness is as follows: When the road surface cleanliness is not greater than 50, the disc brush speed is 60 r / min, the exhaust fan speed is 1200 r / min, and the water pump is not working. When the road surface cleanliness is not greater than 100, the disc brush speed is 80 r / min, the exhaust fan speed is 1500 r / min, and the water pump is not working. When the road surface cleanliness is not greater than 150, the disc brush speed is 90 r / min, the exhaust fan speed is 2000 r / min, and the water pump working pressure is 5 MPa. When the road surface cleanliness is not greater than 200, the disc brush speed is 100 r / min, the exhaust fan speed is 2500 r / min, and the water pump working pressure is 8 MPa. When the road surface cleanliness is greater than 250, the disc brush speed is 120 r / min, the exhaust fan speed is 3000 r / min, and the water pump working pressure is 10 MPa.

[0089] In other embodiments, the inventors may set other power parameters according to actual needs / hardware settings. This invention does not specifically limit the power parameters of each cleaning component.

[0090] Table 1 Power at different cleanliness levels

[0091]

[0092] S10. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive component and the water pump are controlled to operate according to the power.

[0093] Specifically, the road surface cleanliness calculation model of this invention can assess road surface cleanliness based on the category weight value and size information of the garbage. Based on this, it achieves the optimal output power of the intelligent road sweeper's sweeping mechanism under various complex road conditions, thereby effectively solving the problem in existing technologies where the sweeper's operating mechanism cannot automatically adjust its output power in real time according to the garbage situation and road conditions, resulting in high energy loss and poor sweeping effect.

[0094] The intelligent road sweeper of this invention can automatically identify the dirt in front of the vehicle, automatically adjust the suction of the suction cup, the water pressure of the nozzle, and the rotation speed of the brush, and clean the ground with the lowest energy consumption, which has great practical significance.

[0095] Based on the above embodiments, in an optional embodiment of the present invention, the sweeping device 1 further includes a pressure sensor. The sensor is configured to detect the pressure of the disc brush on the road surface.

[0096] Step S05 specifically includes: obtaining the power of the exhaust fan, the first drive component, the second drive component, and the water pump based on the road surface cleanliness.

[0097] Step S10 specifically includes: when the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the first drive component, the second drive component and the water pump are operated according to the power control.

[0098] In this embodiment, the disc brush is driven to rise and fall by a first drive component, and is configured to adjust the pressure of the disc brush on the road surface. Higher pressure results in greater cleaning force, but also faster wear on the disc brush. In other embodiments, the disc brush may be configured with a non-adjustable pressure structure, only having two positions: retracted and extended. This invention does not specifically limit the drive structure of the disc brush. Preferably, the pressure of the disc brush on the road surface is set to five pressure levels.

[0099] Based on the above embodiments, in an optional embodiment of the present invention, the detection device further includes a radar 4 disposed on the vehicle body. The radar 4 is configured to detect the position of the debris to be cleaned in front of the vehicle body.

[0100] Specifically, such as Figure 1 As shown, the lidar 4 is mounted above the front of the sweeper vehicle. The lidar 4 is a high-beam mechanical lidar that collects point cloud data from the surroundings, enabling 360-degree all-around perception of the surrounding environment and obtaining distance information of objects relative to the lidar 4.

[0101] Preferably, steps S06 to S09 are included before step S10.

[0102] S06. Combine the image of the area to be cleaned and the detection data of radar 4 for joint calibration to obtain the location information of the garbage to be cleaned in the area to be cleaned.

[0103] Specifically, point cloud data is acquired using LiDAR 4. The point cloud is preprocessed, and then the coordinate system of the forward-looking camera and LiDAR 4 is jointly calibrated. The image and point cloud data are matched to determine the positional relationship of objects.

[0104] S07. Based on the location information, obtain the distance between the garbage to be cleaned and the vehicle body.

[0105] Specifically, after joint calibration, the location coordinates of each pollutant, captured by the front camera 2, are obtained using information detected by radar 4. This allows for more accurate coordinate information, thus providing the distance information between the pollutants and the vehicle body.

[0106] S08. Obtain the vehicle's movement speed.

[0107] S09. Based on the distance and moving speed, obtain the time it takes for the garbage to be cleaned to pass under the vehicle body.

[0108] Preferably, step S10 specifically includes:

[0109] The operation of the exhaust fan, the second drive component, and the water pump is controlled based on the time it takes for the garbage to pass under the vehicle body and the power.

[0110] Specifically, based on distance information and the vehicle's speed, the time it takes for the waste to pass over each device under the vehicle is determined. During this time, each device is controlled to operate at a pre-determined power level to ensure effective cleaning.

[0111] Based on the above embodiments, in an optional embodiment of the present invention, the detection device further includes a rear camera 3 disposed on the vehicle body. The rear camera 3 is configured to capture images of the road surface behind the vehicle body.

[0112] Specifically, a rear camera (3) is used to capture images behind the vehicle to assess the cleaning results. The driver is promptly notified if the cleaning is incomplete.

[0113] Preferably, after step S10, the method further includes:

[0114] S11. Acquire the image captured by the rear camera 3 and perform preprocessing.

[0115] Specifically, the preprocessing process is the same as the image captured by camera 2 before preprocessing, and will not be described in detail here.

[0116] S12. Based on the pre-processed image captured by the rear camera 3, obtain the image of the area to be cleaned after cleaning through an image matching algorithm.

[0117] In this embodiment, the image matching algorithm employs a feature-based matching method. First, features are extracted from the original image, and then a matching correspondence between features is established between the foreground and background images. Feature matching has three key steps: feature extraction, feature description, and feature matching. Feature extraction involves extracting keypoints, feature points, or corner points from the image. Feature description uses a set of mathematical vectors to describe the feature points, primarily ensuring a correspondence between different vectors and different feature points, while minimizing the differences between similar keypoints. Feature matching is essentially the calculation of the distance between feature vectors, commonly using Euclidean distance.

[0118] In other embodiments, other existing image matching algorithms may be used, and the present invention does not specifically limit them.

[0119] S13. Based on the image of the area to be cleaned after cleaning, the type of uncleaned garbage is obtained through a target detection algorithm.

[0120] Specifically, the target detection process is the same as the steps for recognizing the image captured by the front camera 2, and will not be described in detail here.

[0121] S14. Adjust the weight of the type of uncleaned garbage according to its type. The adjustment of the weight of the type of uncleaned garbage includes: the weight value of the type of uncleaned garbage + 1.

[0122] Specifically, based on the cleaning area extracted by the front-view camera, a feature matching algorithm is used to match the cleaning areas from the front and rear cameras to obtain the cleaning area after the cleaning operation. If the area is not cleaned properly, the weight value of the corresponding uncollected debris is increased, and a signal is sent to the street cleaning personnel for auxiliary cleaning to achieve complete cleaning of the road debris.

[0123] In addition, the acquired cleaned areas can be input into the target detection algorithm to evaluate road surface cleanliness and detect whether the road surface is clean. The road surface cleanliness evaluation model is as follows:

[0124]

[0125] In the formula, CT represents the cleanliness comparison value, and AC represents the cleanliness comparison value. t-1 For the cleanliness of the road surface before sweeping, AC t It refers to the cleanliness of the road surface after sweeping.

[0126] By using a rear camera (3) to identify the cleaned road surface, feedback adjustments are made. The weights of different waste categories are automatically corrected to ensure the accuracy, reliability, and real-time performance of the intelligent road sweeper.

[0127] Example 2

[0128] Please see Figures 2 to 5 This invention provides a control method for an intelligent road sweeper, which can be executed by the intelligent road sweeper. Specifically, it is executed by one or more processors in the intelligent road sweeper to implement steps S01 to S05 and step S10.

[0129] S01. Acquire the image captured by the front camera and perform preprocessing.

[0130] S02. Based on the pre-processed image captured by the front camera, the area to be cleaned is obtained through an image segmentation algorithm.

[0131] S03. Based on the area to be cleaned, obtain the type and area of ​​the garbage to be cleaned through a target detection algorithm.

[0132] S04. Obtain the road surface cleanliness level based on the type and area of ​​the garbage to be cleaned. The calculation model for road surface cleanliness level AC is as follows: In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0133] S05. Obtain the power of the exhaust fan, the second drive assembly, and the water pump based on the road surface cleanliness. The power is a value preset based on the road surface cleanliness.

[0134] S10. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, second drive assembly and water pump are operated at power according to the control.

[0135] Based on the above embodiments, in an optional embodiment of the present invention, step S01 specifically includes:

[0136] S011. Acquire the image captured by the front camera.

[0137] S012. Perform grayscale conversion, geometric transformation, and image enhancement on the image captured by the front camera to obtain the pre-processed image captured by the front camera.

[0138] Step S02 specifically includes:

[0139] Based on the pre-processed images captured by the front camera, the area to be cleaned is obtained using an image semantic segmentation algorithm based on the FCN-8S network structure.

[0140] Step S03 specifically includes:

[0141] Based on the area to be cleaned, a target detection algorithm based on the YOLOv7 network structure is used to obtain the category and area of ​​the debris to be cleaned. The area of ​​the debris is the width multiplied by the height of the bounding box.

[0142] Based on the above embodiments, in an optional embodiment of the present invention,

[0143] Step S05 specifically includes: obtaining the power of the exhaust fan, the first drive component, the second drive component, and the water pump based on the road surface cleanliness.

[0144] Step S10 specifically includes: when the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the first drive component, the second drive component and the water pump are operated according to the power control.

[0145] Based on the above embodiments, in an optional embodiment of the present invention, steps S06 to S09 are further included before step S10.

[0146] S06. Combine the image data of the area to be cleaned with the radar detection data to obtain the location information of the garbage to be cleaned in the area to be cleaned.

[0147] S07. Based on the location information, obtain the distance between the garbage to be cleaned and the vehicle body.

[0148] S08. Obtain the vehicle's movement speed.

[0149] S09. Based on the distance and moving speed, obtain the time it takes for the garbage to be cleaned to pass under the vehicle body.

[0150] Step S10 specifically includes: controlling the operation of the exhaust fan, the second drive component, and the water pump based on the time it takes for the garbage to be cleaned to pass under the vehicle body and the power.

[0151] Based on the above embodiments, in an optional embodiment of the present invention, step S10 is followed by:

[0152] S11. Acquire the image captured by the rear camera and perform preprocessing.

[0153] S12. Based on the pre-processed image captured by the rear camera, obtain the image of the area to be cleaned after cleaning through an image matching algorithm.

[0154] S13. Based on the image of the area to be cleaned after cleaning, the type of uncleaned garbage is obtained through a target detection algorithm.

[0155] S14. Adjust the weight of the type of uncleaned garbage according to its type. The adjustment of the weight of the type of uncleaned garbage includes: the weight value of the type of uncleaned garbage + 1.

[0156] Example 3

[0157] This invention provides a control device for an intelligent road sweeper, comprising:

[0158] The front view acquisition module is used to acquire and preprocess the images captured by the front camera.

[0159] The front view segmentation module is used to obtain the area to be cleaned based on the pre-processed image captured by the front camera using an image segmentation algorithm.

[0160] The vehicle front image recognition module is used to obtain the type and area of ​​the garbage to be cleaned based on the area to be cleaned through a target detection algorithm.

[0161] The cleanliness calculation module is used to obtain the road surface cleanliness based on the type and area of ​​the debris to be cleaned. The calculation model for road surface cleanliness AC is as follows: In the formula, α and β are the road surface cleanliness correction coefficients, C1 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned.

[0162] The cleaning power acquisition module is used to acquire the power of the exhaust fan, the second drive assembly, and the water pump based on the road surface cleanliness. The power is a pre-set value based on the road surface cleanliness.

[0163] The execution module is used to control the exhaust fan, the second drive assembly, and the water pump to operate at power when the area to be cleaned enters the bottom of the vehicle body.

[0164] Based on the above embodiments, in an optional embodiment of the present invention, the vehicle front view acquisition module includes:

[0165] The front view acquisition unit is used to acquire images captured by the front camera.

[0166] The front camera image preprocessing unit is used to perform grayscale conversion, geometric transformation, and image enhancement on the image captured by the front camera to obtain the preprocessed image captured by the front camera.

[0167] The front view segmentation module is specifically used to: obtain the area to be cleaned based on the pre-processed image captured by the front camera and an image semantic segmentation algorithm based on the FCN-8S network structure.

[0168] The vehicle-front image recognition module is specifically used to: based on the area to be cleaned, obtain the category and area of ​​the debris to be cleaned using a target detection algorithm based on a YOLOv7 network structure. The area of ​​the debris to be cleaned is the width multiplied by the height of the target bounding box.

[0169] Based on the above embodiments, in an optional embodiment of the present invention,

[0170] The cleaning power acquisition module is specifically used to acquire the power of the exhaust fan, the first drive component, the second drive component, and the water pump based on the road surface cleanliness.

[0171] The execution module is specifically used to control the operation of the exhaust fan, the first drive component, the second drive component, and the water pump according to the power when the area to be cleaned enters the bottom of the vehicle body.

[0172] Based on the above embodiments, in an optional embodiment of the present invention, the control device further includes:

[0173] The joint calibration module is used to jointly calibrate the image data of the area to be cleaned and the radar detection data to obtain the location information of the garbage to be cleaned in the area.

[0174] The distance acquisition module is used to obtain the distance between the garbage to be cleaned and the vehicle body based on the location information.

[0175] The movement speed acquisition module is used to acquire the movement speed of the vehicle body.

[0176] The movement time acquisition module is used to obtain the time it takes for the garbage to be cleaned to pass under the vehicle body based on the distance and movement speed.

[0177] The execution module is specifically used to control the operation of the exhaust fan, the second drive component, and the water pump based on the time it takes for the garbage to be cleaned to pass under the vehicle body and the power.

[0178] Based on the above embodiments, in an optional embodiment of the present invention, the control device further includes:

[0179] The rear view acquisition unit is used to acquire the image captured by the rear camera and perform preprocessing.

[0180] The rear-view image matching unit is used to obtain the image of the area to be cleaned after cleaning by using an image matching algorithm based on the pre-processed image captured by the rear camera.

[0181] The rear-view image recognition unit is used to identify the types of uncleaned garbage based on the image of the area to be cleaned after cleaning, using a target detection algorithm.

[0182] The weight correction unit is used to correct the weight of the type of uncleaned garbage based on its type. The correction includes: the weight value of the type of uncleaned garbage + 1.

[0183] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0184] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0185] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0186] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0187] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0188] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0189] The use of "first" and "second" in the embodiments is merely to distinguish similar objects and does not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0190] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent road sweeper, characterized in that, Include: Vehicle body; The vacuuming device (5) includes a trash can (6) disposed on the vehicle body, a suction nozzle and an exhaust fan; the suction nozzle is located below the vehicle body; the exhaust fan is connected to the suction nozzle and the trash can (6) and is configured to suck an object in front of the suction nozzle into the trash can (6); The cleaning device (1) includes a disc brush, a first drive assembly, and a second drive assembly disposed on the vehicle body; the disc brush is located on both sides of the bottom of the vehicle body; the first drive assembly is configured to drive the disc brush to rise and fall; the second drive assembly is configured to drive the disc brush to rotate. A flushing device includes a water tank, a nozzle, and a water pump disposed on the vehicle body; the nozzle is located below the vehicle body; the water pump is connected to the water tank and the nozzle and is configured to spray water from the water tank onto the road surface through the nozzle; The detection device includes a front camera (2) disposed on the vehicle body; the front camera (2) is configured to capture images of the road surface in front of the vehicle body; A control device (7) is electrically connected to the front camera (2), the exhaust fan, the first drive assembly, the second drive assembly, and the water pump; the control device (7) includes a processor and a memory; the processor is configured to execute a computer program stored in the memory to achieve: Acquire the image captured by the front camera (2) and perform preprocessing; Based on the pre-processed image captured by the front camera (2), the area to be cleaned is obtained through an image segmentation algorithm; Based on the area to be cleaned, the type and area of ​​the garbage to be cleaned are obtained through a target detection algorithm; The road surface cleanliness is obtained based on the type and area of ​​the garbage to be cleaned; wherein, the road surface cleanliness... The calculation model is as follows: In the formula, and C1 is the road surface cleanliness correction coefficient, C2 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned. The power of the exhaust fan, the second drive assembly, and the water pump is obtained based on the road surface cleanliness; wherein the power is a value preset based on the road surface cleanliness. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive assembly, and the water pump are controlled to operate according to the power. The detection device also includes a rear camera (3) disposed on the vehicle body; the rear camera (3) is configured to capture images of the road surface behind the vehicle body; When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive assembly, and the water pump are operated according to the power control, and then the process further includes: Acquire the image captured by the rear camera (3) and perform preprocessing; Based on the pre-processed image captured by the rear camera (3), the image of the area to be cleaned after cleaning is obtained through an image matching algorithm; Based on the images of the area to be cleaned after cleaning, the types of uncleaned garbage are obtained through a target detection algorithm; Based on the types of uncleaned garbage, the weights of the types of uncleaned garbage are adjusted; wherein, adjusting the weights of the types of uncleaned garbage includes: the weight value of the type of uncleaned garbage + 1.

2. The intelligent road sweeper according to claim 1, characterized in that, The sweeping device (1) also includes a pressure sensor; the sensor is configured to detect the pressure of the disc brush on the road surface; The power of the exhaust fan, the second drive assembly, and the water pump is obtained based on the road surface cleanliness, specifically including: The power of the exhaust fan, the first drive assembly, the second drive assembly, and the water pump is obtained based on the road surface cleanliness. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive assembly, and the water pump are controlled to operate according to the power, specifically including: When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the first drive component, the second drive component, and the water pump are operated according to the power control.

3. The intelligent road sweeper according to claim 1, characterized in that, The initial weights for each type of waste are as follows: leaves and cigarette butts: 1; paper towels: 2; garbage bags: 3; soil: 4; and stones:

5. The weights of all trash in the area to be cleaned The calculation model is as follows: In the formula, The number of pieces of trash detected in the area to be cleaned, Let n be the weight of the nth garbage item; The ratio of the area of ​​all trash in the area to be cleaned to the area of ​​the area to be cleaned. The calculation model is as follows: In the formula, The number of pieces of trash detected in the area to be cleaned, The width of the target detection box representing the object, S represents the height of the object detection bounding box, and S represents the area of ​​the cleaning area. The relationship between the power and the road surface cleanliness is as follows: When the road surface cleanliness is no greater than 50, the disc brush speed is 60 r / min, the exhaust fan speed is 1200 r / min, and the water pump is not working; When the road surface cleanliness is no more than 100, the disc brush speed is 80 r / min, the exhaust fan speed is 1500 r / min, and the water pump is not working; When the road surface cleanliness is no more than 150, the disc brush speed is 90 r / min, the exhaust fan speed is 2000 r / min, and the water pump working pressure is 5 MPa. When the road surface cleanliness is no greater than 200, the disc brush speed is 100 r / min, the exhaust fan speed is 2500 r / min, and the water pump working pressure is 8 MPa. When the road surface cleanliness is greater than 250, the disc brush speed is 120 r / min, the exhaust fan speed is 3000 r / min, and the water pump working pressure is 10 MPa.

4. An intelligent road sweeper according to any one of claims 1 to 3, characterized in that, The process of acquiring and preprocessing the images captured by the front camera (2) specifically includes: The image captured by the front camera (2) is obtained; The image captured by the front camera (2) is subjected to grayscale conversion, geometric transformation and image enhancement to obtain the pre-processed image captured by the front camera (2); The step of obtaining the area to be cleaned based on the pre-processed image captured by the front camera (2) using an image segmentation algorithm specifically includes: Based on the pre-processed images captured by the front camera (2), the area to be cleaned is obtained through an image semantic segmentation algorithm based on the FCN-8S network structure; The step of obtaining the type and area of ​​the debris to be cleaned based on the area to be cleaned using a target detection algorithm specifically includes: The method involves obtaining the category and area of ​​the garbage to be cleaned based on the area to be cleaned using a target detection algorithm based on the YOLOv7 network structure; wherein the area of ​​the garbage to be cleaned is the width multiplied by the height of the target bounding box.

5. An intelligent road sweeper according to any one of claims 1 to 3, characterized in that, The detection device also includes a radar (4) disposed on the vehicle body; the radar (4) is configured to detect the position of the garbage to be cleaned in front of the vehicle body; When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive assembly, and the water pump are operated according to the power control, and the process also includes: The image of the area to be cleaned and the detection data of the radar (4) are jointly calibrated to obtain the location information of the garbage to be cleaned in the area to be cleaned; Based on the location information, the distance between the garbage to be cleaned and the vehicle body is obtained; Obtain the moving speed of the vehicle body; Based on the distance and the moving speed, the time it takes for the garbage to be cleaned to pass under the vehicle body is obtained; When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive assembly, and the water pump are controlled to operate according to the power, specifically including: The operation of the exhaust fan, the second drive component, and the water pump is controlled based on the time it takes for the garbage to pass under the vehicle body and the power.

6. A control method for an intelligent road sweeper, characterized in that, Used to control an intelligent road sweeper as described in any one of claims 1 to 5; The control methods include: The image captured by the front camera is acquired and preprocessed; Based on the pre-processed images captured by the front camera, the area to be cleaned is obtained through image segmentation algorithms; Based on the area to be cleaned, the type and area of ​​the garbage to be cleaned are obtained through a target detection algorithm; The road surface cleanliness is obtained based on the type and area of ​​the garbage to be cleaned; wherein, the road surface cleanliness... The calculation model is as follows: In the formula, and C1 is the road surface cleanliness correction coefficient, C2 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned. The power of the exhaust fan, the second drive assembly, and the water pump is obtained based on the road surface cleanliness; wherein the power is a value preset based on the road surface cleanliness. When the area to be cleaned enters the bottom of the vehicle body, the exhaust fan, the second drive component, and the water pump are controlled to operate according to the power.

7. A control device for an intelligent road sweeper, characterized in that, Used to execute the control method for an intelligent road sweeper as described in claim 6; The control device includes: The front view acquisition module is used to acquire the image captured by the front camera and perform preprocessing. The front view segmentation module is used to obtain the area to be cleaned based on the pre-processed image captured by the front camera using an image segmentation algorithm. The vehicle front image recognition module is used to obtain the type and area of ​​the garbage to be cleaned based on the area to be cleaned through a target detection algorithm; A cleanliness calculation module is used to obtain the road surface cleanliness based on the type and area of ​​the garbage to be cleaned; wherein, the road surface cleanliness... The calculation model is as follows: In the formula, and C1 is the road surface cleanliness correction coefficient, C2 is the weighted sum of all garbage in the area to be cleaned, C2 is the ratio of the area of ​​all garbage in the area to be cleaned to the area of ​​the area to be cleaned, and S is the area of ​​the area to be cleaned. A cleaning power acquisition module is used to acquire the power of the exhaust fan, the second drive component, and the water pump based on the road surface cleanliness; wherein the power is a value preset based on the road surface cleanliness. The execution module is used to control the operation of the exhaust fan, the second drive component, and the water pump according to the power when the area to be cleaned enters the bottom of the vehicle body.

Images