Unmanned sweeper truck control method, unmanned sweeper truck, storage medium and program

Abstract

An unmanned sweeper truck control method, an unmanned sweeper truck, a storage medium, and a program. The method is applied to an unmanned sweeper truck, the unmanned sweeper truck at least comprising an image acquisition module and a garbage recognition module. The method comprises: by means of the image acquisition module, acquiring image information of an operation area of the unmanned sweeper truck (S110); and using the garbage recognition module to recognize a garbage condition in the image information and, on the basis of the garbage condition, adjusting the operation mode of the unmanned sweeper truck and the working state of a front baffle (S120).

Description

Unmanned sweeper control method, unmanned sweeper, storage medium and program

[0001] The present application claims priority to the Chinese patent application No. 202411356445.3 filed on September 26, 2024 with the China Patent Office, the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer application, for example, to an unmanned sweeper control method, an unmanned sweeper, a storage medium and a program. BACKGROUND

[0003] With the continuous improvement of technology maturity, unmanned sweepers integrated with advanced technologies such as high-definition cameras, laser types and ultrasonic sensors have realized high-precision positioning and obstacle sensing and avoidance, and have more stable performance. Therefore, unmanned sweepers have been widely used in municipal roads, campuses, communities, parks and scenic spots. Unmanned sweepers still have defects in some subtle scenarios. In the working mechanism of the unmanned sweeper, a front baffle is generally provided, which is generally in a closed state. When large objects such as mineral water bottles, cigarette boxes and branches are encountered during the cleaning process, the front baffle can be controlled to raise the front baffle, so that the objects with a larger size are sucked into the working mechanism of the unmanned sweeper to improve the cleaning effect.

[0004] However, the current front baffle control method is relatively simple, and the front baffle is generally controlled periodically. For example, the front baffle is raised for one minute, which can allow the large objects blocked in front of the front baffle to be processed by the working mechanism. However, this control method causes the front baffle to be raised periodically regardless of whether there are large objects in front of the front baffle of the unmanned sweeper, and the unmanned sweeper enters a high-power rolling sweeping mode after the front baffle is raised, resulting in unnecessary energy consumption and seriously restricting the working endurance of the unmanned sweeper. SUMMARY

[0005] The present application provides an unmanned sweeper control method, an unmanned sweeper, a storage medium and a program to solve the problem of inaccurate control of the baffle of the unmanned sweeper. The baffle of the unmanned sweeper can be controlled by the image of the road surface, the unmanned sweeper can be controlled to work as needed, the energy consumption of the unmanned sweeper can be reduced, and the working endurance time can be improved.

[0006] According to an aspect of the present application, an unmanned sweeper control method is provided, comprising:

[0007] acquiring image information of a working area of the unmanned sweeper by the image acquisition module;

[0008] The garbage recognition module is configured to recognize a garbage condition in the image information, and adjust a working mode and a front baffle working state of the unmanned sweeper according to the garbage condition.

[0009] According to another aspect of the present application, an unmanned sweeper is provided, which comprises at least an image acquisition module and a garbage recognition module; the image acquisition module is configured to acquire image information of a working area of the unmanned sweeper; the garbage recognition module is configured to recognize a garbage condition in the image information, and adjust a working mode and a front baffle working state of the unmanned sweeper according to the garbage condition.

[0010] According to another aspect of the present application, a computer readable storage medium is also provided, which stores computer instructions for causing a processor to implement the unmanned sweeper control method according to any of the embodiments of the present application.

[0011] According to another aspect of the present application, a computer program product is also provided, which comprises a computer program, and the computer program, when executed by a processor, implements the unmanned sweeper control method according to any of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0012] Fig. 1 is a flow chart of an unmanned sweeper control method according to an embodiment of the present application;

[0013] Fig. 2 is a flow chart of another unmanned sweeper control method according to another embodiment of the present application;

[0014] Fig. 3 is a flow chart of another unmanned sweeper control method according to another embodiment of the present application;

[0015] Fig. 4 is a working example diagram of an unmanned sweeper according to another embodiment of the present application;

[0016] Fig. 5 is a system architecture diagram of an unmanned sweeper according to another embodiment of the present application;

[0017] Fig. 6 is a structural schematic diagram of an unmanned sweeper control device according to another embodiment of the present application;

[0018] Fig. 7 is a structural schematic diagram of an electronic device implementing the unmanned sweeper control method according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, in addition to the processes, methods, systems, products or devices comprising a series of steps or units illustrated by the embodiments of the present application, other processes, methods, systems, products or devices not clearly listed in the series of steps or units, or other steps or units inherent to these processes, methods, systems, products or devices.

[0020] Embodiment one

[0021] Fig. 1 is a flowchart of a control method of an unmanned sweeper according to an embodiment of the present application. The embodiment can be applicable to the control of the unmanned sweeper for sweeping larger garbage. The method can be executed by the unmanned sweeper, which can be implemented in the form of hardware and / or software. The unmanned sweeper can at least include an image acquisition module and a garbage identification module. As shown in Fig. 1, the method includes:

[0022] S110, acquiring image information of a working area of the unmanned sweeper by the image acquisition module.

[0023] The image acquisition module can be a software and / or hardware module for acquiring image information in the unmanned sweeper. The image acquisition module can include a camera, a radar sensor, etc. The working area can be an area where the unmanned sweeper performs sweeping work. The working area can be determined by the sweeping task or the running route of the unmanned sweeper. It can be understood that the working area can include a ground area to be swept by the unmanned sweeper. The ground area can be a ground area configured in the working task of the unmanned sweeper, or it can be an area within a threshold range in front of the unmanned sweeper.

[0024] In the embodiment of the present application, the image information can be data acquired by the image acquisition module. The data format of the image information can include pictures or videos. The image information at least includes data corresponding to the working area. The unmanned sweeper can be configured with the image acquisition module. The image acquisition module can collect image information by real-time acquisition of the working area after the unmanned sweeper is powered on.

[0025] S120, identifying garbage conditions in the image information by using the garbage identification module, and adjusting the working mode of the unmanned sweeper and the working state of the front baffle according to the garbage conditions.

[0026] The garbage recognition module can be an image processing module for processing image information. The garbage recognition module can be implemented through an image recognition network, for example, through a YOLO network, a Single Shot MultiBox Detector (SSD) network, a Regions-Convolutional Neural Networks (R-CNN) network, or the like. The garbage recognition module can recognize garbage in the image information, thereby generating a garbage condition. The garbage condition can represent the presence of garbage in the image information, which can be divided according to whether garbage is present or not, or according to the amount of garbage present. That is, the garbage condition can include the presence of garbage, the absence of garbage, the amount of garbage being less than a threshold, or the amount of garbage being greater than a threshold, and the like. The operation mode is a mode in which the unmanned sweeper performs cleaning operations, including a high-power rolling sweeping mode and a standard-power sweeping mode, and the like. The front apron working state is information indicating the state of the front apron, including the front apron being in a raised state and not being in a raised state, and the like.

[0027] In the embodiments of the present application, the garbage recognition module can be pre-configured in the unmanned sweeper. When the unmanned sweeper is in a working state, the garbage recognition module is activated to process image information. The garbage recognition module can process image information in a timed or untimed manner. The garbage recognition module can identify garbage in the image information through an object detection algorithm, thereby generating a garbage condition. The unmanned sweeper can adjust the operation mode and the front apron working state according to the garbage condition identified in the image information, thereby increasing the operation power of the unmanned sweeper and raising the front apron when garbage is present in the operation area of the unmanned sweeper.

[0028] In the embodiments of the present application, the image acquisition module of the unmanned sweeper acquires image information in the operation area. The garbage recognition module of the unmanned sweeper identifies the garbage condition in the image information, and adjusts the operation mode and the front apron working state of the unmanned sweeper according to the garbage condition. This can adjust the operation mode and the front apron working state of the unmanned sweeper according to the actual garbage condition, accurately clean larger garbage, and reduce the energy consumption of the unmanned sweeper, thereby prolonging the working duration.

[0029] In some embodiments, the unmanned sweeper control method further includes identifying the height of the garbage in the image information through the garbage recognition module, and raising the front apron to a corresponding height according to the height of the garbage.

[0030] The garbage height can be height information corresponding to the garbage in the image information, and the garbage height can be determined by a pixel height in the image information. The image recognition model configured by the garbage recognition module can recognize the image information, so as to determine the garbage height.

[0031] In the embodiment of the application, the garbage recognition module can be called to recognize the image information, so as to obtain the garbage height of the garbage in the image information. The front baffle can be controlled by the garbage height, so as to be lifted to a height matched with the garbage height. The height matched with the garbage height at least includes a height at which the front baffle is lifted and is greater than or equal to the garbage height, so that the corresponding garbage in the image information can be swept by the unmanned sweeper.

[0032] Embodiment two

[0033] FIG. 2 is a flowchart of another method for controlling an unmanned sweeper according to an embodiment of the application. The embodiment of the application describes the process of collecting image information and the process of identifying garbage conditions. Referring to FIG. 2, the method provided by the embodiment of the application specifically includes the following steps:

[0034] S210, after the unmanned sweeper is powered on, starting the image collection module.

[0035] In the embodiment of the application, after the unmanned sweeper is powered on, the image collection module can be started, and the image collection module is switched to a working state.

[0036] S220, calling the image collection module to collect the working area in real time to obtain image information, wherein the working area at least includes the ground in the running route of the unmanned sweeper.

[0037] The running route can be a route in which the unmanned sweeper works, and the running route can be all or part of the working route of the unmanned sweeper. The working route can be pre-configured or automatically generated, and the working area can be the ground in the running route.

[0038] In the embodiment of the application, the image collection module can be installed on the unmanned sweeper, and the image collection angle of the image collection module can be configured to correspond to the ground in the running route. During the working process of the unmanned sweeper, the image collection module can be called to collect the image of the working area such as the ground in the running route, and the image information including the ground in the running route can be obtained.

[0039] S230, reading the image information collected by the image collection module through the garbage recognition module, wherein the garbage recognition module and the image collection module are connected through a serial data interface data line.

[0040] In the embodiment of the present application, the garbage recognition module can be connected to the image acquisition module through a serial data interface data line. When the image acquisition module acquires image information, the garbage recognition module can read the image information in the image acquisition module to perform object detection on the image information and determine the garbage condition in the image information.

[0041] In S240, a garbage recognition algorithm configured in the garbage recognition module is used to recognize garbage objects in the image information to obtain the garbage condition.

[0042] The garbage recognition module can include software and / or hardware devices, and the garbage recognition algorithm can be configured in the garbage recognition module to detect garbage objects in the image information. After the garbage recognition module acquires the image information, the garbage recognition algorithm can be loaded and called to recognize garbage objects in the image information, thereby obtaining the garbage condition.

[0043] In S250, the garbage recognition module adjusts the operation mode and the front baffle working state of the unmanned sweeper according to the garbage condition.

[0044] In the embodiment of the present application, the garbage recognition module can adjust the operation mode and the front baffle working state. The garbage recognition module can have different adjustment conditions. When the garbage condition recognized in the image information meets the corresponding adjustment condition in the garbage recognition module, the garbage recognition module can adjust the operation mode and the front baffle working state of the unmanned sweeper according to the adjustment condition. For example, the garbage recognition module can be configured with control instructions for adjusting the operation mode and the front baffle working state. The control instructions can be managed according to different garbage conditions. After determining the garbage condition, the garbage recognition module can trigger the control instruction associated with the garbage condition, thereby adjusting the operation mode and the front baffle working state of the unmanned sweeper.

[0045] In the embodiment of the present application, after the unmanned sweeper is powered on, the image acquisition module is started to collect the working area in real time to acquire image information. The garbage recognition module collects the image information collected in the image acquisition module, detects garbage objects in the image information according to the garbage recognition algorithm configured in the garbage recognition module, obtains the garbage condition, and controls the garbage recognition module to adjust the operation mode and the front baffle working state of the unmanned sweeper according to the garbage condition. The operation mode and the front baffle working state of the unmanned sweeper can be adjusted as needed based on the actual garbage condition. Large-sized garbage can be accurately swept. The energy consumption problem of the unmanned sweeper caused by lifting the front baffle and adjusting the operation mode according to a fixed period in the related art can be solved. The operation energy consumption of the unmanned sweeper can be reduced, thereby prolonging the working endurance time.

[0046] In some embodiments, the spam recognition algorithm comprises a pre-trained generated spam image recognition model, the spam image recognition model at least comprises a backbone network, a neck network and a head network, the backbone network comprises a convolution module, a convolution-to-feature module and a spatial pyramid pooling module, the neck network comprises a feature fusion module and a feature concatenation module, and the head network comprises a decoupled head structure, the decoupled head structure comprises a classification decoupled head and a detection decoupled head.

[0047] In the embodiments of the present application, the spam recognition algorithm at least comprises a pre-trained spam image recognition model, which can be composed of a backbone network, a neck network and a head network. The backbone network is mainly responsible for feature extraction of image information, and usually adopts a structure such as CSPDarknet. The CSPDarknet structure can pass through a residual structure and a cross-stage partial connection (CSP) structure. The backbone network can extract features of image information including a work area. The granularity of the extracted features can include edges, textures, shapes, etc.

[0048] The feature extraction of the backbone network can mainly depend on the convolution module and the convolution-to-feature module. The convolution module can be composed of a convolution kernel, a bias term and an activation function. The convolution kernel is the core element of the convolution operation, which is a small size matrix. By sliding on the input data, it performs weighted summation with the local region of the input data to extract specific features. The bias term can be a bias value added after the convolution operation, which can be used to adjust the overall offset of the output features. The activation function can be a non-linear factor introduced by the convolution module, which can improve the expression ability of the convolution module. The activation function can include ReLU, Tanh and Sigmoid, etc. The convolution-to-feature module can further process and transform the results of convolution into feature representations with specific semantic information based on a series of convolution operations. It can include connected convolution, feature fusion and feature transformation processes, etc. It can gradually transform low-level features obtained by convolution into high-level features with higher semantic information, so that the model can better understand the essential features of the input data and improve the performance and generalization ability of the model.

[0049] The neck network can serve as a connection between the backbone network and the head network. The neck network can realize multi-scale feature fusion. The neck network can be realized through a spatial pyramid pooling (SPP) module. SPP is mainly used to solve the problem of variable input image size, while extracting multi-scale feature information. The spatial pyramid pooling module can include multiple scale pooling layers, which can obtain feature representations of multiple scales. The different scale results are spliced through a splicing operation to obtain a fixed-length feature vector.

[0050] The head network can be mainly used for converting the feature maps of the backbone network and other feature maps into a detection result of the garbage object detection, which can include the category, confidence and bounding box coordinates of the garbage object, etc. The head network can be composed of a decoupled head structure, which includes a classification decoupled head and a detection decoupled head. The classification decoupled head can be used to predict the category probability of the garbage object, and the detection decoupled head is used to predict the bounding box coordinates of the garbage object.

[0051] In some embodiments, the garbage recognition algorithm configured by the garbage recognition module identifies the garbage object in the image information to obtain the garbage condition, including:

[0052] The image information is input into the garbage recognition algorithm through the garbage recognition module; the backbone network of the garbage recognition algorithm extracts feature images of at least one preset size from the image information; the neck network of the garbage recognition algorithm is called to splice the extracted multiple feature images, and the spliced feature images are fused with the original proportion feature map of the image information into a fusion feature map; the head network of the garbage recognition algorithm is used for prediction processing of the fusion feature map to obtain a preliminary prediction result; the confidence threshold of the garbage recognition algorithm is used to filter the preliminary prediction result, and the non-maximum suppression is performed on the preliminary prediction result to obtain the final detection result of the removal of overlapping detection boxes as the garbage condition.

[0053] The non-maximum suppression can be a post-processing technique for garbage object detection, which can be used to remove redundant detection boxes in the detection result. The multiple detection boxes in the preliminary prediction result can be arranged in descending order according to the corresponding confidence, and then the detection boxes are compared with each other from the detection box with the highest confidence, the overlapping degree of each detection box with other detection boxes is determined, and the confidence of the detection box with an overlapping degree greater than a threshold is set to 0, so as to achieve the effect of removing the overlapping detection boxes.

[0054] In the embodiments of the present application, after the image information is input into the garbage recognition module, the backbone network of the garbage recognition algorithm starts to work. The role of the backbone network is to extract the features of the image, which can reflect the shape, color, texture and other information of different objects in the image. The backbone network can extract at least one feature image of a preset size. Different sizes of feature images contain different levels of information, for example, a small size feature image may pay more attention to global semantic information, while a large size feature image may pay more attention to local detail information. For example, for image information of a working area including an unmanned sweeper, the backbone network may extract multiple feature images of different sizes, one of which may capture the general layout of the entire scene, and one of which may display the detailed features of a specific garbage item more clearly. After the backbone network determines the multi-size feature images, the neck network of the garbage recognition algorithm can be called to splice the multiple feature images extracted by the backbone network. This splicing operation can fuse feature images of different sizes together, thereby comprehensively utilizing feature information of different levels. The spliced feature image can be fused with the original scale feature image of the image information to form a fused feature image. The original scale feature image retains some important information of the original image, such as the overall structure and scale relationship. The expression of the features can be further enriched by fusing with the spliced feature image. The head network of the garbage recognition algorithm performs prediction processing on the fused feature image to obtain a preliminary prediction result. The head network usually includes two parts of classification and regression, which are respectively used to predict the category and position of the object in the image. The head network outputs the category (whether it is garbage), confidence and coordinate information of the bounding box of each detection box. For example, the head network may predict that there are several garbage items in the image information, the category of each garbage item (such as recyclable garbage, hazardous garbage, etc.), and their positions in the image. The result output by the head network can be used as the preliminary prediction result. Since there may be redundant detection boxes in the preliminary prediction result output by the head network, the preliminary prediction result can be subjected to non-maximum suppression to remove overlapping detection boxes. The purpose of non-maximum suppression is to eliminate redundant detection results and ensure that each object is detected only once. It compares the overlapping degree and confidence between detection boxes, retains the most likely detection box, suppresses other detection boxes with larger overlap and lower confidence, and the final detection result obtained after processing can be used as the garbage situation reflecting the image information, which can include the category of the garbage object, the detection box of the garbage object and the confidence of the garbage object, etc.

[0055] Embodiment three

[0056] FIG. 3 is a flow chart of another method for controlling the unmanned sweeper according to the third embodiment of the present application, which describes the image information acquisition process and the garbage condition identification process. Referring to FIG. 3, the method according to the third embodiment of the present application specifically includes the following steps:

[0057] S310, acquiring image information of the working area of the unmanned sweeper by the image acquisition module.

[0058] In the third embodiment of the present application, the image acquisition module can be used to acquire images of the working area, and the real-time acquired data can be used as the image information. The data format of the image information can include pictures or videos, and the image information at least includes data corresponding to the working area.

[0059] S320, identifying the garbage condition in the image information by the garbage identification module.

[0060] In the third embodiment of the present application, the garbage identification module can be preconfigured with a detection algorithm of garbage objects, and the garbage objects in the image information can be detected by the garbage identification module, so as to obtain the garbage condition. The garbage condition can reflect the existence of garbage in the working area of the unmanned sweeper, and the garbage condition can include existence of garbage or non-existence of garbage.

[0061] S330, generating an adjustment control instruction when the garbage condition is determined to be existence of garbage by the garbage identification module.

[0062] The adjustment control instruction can be an instruction for adjusting the working mode and the front baffle working state of the unmanned sweeper. The adjustment control instruction can be preconfigured in the garbage identification module, and the instruction content of the adjustment control instruction can include information for adjusting the working mode and the front baffle working state of the unmanned sweeper. Different garbage conditions can be associated with different adjustment control instructions, and the different adjustment control instructions can include different instruction contents.

[0063] In the third embodiment of the present application, the garbage identification module can be configured with different adjustment control instructions for different garbage conditions. When the garbage condition of the image information is identified to be existence of garbage, the garbage identification module can obtain the adjustment control instruction associated with the existence of garbage.

[0064] S340, transmitting the adjustment control instruction to the domain controller of the unmanned sweeper by the garbage identification module.

[0065] The domain controller is an important component of the unmanned sweeper for realizing the unmanned sweeping operation, and can be used to adjust the working mode and the front baffle working state of the unmanned sweeper. The number of the domain controllers can be one or more, and each domain controller can be responsible for one or more working functions of the unmanned sweeper.

[0066] In the embodiment of the present application, after the garbage identification module generates the adjustment control instruction, the adjustment control instruction can be transmitted to the domain controller of the unmanned sweeper, and the domain controller responds to the adjustment control instruction to make corresponding control operation.

[0067] S350, adjusting the working mode of the unmanned sweeper to the powerful mode and adjusting the working state of the front apron to the lifted working state through the domain controller.

[0068] The powerful mode is a mode in which the unmanned sweeper enters high-power operation. In the powerful mode, the unmanned sweeper can take measures such as increasing suction and improving cleaning intensity on the garbage in the working area.

[0069] In the embodiment of the present application, the unmanned sweeper can adjust the working mode through the domain controller and switch it to the powerful mode. At the same time, the working state of the front apron is adjusted through the domain controller, and the working state of the front apron of the unmanned sweeper is adjusted to the lifted working state, and then the front apron of the unmanned sweeper is lifted to facilitate the unmanned sweeper to suck the garbage into the vehicle.

[0070] In the embodiment of the present application, the image acquisition module is called to acquire the image information of the working area of the unmanned sweeper, the garbage identification module identifies the garbage in the image information, determines that the garbage situation is that there is garbage, generates an adjustment control instruction, and transmits the adjustment control instruction to the domain controller of the unmanned sweeper. The working mode is adjusted to the powerful mode and the working state of the front apron is adjusted to the lifted working state through the domain controller according to the adjustment control instruction. The working mode and the working state of the front apron of the unmanned sweeper can be adjusted according to the actual garbage situation, the garbage with large size can be accurately cleaned, the energy consumption problem of the unmanned sweeper caused by lifting the front apron and adjusting the working mode according to the fixed period in the related technology can be solved, the working energy consumption of the unmanned sweeper can be reduced, and the working endurance time can be improved.

[0071] In some embodiments, the unmanned sweeper control method further comprises:

[0072] If the garbage situation is determined by the garbage identification module to be that there is no garbage, the domain controller is not triggered.

[0073] In the embodiment of the present application, when the garbage identification module determines that the garbage situation in the image information is that there is no garbage, the domain controller can not be triggered, that is, the working mode and the working state of the front apron of the unmanned sweeper do not need to be adjusted.

[0074] Embodiment four

[0075] In the embodiment of the present application, referring to FIG. 4, the unmanned sweeper cleaning operation mechanism generally has a front baffle. The front baffle is generally in a closed state. When large objects such as mineral water bottles, cigarette boxes, branches and the like are encountered during the cleaning process, the front baffle can be raised to allow such large objects to be sucked into the operation mechanism, thereby achieving a better cleaning effect. However, the current control method is mainly periodical control, which leads to high power consumption in the control process and affects the operation time of the unmanned sweeper.

[0076] To solve this problem, referring to FIG. 5, the embodiment of the present application provides an unmanned sweeper control method. A camera is installed on the unmanned sweeper to collect video of the road in front of the unmanned sweeper. An artificial intelligence (AI) computing unit is installed in the unmanned sweeper. After obtaining real-time video information, the AI computing unit identifies whether there are large objects in front of the unmanned sweeper in the real-time video information through a garbage recognition algorithm. When the AI computing unit identifies a large object, the event can be sent to a vehicle domain controller, which controls the unmanned sweeper to raise the front baffle and change the cleaning mode. The embodiment of the present application can raise the front baffle on demand and change the cleaning power on demand, which can reduce energy consumption while ensuring operation effect, save the power of the unmanned sweeper, and prolong the operation endurance time of the unmanned sweeper.

[0077] Embodiment five

[0078] FIG. 6 is a structural schematic diagram of an unmanned sweeper according to the embodiment five of the present application. As shown in FIG. 6, the unmanned sweeper includes an image collection module 401 and a garbage recognition module 402.

[0079] The image collection module 401 is configured to collect image information of an operation area of the unmanned sweeper.

[0080] The garbage recognition module 402 is configured to identify garbage conditions in the image information and adjust the operation mode of the unmanned sweeper and the working state of the front baffle according to the garbage conditions.

[0081] In some embodiments, the image collection module 401 includes:

[0082] The starting unit is configured to start the image collection module after the unmanned sweeper is powered on.

[0083] The collection unit is configured to collect the operation area in real time to obtain the image information, wherein the operation area at least includes the ground in the running route of the unmanned sweeper.

[0084] In some embodiments, the garbage recognition module 402 includes:

[0085] The information reading unit is configured to read image information collected by the image collection module.

[0086] The garbage recognition unit is configured to recognize garbage objects in the image information by using a garbage recognition algorithm configured by the garbage recognition module, to obtain the garbage condition.

[0087] In some embodiments, the garbage recognition algorithm of the garbage recognition module includes a garbage image recognition model generated by pre-training, and the garbage image recognition model at least includes a backbone network, a neck network, and a head network.

[0088] In some embodiments, the garbage recognition unit is configured to:

[0089] input the image information into the garbage recognition algorithm;

[0090] extract feature images of at least one preset size of the image information by using the backbone network of the garbage recognition algorithm;

[0091] call the neck network of the garbage recognition algorithm to splice the extracted multiple feature images, and fuse the spliced feature images with the original proportion feature map of the image information into a fused feature map;

[0092] perform prediction processing on the fused feature map by using the head network of the garbage recognition algorithm, to obtain a preliminary prediction result;

[0093] filter the preliminary prediction result by using a confidence threshold of the garbage recognition algorithm, and perform non-maximum suppression on the preliminary prediction result, to obtain a final detection result with overlapping detection boxes removed as the garbage condition.

[0094] In some embodiments, the garbage recognition module 402 is configured to:

[0095] generate an adjustment control instruction in response to the garbage condition being that garbage exists;

[0096] transmit the adjustment control instruction to the domain controller of the unmanned sweeper;

[0097] The domain controller is configured to adjust the operation mode of the unmanned sweeper to a forced mode and adjust the working state of the front apron to a raised working state.

[0098] In some embodiments, the garbage recognition module is further configured to not trigger the domain controller in response to the garbage condition including that garbage does not exist.

[0099] In some embodiments, the garbage recognition module 402 is further configured to recognize a garbage height in the image information, and control the front baffle to be lifted to a corresponding height according to the garbage height.

[0100] The unmanned sweeper provided by the embodiments of the present application can execute the unmanned sweeper control method provided by any of the embodiments of the present application, has the function modules and effects corresponding to the execution method.

[0101] Embodiment six

[0102] FIG. 7 is a structural schematic diagram of an electronic device implementing the unmanned sweeper control method according to the embodiments of the present application. The electronic device can be various forms of digital computers, such as a laptop computer, a desktop computer, a workstation, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also be various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.), and other similar computing devices. The components shown herein, their connections, and relationships, and their functions, are shown by way of example only and are not meant to limit the implementations of the present application described and / or claimed herein.

[0103] As shown in FIG. 7, the electronic device 10 includes at least one processor 11, and a memory, such as a Read-Only Memory (ROM) 12, a Random Access Memory (RAM) 13, etc., which is in communication with the at least one processor 11, wherein the memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the Read-Only Memory (ROM) 12 or loaded from the storage unit 18 into the Random Access Memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An Input / Output (I / O) interface 15 is also connected to the bus 14.

[0104] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, a loudspeaker, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0105] The processor 11 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphic processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the unmanned sweeper control method.

[0106] In some embodiments, the unmanned sweeper control method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the unmanned sweeper control method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the unmanned sweeper control method by any other suitable means, such as by means of firmware.

[0107] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0108] Computer programs for implementing the methods of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / acts specified in the flow diagrams and / or block diagrams to be implemented. The computer programs can be executed in whole on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0109] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. The machine-readable signal medium can include a wired electrical connection, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), or a flash memory, a fiber optic, a compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0110] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a Cathode Ray Tube (CRT) or a Liquid Crystal Display (LCD) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0111] The systems and techniques described herein can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein, or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0112] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS) services.

[0113] It should be understood that the steps shown in the above forms can be reordered, added, or deleted. For example, the steps described in the present application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions of the present application can be achieved, and the present application does not limit herein.

Claims

1. A method for controlling an unmanned sweeper, applied to an unmanned sweeper, the unmanned sweeper comprising at least an image acquisition module and a garbage recognition module, the method comprising: acquiring image information of a working area of the unmanned sweeper by the image acquisition module; recognizing a garbage condition in the image information by the garbage recognition module, and adjusting a working mode of the unmanned sweeper and a front apron working state according to the garbage condition. The acquiring image information of a working area of the unmanned sweeper by the image acquisition module comprises: starting the image acquisition module after the unmanned sweeper is powered on; calling the image acquisition module to collect the working area in real time to obtain the image information, wherein the working area at least includes the ground in the running route of the unmanned sweeper. The recognizing a garbage condition in the image information by the garbage recognition module comprises: reading the image information collected by the image acquisition module by the garbage recognition module, wherein the garbage recognition module and the image acquisition module are connected through a serial data interface data line; recognizing garbage objects in the image information by using a garbage recognition algorithm configured by the garbage recognition module to obtain the garbage condition.

2. The method of claim 1, wherein, The garbage recognition algorithm comprises a garbage image recognition model generated by pre-training, the garbage image recognition model at least comprises a backbone network, a neck network and a head network, the backbone network comprises a convolution module and a convolution-to-feature module, the neck network comprises a feature fusion module, the head network comprises a decoupling head structure, the decoupling head structure comprises a classification decoupling head and a detection decoupling head. The recognizing garbage objects in the image information by using a garbage recognition algorithm configured by the garbage recognition module to obtain the garbage condition comprises: inputting the image information into the garbage recognition algorithm by the garbage recognition module; extracting at least one feature image of a preset size of the image information by using the backbone network of the garbage recognition algorithm; splicing the extracted multiple feature images by calling the neck network of the garbage recognition algorithm, and fusing the spliced feature image with the original proportion feature map of the image information into a fusion feature map; obtaining a preliminary prediction result by performing prediction processing on the fusion feature map by the head network of the garbage recognition algorithm; filtering the preliminary prediction result by using a confidence threshold of the garbage recognition algorithm, and performing non-maximum suppression on the preliminary prediction result to obtain a final detection result without overlapping bounding boxes as the garbage condition. The unmanned sweeper further comprises a domain controller, and the adjusting a working mode of the unmanned sweeper and a front apron working state according to the garbage condition comprises: generating an adjustment control instruction in response to the garbage condition being garbage; transmitting the adjustment control instruction to the domain controller of the unmanned sweeper by the garbage recognition module; adjusting the working mode of the unmanned sweeper to a powerful mode and adjusting the front apron working state to a raised working state by the domain controller.

3. The method of claim 1, wherein, 7. The method of claim 1, further comprising: ​ ​ 4. The method of claim 3, wherein, ​ 5. The method of claim 3, wherein, ​ ​ ​ ​ ​ ​ 6. The method of claim 1, wherein, ​ ​ ​ ​ ​ The garbage recognition module recognizes the garbage height in the image information, and controls the front baffle to be lifted to a corresponding height according to the garbage height.

8. An unmanned sweeper vehicle comprising at least: An image acquisition module and a garbage recognition module; The image acquisition module is configured to acquire image information of a working area of the unmanned sweeper. The garbage recognition module is configured to recognize a garbage condition in the image information, and adjust a working mode of the unmanned sweeper and a working state of the front baffle according to the garbage condition. 9.A computer readable storage medium, storing computer instructions, which, when executed by a processor, implement the unmanned sweeper control method of any one of claims 1-7. 10.A computer program product, comprising a computer program, which, when executed by a processor, implements the unmanned sweeper control method of any one of claims 1-7.

Images