Unmanned sweeper control method, unmanned sweeper, storage medium, and program

Abstract

The invention discloses an unmanned sweeper control method, an unmanned sweeper, a storage medium and a program.The method is applied to the unmanned sweeper, the unmanned sweeper at least comprises an image acquisition module and a garbage recognition module, and the method comprises the steps that image information of an operation area of the unmanned sweeper is acquired through the image acquisition module; and the garbage condition in the image information is recognized through a garbage recognition module, and the working mode of the unmanned sweeper and the working state of a front baffle are adjusted according to the garbage condition. According to the embodiment of the invention, the problem of inaccurate control of the baffle plate of the unmanned sweeper can be solved, the baffle plate of the unmanned sweeper can be controlled through the road surface image, the unmanned sweeper is controlled to work as required, the action energy consumption of the unmanned sweeper can be reduced, and the working endurance time can be prolonged.

Description

Technical Field

[0001] The present invention relates to the field of computer application technology, and in particular to an unmanned sweeper control method, an unmanned sweeper, a storage medium and a program. Background Art

[0002] As the technology maturity of unmanned road sweepers continues to improve, unmanned road sweepers that integrate advanced technologies such as high-definition cameras, laser types, and ultrasonic sensors have achieved high-precision positioning and perception and obstacle avoidance, and their performance has become more stable. Therefore, unmanned road sweepers have been widely used in municipal roads, campuses, communities, parks, scenic spots and other scenes. Unmanned road sweepers still have defects in some subtle scenes. In the operating mechanism of unmanned road sweepers, there is generally a front baffle, which is generally in a closed state. When encountering large objects such as mineral water bottles, cigarette boxes, and branches during the cleaning process, the front baffle can be controlled to raise the front baffle, so that these large objects can be sucked into the operating mechanism of the unmanned road sweeper to improve the cleaning effect.

[0003] However, the current front baffle control method is relatively simple, and generally controls the opening and closing of the front baffle periodically, such as opening and raising the front baffle once a minute, so that larger objects outside the front baffle can be handled by the operating mechanism. However, this control method causes the front baffle to be raised periodically regardless of whether there is a large object blocking the front baffle in front of the unmanned sweeper, and the unmanned sweeper enters a high-power rolling sweeping mode after the front baffle is raised, resulting in unnecessary energy consumption and seriously restricting the operation endurance of the unmanned sweeper. Summary of the invention

[0004] The present invention provides an unmanned sweeper control method, an unmanned sweeper, a storage medium and a program to solve the problem of inaccurate control of the baffle of the unmanned sweeper. The baffle of the unmanned sweeper can be controlled through a road surface image, and the unmanned sweeper can be controlled to operate on demand, which can reduce the energy consumption of the unmanned sweeper and increase the working endurance time.

[0005] According to one aspect of the present invention, a control method for an unmanned sweeper is provided, wherein the method comprises:

[0006] Image information of the working area of ​​the unmanned sweeper obtained by the image acquisition module;

[0007] The garbage identification module is used to identify the garbage situation in the image information, and the operation mode and the front baffle working state of the unmanned sweeper are adjusted according to the garbage situation.

[0008] According to another aspect of the present invention, an unmanned sweeper is provided, which comprises at least: an image acquisition module and a garbage recognition module, wherein the image acquisition module acquires image information of an operating area of ​​the unmanned sweeper; the garbage recognition module recognizes the garbage situation in the image information, and adjusts the operating mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation.

[0009] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the unmanned sweeper control method described in any embodiment of the present invention when executed.

[0010] According to another aspect of the present invention, a computer program product is provided. The computer program product includes a computing program. When the computer program is executed by a processor, the unmanned sweeper control method as described in any embodiment of the present invention is implemented.

[0011] The technical solution of the embodiment of the present invention collects image information in the working area through the image acquisition module of the unmanned sweeper, identifies the garbage situation in the image information based on the garbage recognition module of the unmanned sweeper, and the garbage recognition module adjusts the action mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation. The unmanned sweeper can adjust the working mode and the working state of the front baffle as needed based on the actual garbage presence, and can accurately clean larger garbage. It can solve the energy consumption problem of the existing unmanned sweeper caused by lifting the front baffle and adjusting the working mode according to a fixed period, and can reduce the operating energy consumption of the unmanned sweeper, thereby improving the working endurance time.

[0012] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0014] Figure 1 is a flow chart of a control method of an unmanned sweeper provided according to Embodiment 1 of the present invention;

[0015] Figure 2 is a flow chart of another unmanned sweeper control method provided according to the second embodiment of the present invention;

[0016] Figure 3 is a flow chart of another unmanned sweeper control method provided according to Embodiment 3 of the present invention;

[0017] Figure 4 This is an example diagram of another operation of an unmanned sweeper provided according to the fourth embodiment of the present invention;

[0018] Figure 5 This is a system architecture diagram of an unmanned sweeper provided according to Embodiment 4 of the present invention;

[0019] Figure 6 is a structural schematic diagram of an unmanned sweeper control device provided according to Embodiment 5 of the present invention;

[0020] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the unmanned sweeper control method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0023] Embodiment 1

[0024] Figure 1This is a flow chart of a control method for an unmanned sweeper according to the first embodiment of the present invention. This embodiment is applicable to the case where an unmanned sweeper is used to control the cleaning of larger garbage. The method can be executed by an unmanned sweeper, which can be implemented in the form of hardware and / or software. The unmanned sweeper can at least include an image acquisition module and a garbage recognition module. Figure 1 As shown, the method includes:

[0025] Step 110: Obtain image information of the working area of ​​the unmanned sweeper through an image acquisition module.

[0026] The image acquisition module may be a software and / or hardware module for collecting image information in the unmanned sweeper, and the image acquisition module may include a camera, a radar sensor, etc. The operation area may be an area where the unmanned sweeper performs cleaning operations, and the operation area may be determined by the cleaning task or operation route of the unmanned sweeper. It is understandable that the operation area may include the ground area that the unmanned sweeper is about to clean, and the ground area may be the ground area configured in the operation task of the unmanned sweeper, or may also be the area within the threshold range in front of the unmanned sweeper.

[0027] In an embodiment of the present invention, the image information may be data collected by an image acquisition module. The data format of the image information may include pictures or videos. The image information at least includes data corresponding to the working area. Specifically, the unmanned sweeper may be equipped with an image acquisition module. The image acquisition module may collect image information in real time from the working area after the unmanned sweeper is powered on.

[0028] Step 120: Use the garbage recognition module to identify the garbage situation in the image information, and adjust the operation mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation.

[0029] Among them, the garbage recognition module can be an image processing module that processes image information. The garbage recognition module can be implemented through an image recognition network, for example, it can be implemented through a YOLO network, an SSD network, an R-CNN network, etc. The module can identify the garbage in the image information, thereby generating a garbage situation. The garbage situation can indicate the presence of garbage in the image information, and can be divided based on whether garbage exists, or based on the amount of garbage. In other words, the garbage situation may include situations such as the presence of garbage, the absence of garbage, the amount of garbage is less than a threshold, or the amount of garbage is greater than a threshold. The operating mode is the mode in which the unmanned sweeper performs cleaning operations, including a high-power rolling sweeping mode and a standard power sweeping mode. The front baffle working status is information that identifies the state of the front baffle, including whether the front baffle is in a raised state or not raised state.

[0030] In an embodiment of the present invention, a garbage recognition module can be pre-configured in an unmanned sweeper. When the unmanned sweeper is in working condition, the garbage recognition module is called to process image information. The garbage recognition model can process the image information in a regular or irregular manner. The garbage in the image information can be identified by the object detection algorithm in the garbage recognition model, thereby generating a garbage situation. The unmanned sweeper can adjust the operating mode and the working state of the front baffle according to the garbage situation identified in the image information, thereby enhancing the operating power of the unmanned sweeper and lifting the working state of the front baffle when there is garbage in the operating area of ​​the unmanned sweeper.

[0031] In the embodiments of the present invention, the image information in the working area is collected by the image acquisition module of the unmanned sweeper, the garbage recognition module of the unmanned sweeper recognizes the garbage situation in the image information, and the garbage recognition module adjusts the action mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation. The unmanned sweeper can adjust the working mode and the working state of the front baffle as needed based on the actual garbage presence, and can accurately clean larger garbage. The energy consumption problem of the existing unmanned sweeper caused by lifting the front baffle and adjusting the working mode according to a fixed period can be solved, and the operating energy consumption of the unmanned sweeper can be reduced, thereby improving the working endurance time.

[0032] Furthermore, based on the above-mentioned embodiments of the invention, it also includes: identifying the height of garbage in the image information through a garbage identification module, and controlling the front baffle to be lifted to a corresponding height according to the garbage height.

[0033] The garbage height may be height information corresponding to the garbage in the image information, and the garbage height may be determined by the pixel height in the image information. The image recognition model configured by the garbage recognition module may recognize the image information, thereby determining the garbage height.

[0034] In an embodiment of the present invention, a garbage recognition model can be called to identify and process the image information, thereby obtaining the garbage height of the garbage in the image information. The front baffle can be controlled by the garbage height to be lifted to a height that matches the garbage height. The match at least includes that the height of the front baffle is greater than or equal to the garbage height, so that the unmanned sweeper can clean the corresponding garbage in the image information.

[0035] Embodiment 2

[0036] Figure 2 is a flow chart of another unmanned sweeper control method provided according to the second embodiment of the present invention. The embodiment of the present invention is a specific embodiment of the above invention, and describes the image information collection process and the garbage situation recognition process. Figure 2 The method provided in the embodiment of the present invention specifically includes the following steps:

[0037] Step 210: After the unmanned cleaning vehicle is powered on, the image acquisition module is started.

[0038] In the embodiment of the present invention, after the unmanned sweeper is started and powered on, the image acquisition module can be started and switched to a working state.

[0039] Step 220: Call the image acquisition module to perform real-time acquisition of the working area to obtain image information, wherein the working area at least includes the ground in the operating route of the unmanned sweeper.

[0040] The operation route may be a route along which the unmanned sweeper operates, the operation route may be a full or partial operation video of the unmanned sweeper, the operation route may be pre-configured or automatically generated, and the operation area may be the ground of the operation route.

[0041] In an embodiment of the present invention, an image acquisition module can be installed on an unmanned sweeper, and the image acquisition angle of the image acquisition module can be configured to correspond to the ground in the operating route. During the operation process, the unmanned sweeper can call the image acquisition module to perform image acquisition on the ground in the operating route and other working areas, and can obtain image information including the ground in the operating route.

[0042] Step 230: Read the image information collected by the image acquisition device through the garbage recognition module, wherein the garbage recognition module is connected to the image acquisition device through a serial data interface data line.

[0043] In an embodiment of the present invention, the garbage identification module can be connected to the image acquisition device via a serial data interface data line. When the image acquisition device acquires image information, the garbage identification module can read the image information in the image acquisition device so as to perform object detection on the image information and determine the garbage situation in the image information.

[0044] Step 240: Use the garbage recognition algorithm configured in the garbage recognition module to identify garbage objects in the image information to obtain garbage conditions.

[0045] Specifically, the garbage identification module may include software and / or hardware devices, and a garbage identification algorithm may be configured in the garbage identification module, and the garbage identification algorithm may detect garbage objects in the object information. After acquiring the image information, the garbage identification module may load the garbage identification algorithm and call the garbage identification algorithm to identify garbage objects in the image information, thereby obtaining the garbage situation.

[0046] Step 250: Use the garbage identification module to adjust the operation mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation.

[0047] In an embodiment of the present invention, the garbage identification module can adjust the operation mode and the working state of the front baffle, and the garbage identification module can have different adjustment conditions. When the garbage situation identified in the image information meets the corresponding adjustment conditions in the garbage identification module, the garbage identification module can adjust the operation mode and the working state of the front baffle of the unmanned sweeper according to the adjustment conditions. Exemplarily, the garbage identification module can be configured with a control instruction for adjusting the operation mode and the working state of the front baffle, and the control instruction can be managed for different garbage situations. After determining the garbage situation, the garbage identification module can trigger the control instruction associated with the garbage situation, thereby realizing the adjustment of the operation mode and the working state of the front baffle of the unmanned sweeper.

[0048] In the embodiment of the present invention, after the unmanned sweeper is powered on, the image acquisition module is started to perform real-time acquisition of the working area to obtain image information, the garbage recognition module acquires the image information collected in the image acquisition module, and detects garbage objects in the image information according to the garbage recognition algorithm configured by the garbage recognition module to obtain the garbage situation, and controls the garbage recognition module to adjust the working mode and the working state of the front baffle of the unmanned sweeper according to the garbage situation. The unmanned sweeper can adjust the working mode and the working state of the front baffle as needed based on the actual garbage presence, and can accurately clean larger garbage. The energy consumption problem of the existing unmanned sweeper caused by lifting the front baffle and adjusting the working mode according to a fixed period can be solved, and the operating energy consumption of the unmanned sweeper can be reduced, thereby improving the working endurance time.

[0049] Furthermore, based on the above-mentioned embodiments of the invention, the garbage recognition algorithm includes a pre-trained garbage image recognition model, the garbage image recognition model includes at least a backbone network, a neck network and a head network, the backbone network includes a convolution module, a convolution to feature module and a spatial pyramid pooling module, the neck network includes a feature fusion module and a feature splicing module, the head network includes a decoupling head structure, and the decoupling head structure includes a classification decoupling head and a detection decoupling head.

[0050] In an embodiment of the present invention, the garbage recognition algorithm at least includes a pre-trained garbage image recognition model, which can be composed of a backbone network, a neck network and a head network. The backbone network is mainly responsible for feature extraction of image information, and usually adopts structures such as CSPDarknet. The CSPDarknet structure can use a residual structure and a CSP structure. The backbone network can extract features from image information including the working area, and the granularity of the extracted features can specifically include edges, textures, shapes, etc.

[0051] The backbone network can be mainly based on the convolution module and the convolution to feature module for feature extraction. The convolution module can be composed of a convolution kernel, a bias term and an activation function. The convolution kernel is the core element of the convolution operation. It is a small-sized matrix that extracts specific features by sliding on the input data and performing weighted summation with the local area of ​​the input data; the bias term can be the bias value added after the convolution operation, which can be used to adjust the overall offset of the output feature, and the activation function can be a nonlinear factor introduced by the convolution module to improve the expression ability of the convolution module. The activation function can include ReLU, Tanh and Sigmoid. The convolution to feature module can be based on a series of convolution operations, and further process and transform the convolution results into feature representations with specific semantic information. It can include processes such as convolution, feature fusion and feature transformation. The low-level features obtained by the convolution operation can be gradually transformed into high-level features with higher semantic information, so that the model can better understand the essential characteristics of the input data and improve the performance and generalization ability of the model.

[0052] The neck network can play the role of connecting the backbone network and the head network. The neck network can realize multi-scale feature fusion. The neck network can be realized through the spatial pyramid pooling (SPP) module. SPP is mainly used to solve the problem of non-fixed input image size and extract multi-scale feature information at the same time. The spatial pyramid pooling module can include pooling layers of multiple scales, which can obtain feature representations of multiple scales, and splice the size results of different scales through splicing operations to obtain a feature vector of fixed length.

[0053] The head network can be mainly used to convert other feature maps of the backbone network features into detection results of garbage object detection, which may include the category, confidence, and bounding box coordinates of the garbage object. The head network may include a decoupling head structure, and the decoupling head structure includes a classification decoupling head and a detection decoupling head. The classification decoupling head can be used to predict the category probability of the garbage object, and the detection decoupling head is used to predict the bounding box coordinates of the garbage object.

[0054] Further, based on the above-mentioned embodiment of the invention, the garbage identification algorithm configured by the garbage identification module is used to identify garbage objects in the image information to obtain the garbage situation, including:

[0055] The image information is input into the garbage recognition algorithm through the garbage recognition module; the backbone network of the garbage recognition algorithm is used to extract at least one feature image of a preset size from the image information; the neck network of the garbage recognition algorithm is called to splice the feature images, and the spliced ​​feature images are fused with the original scale feature map of the image information into a fused feature map; the fused feature map is predicted and processed by the head network of the garbage recognition algorithm to obtain a preliminary prediction result; the preliminary prediction result is filtered using the confidence threshold of the garbage recognition algorithm, and the preliminary prediction result is non-maximum suppression is performed on the preliminary prediction result to obtain the final detection result with the overlapping detection frames removed as the garbage situation.

[0056] Among them, non-maximum suppression can be a post-processing technology for junk object detection, which can be used to remove redundant detection frames in the detection results. The detection frames in the preliminary prediction results can be arranged in descending order according to the corresponding confidence levels, and then the detection frames with the highest confidence levels are compared with other detection frames in turn to determine the degree of overlap between each detection frame and other detection frames, and the confidence levels of detection frames with overlap levels greater than a threshold are set to 0, thereby achieving the effect of removing overlapping detection frames.

[0057] In an embodiment of the present invention, after the image information is input into the garbage identification module, the backbone network of the garbage identification algorithm starts to work. The function of the backbone network is to extract the features of the image, which can reflect the shape, color, texture and other information of different objects in the image. The backbone network can extract at least one feature image of a preset size. Feature images of different sizes contain information at different levels. For example, feature images of small sizes may pay more attention to global semantic information, while feature images of large sizes may pay more attention to local detail information. For example, for image information of the working area including an unmanned sweeper, the backbone network may extract multiple feature images of different sizes, wherein a feature image of a smaller size may capture the general layout of the entire scene, while a feature image of a larger size may more clearly show the detail features of a specific garbage item. After the backbone network determines the feature images of multiple sizes, the neck network of the garbage identification algorithm can be called to splice the feature images extracted by the backbone network. This splicing operation can fuse feature images of different sizes together, thereby comprehensively utilizing feature information of different levels. The spliced ​​feature image can be fused with the original proportional feature map of the image information to form a fused feature map. The original scale feature map retains some important information of the original image, such as the overall structure and proportional relationship. It can be further enriched by fusing with the spliced ​​feature image. The head network of the garbage recognition algorithm predicts the fused feature map to obtain a preliminary prediction result. The head network usually includes two parts, classification and regression, which are used to predict the category and position of objects in the image respectively. The head network outputs the category (whether it is garbage), confidence, and coordinate information of the bounding box of each detection box. For example, the head network may predict how many garbage items there are in the image information, what the category of each garbage item is (such as recyclable garbage, hazardous garbage, etc.), and their location in the image. The result output by the head network can be used as the preliminary prediction result. Since there may be redundant detection boxes in the preliminary prediction results output by the head network, non-maximum suppression can be performed on the preliminary prediction results to remove overlapping detection boxes. The purpose of non-maximum suppression is to eliminate redundant detection results and ensure that each object is detected only once. It compares the overlap and confidence between detection frames, retains the most likely detection frame, and suppresses other detection frames with large overlap and low confidence. The final detection result obtained after processing can be used as a reflection of the garbage situation of the image information, which can include information such as the category of the garbage object, the detection frame of the garbage object, and the confidence of the garbage object.

[0058] Embodiment 3

[0059] Figure 3is a flow chart of another unmanned sweeper control method provided according to the third embodiment of the present invention. The embodiment of the present invention is a specific embodiment of the above invention, and describes the image information collection process and the garbage situation recognition process. Figure 3 The method provided in the embodiment of the present invention specifically includes the following steps:

[0060] Step 310: Obtain image information of the working area of ​​the unmanned sweeper through an image acquisition module.

[0061] In an embodiment of the present invention, the unmanned sweeper can control the image acquisition module to acquire images of the working area, and can use the real-time acquired data as image information. The data format of the image information may include pictures or videos, and the image information at least includes data corresponding to the working area.

[0062] Step 320: Use a garbage identification module to identify garbage in the image information.

[0063] In an embodiment of the present invention, the garbage recognition module can be pre-configured with a garbage object detection algorithm, and the garbage objects in the image information can be detected by the garbage recognition module to obtain the garbage situation, which can reflect the existence of garbage in the operating area of ​​the unmanned sweeper. The garbage situation may include the presence or absence of garbage, etc.

[0064] Step 330: If the garbage identification module determines that there is garbage, an adjustment control instruction is generated.

[0065] Among them, the adjustment control instruction can be an instruction for controlling the operating mode adjustment of the unmanned sweeper and the working state adjustment of the front baffle. The adjustment control instruction can be pre-configured in the garbage identification module. The instruction content of the adjustment control instruction may include information for adjusting the working mode of the unmanned sweeper and the working state of the front baffle. Different garbage situations can manage and configure different adjustment control instructions, and the difference may include different instruction contents of the adjustment control instructions.

[0066] In the embodiment of the present invention, the garbage identification module may configure different adjustment control instructions for different garbage situations. When the garbage identification module identifies that the garbage situation of the image information includes the presence of garbage, it may obtain the adjustment control instruction associated with the configuration corresponding to the presence of garbage.

[0067] Step 340: Transmit the adjustment control instruction to the domain controller of the unmanned sweeper through the garbage identification module.

[0068] The domain controller is an important part of the unmanned sweeper to realize unmanned sweeping operations. It can adjust the operation mode of the unmanned sweeper and the working status of the front baffle. There can be one or more domain controllers, and each domain controller can be responsible for one or more operation functions in the unmanned sweeper.

[0069] In the embodiment of the present invention, after generating the adjustment control instruction, the garbage identification module may transmit the adjustment control instruction to the domain controller of the unmanned sweeper, and the domain controller responds to the adjustment control instruction and performs corresponding control operations.

[0070] Step 350: adjust the operation mode to the strong mode and adjust the working state of the front baffle to the lifted working state through the domain controller.

[0071] Among them, the strong mode is the mode in which the unmanned sweeper enters high-power operation. In the strong mode, the unmanned sweeper can take measures such as increasing suction and cleaning strength for the garbage in the operating area.

[0072] In the implementation of the present invention, the unmanned sweeper can adjust the operation mode through the domain controller and switch it to the strong mode. At the same time, the domain controller adjusts the working state of the front baffle to the lifting working state, thereby controlling the front baffle of the unmanned sweeper to lift up, so that the unmanned sweeper can suck the garbage into the vehicle.

[0073] In an embodiment of the present invention, an image acquisition module is called to collect image information of the working area of ​​the unmanned sweeper, and the garbage situation in the image information is identified according to the garbage recognition module. When it is determined that the garbage situation is garbage, an adjustment control instruction is generated, and the adjustment control instruction is transmitted to the domain controller of the unmanned sweeper. The domain controller adjusts the operating mode to the strong mode and the working state of the front baffle to the lifted working state according to the adjustment control instruction. The unmanned sweeper can adjust the working mode and the working state of the front baffle as needed based on the actual garbage presence, and can accurately clean larger garbage. The energy consumption problem of the existing unmanned sweeper caused by lifting the front baffle and adjusting the working mode according to a fixed period can be solved, and the operating energy consumption of the unmanned sweeper can be reduced, thereby improving the working endurance time.

[0074] Furthermore, based on the above-mentioned embodiments of the invention, the present invention further includes:

[0075] If the garbage identification module determines that the garbage situation includes the absence of garbage, the domain controller will not be triggered.

[0076] In an embodiment of the present invention, when the garbage identification module determines that there is no garbage in the image information, the domain controller may not be triggered, that is, there is no need to adjust the operating mode of the unmanned sweeper and the working status of the front baffle.

[0077] Embodiment 4

[0078] In the embodiments of the present invention, see Figure 4 In the cleaning mechanism of an unmanned sweeper, there is usually a front baffle. The front baffle is usually in a closed state. When encountering large objects such as mineral water bottles, cigarette boxes, branches, etc. during the cleaning process, the front baffle can be raised so that such large objects can be sucked into the operating mechanism, thereby achieving a better cleaning effect. However, the current control method is mainly based on periodic control, which results in high power consumption during the control process, affecting the operating time of the unmanned sweeper.

[0079] For this issue, see Figure 5 , an embodiment of the present invention provides a control method for an unmanned sweeper, wherein a camera is installed on the unmanned sweeper, which can capture video of the road surface in front of the unmanned sweeper. An artificial intelligence (AI) computing unit is installed in the unmanned sweeper. After the AI ​​computing unit obtains real-time video information, it uses a garbage recognition algorithm to identify whether there is a large object in front of the unmanned sweeper in the real-time video information. When the AI ​​computing unit recognizes a large object, it can send an event to the vehicle domain control unit, and the vehicle domain control unit controls the unmanned sweeper to raise the front baffle and change the cleaning mode. The embodiment of the present invention can realize the lifting of the front baffle and the change of the cleaning power on demand, which can reduce energy consumption while ensuring the operation effect, save the power of the unmanned sweeper, and increase the operation endurance of the unmanned sweeper.

[0080] Embodiment 5

[0081] Figure 6 Schematic diagram of the structure of an unmanned sweeper provided according to Embodiment 5 of the present invention. Figure 6 As shown, the device includes: an image acquisition module 401 and a garbage identification module 402.

[0082] The image acquisition module 401 is used for acquiring image information of the working area of ​​the unmanned sweeper.

[0083] The garbage identification module 402 is used to identify the garbage situation in the image information and adjust the operation mode and the working state of the front baffle of the unmanned sweeper according to the garbage situation.

[0084] Furthermore, the image information of the image acquisition module in the working area of ​​the unmanned sweeper specifically includes:

[0085] The starting unit is used to start the image acquisition module after the unmanned sweeper is powered on.

[0086] The acquisition unit is used to acquire real-time data of the working area to obtain image information, wherein the working area at least includes the ground in the running route of the unmanned sweeper.

[0087] Furthermore, the garbage identification module identifies garbage in the image information, including:

[0088] An information reading unit, used to read the image information collected by the image collection device, wherein the garbage identification module is connected to the image collection device via a serial data interface data line;

[0089] The garbage identification unit is used to identify garbage objects in the image information using the garbage identification algorithm configured by the garbage identification module to obtain the garbage situation.

[0090] Furthermore, based on the above-mentioned embodiments of the invention, the garbage recognition algorithm of the garbage recognition module includes a pre-trained garbage image recognition model, and the garbage image recognition model includes at least a backbone network, a neck network and a head network. The backbone network includes a convolution module and a convolution to feature module, the neck network includes a feature fusion module, and the head network includes a decoupling head structure. The decoupling head structure includes a classification decoupling head and a detection decoupling head.

[0091] Furthermore, based on the above-mentioned embodiments of the invention, the garbage recognition unit is specifically used to: input the image information into the garbage recognition algorithm through the garbage recognition module; use the backbone network of the garbage recognition algorithm to extract at least one feature image of a preset size of the image information; call the neck network of the garbage recognition algorithm to splice the feature images, and fuse the spliced ​​feature images with the original proportion feature map of the image information into a fused feature map; perform prediction processing on the fused feature map through the head network of the garbage recognition algorithm to obtain a preliminary prediction result; use the confidence threshold of the garbage recognition algorithm to filter the preliminary prediction result, and perform non-maximum suppression on the preliminary prediction result to obtain the final detection result with the overlapping detection boxes removed as the garbage situation.

[0092] Further, based on the above-mentioned embodiment of the invention, the unmanned sweeper also includes a domain controller, which adjusts the operation mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation, including:

[0093] If the garbage identification module determines that the garbage situation is garbage, an adjustment control instruction is generated;

[0094] The garbage identification module transmits the adjustment control instructions to the domain controller of the unmanned sweeper;

[0095] Adjust the operation mode to strong mode and the front baffle working state to lifted working state through the domain controller.

[0096] Further, based on the above-mentioned embodiment of the invention, the garbage identification module further includes: if it is determined that the garbage situation includes that there is no garbage, then the domain controller is not triggered.

[0097] The unmanned sweeper provided by the embodiment of the present invention can execute the unmanned sweeper control method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] Embodiment 6

[0099] Figure 7 : is a schematic diagram of the structure of an electronic device for implementing the unmanned sweeper control method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0100] like Figure 7 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0101] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the unmanned sweeper control method.

[0103] In some embodiments, the unmanned sweeper control method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the unmanned sweeper control method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the unmanned sweeper control method in any other appropriate manner (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0109] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A control method for an unmanned sweeper, characterized in that: Applied to an unmanned sweeper, the unmanned sweeper includes at least an image acquisition module and a garbage recognition module, and the method includes: Image information of the working area of ​​the unmanned sweeper obtained by the image acquisition module; The garbage identification module is used to identify the garbage situation in the image information, and the operation mode and the working state of the front baffle of the unmanned sweeper are adjusted according to the garbage situation.

2. The method according to claim 1, characterized in that: The image information of the working area of ​​the unmanned sweeper obtained by the image acquisition module includes: After the unmanned sweeper is powered on, the image acquisition module is started; The image acquisition module is called to perform real-time acquisition of the operating area to obtain the image information, wherein the operating area at least includes the ground in the operating route of the unmanned sweeper.

3. The method according to claim 1, characterized in that: The step of using the garbage identification module to identify garbage in the image information includes: The image information collected by the image acquisition device is read by the garbage recognition module, wherein the garbage recognition module is connected to the image acquisition device via a serial data interface data line; The garbage identification algorithm configured by the garbage identification module is used to identify garbage objects in the image information to obtain the garbage situation.

4. The method according to claim 3, characterized in that: The garbage recognition algorithm includes a pre-trained garbage image recognition model, and the garbage image recognition model includes at least a backbone network, a neck network and a head network. The backbone network includes a convolution module and a convolution to feature module, the neck network includes a feature fusion module, and the head network includes a decoupling head structure, and the decoupling head structure includes a classification decoupling head and a detection decoupling head.

5. The method according to claim 3, characterized in that: The step of identifying garbage objects in the image information using the garbage identification algorithm configured by the garbage identification module to obtain the garbage situation includes: inputting the image information into the garbage identification algorithm through the garbage identification module; Extracting at least one characteristic image of a preset size from the image information using the backbone network of the garbage identification algorithm; The neck network of the garbage identification algorithm is called to splice the feature images, and the spliced ​​feature images are fused with the original scale feature image of the image information to form a fused feature image; Performing prediction processing on the fused feature map through the head network of the garbage identification algorithm to obtain a preliminary prediction result; The confidence threshold of the junk identification algorithm is used to filter the preliminary prediction results, and non-maximum suppression is performed on the preliminary prediction results to obtain a final detection result with overlapping detection frames removed as the junk situation.

6. The method according to claim 1, characterized in that: The unmanned sweeper also includes a domain controller, and the operation mode and the front baffle working state of the unmanned sweeper are adjusted according to the garbage situation, including: If the garbage identification module determines that the garbage situation is garbage, an adjustment control instruction is generated; Transmitting the adjustment control instruction to the domain controller of the unmanned sweeper through the garbage identification module; The domain controller is used to adjust the operation mode to a strong mode and the front baffle working state to a lifted working state.

7. The method according to claim 1, characterized in that: Also includes: The garbage recognition module recognizes the height of the garbage in the image information, and controls the front baffle to be raised to a corresponding height according to the garbage height.

8. An unmanned sweeper, characterized in that: The unmanned sweeper at least includes: an image acquisition module and a garbage recognition module, the image acquisition module acquires image information of the operating area of ​​the unmanned sweeper; the garbage recognition module recognizes the garbage situation in the image information, and adjusts the operating mode of the unmanned sweeper and the working state of the front baffle according to the garbage situation.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the unmanned sweeper control method according to any one of claims 1 to 7 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program implements the unmanned sweeper control method according to any one of claims 1 to 7.

Images