A method and apparatus for stain treatment of a robotic vacuum cleaner

Abstract

The application relates to the technical field of visual identification, in particular to a stain treatment method and device for a sweeping robot, the method comprising the following steps: receiving a first monocular image and a second monocular image, determining a stain identification frame in the two images, obtaining the mileage distance of the robot when the two images are collected respectively, then calling a stain position calculation model to input the coordinate value of the stain identification frame, the mileage distance, the height value and the focal length value, obtaining a target relative distance from the sweeping robot to a stain target point, and then controlling the sweeping robot to walk and execute a preset cleaning strategy to clean the stain target point and the nearby area according to the target relative distance. The distance value obtained through one image is more accurate, and the sweeping robot can reach an accurate ground position, so that the sweeping robot can effectively clean.

Description

Technical Field

[0001] This application relates to the field of visual recognition technology, and in particular to a method and apparatus for cleaning stains with a sweeping machine. Background Technology

[0002] In the field of intelligent cleaning technology, the identification and cleaning methods for surface stains are one way to improve the cleaning effect of intelligent devices.

[0003] Currently, there are two main methods for identifying and cleaning floor stains in robotic vacuum cleaners. One method adjusts the cleaning intensity of the cleaning components based on data from the dust detection module. This method is simple and low-cost, but its drawback is that it can only identify a limited range of stain types. The other method adjusts the cleaning intensity based on user interaction, which requires user involvement. With the evolution of robotic vacuum cleaners and the continuous development of artificial intelligence technology, vision-based solutions are becoming a trend, but they still need further improvement. Summary of the Invention

[0004] In view of the aforementioned problems, this application is made to provide a method and apparatus for cleaning stains with a sweeping machine that overcomes or at least partially solves the aforementioned problems.

[0005] The sweeping machine is equipped with a monocular camera whose optical axis is parallel to the ground. The method includes:

[0006] Receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and determine a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image;

[0007] The robot acquires the first mileage distance and the second mileage distance when acquiring the first monocular image and the second monocular image, respectively, and acquires the height value and focal length value of the monocular camera;

[0008] Call the preset stain location calculation model, and input the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value and the focal length value into the stain location calculation model. The stain location calculation model outputs the target relative distance from the sweeper to the stain target point.

[0009] The sweeper is controlled to move according to the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and the preset stain cleaning mode is triggered to clean the area around the stain target point.

[0010] Preferably, receiving the first monocular image and the second monocular image continuously acquired by the monocular camera, and determining the first stain marker frame in the first monocular image and the second stain marker frame in the second monocular image, includes:

[0011] Receive the first monocular image and call the preset stain recognition model to perform stain recognition on the first monocular image;

[0012] When a stain target image is detected in the first monocular image by the stain recognition model, the first stain identification box is generated based on the stain target image and the first bottom line ordinate value of the first stain identification box in the first monocular image is determined.

[0013] The second monocular image is received and the second stain identification box is generated through the stain recognition model, and the second bottom line ordinate value of the second stain identification box in the second monocular image is determined.

[0014] Preferably, the step of receiving the first monocular image and calling a preset stain recognition model to perform stain recognition on the first monocular image includes:

[0015] Multiple stain training images are acquired and stain features are extracted from each stain training image. The stain features include category features, ground texture features, and environmental features.

[0016] Based on the location information of the stain features in the corresponding stain training image, a stain identification box is marked;

[0017] The stain training image is used as the training input of the stain recognition model, and the corresponding labeled stain identification box is used as the training output of the stain recognition model to train the target stain recognition model.

[0018] Input the first monocular image into the target stain recognition model for stain recognition.

[0019] Preferably, the step of calling a preset stain location calculation model and inputting the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model, and outputting the target relative distance from the sweeper to the stain target point through the stain location calculation model, includes:

[0020] Call the stain location calculation model;

[0021] Input the first baseline ordinate value, the second baseline ordinate value, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model, and output the target relative distance from the sweeper to the stain target point through the stain location calculation model.

[0022] Preferably, the step of outputting the target relative distance from the sweeper to the target stain point through the stain location calculation model includes:

[0023] A first functional relationship is constructed based on the first baseline ordinate value, the first mileage distance, the height value, and the focal length value to establish a first relative distance between the sweeping machine and the stain target point when acquiring the first monocular image;

[0024] A second functional relationship is constructed based on the second baseline ordinate value, the second mileage distance, the height value, and the focal length value, establishing a second relative distance between the sweeping machine and the stain target point when acquiring the second monocular image;

[0025] The first relative distance and the second relative distance are determined by the first functional relationship and the second functional relationship, and the target relative distance is the average of the first relative distance and the second relative distance.

[0026] Preferably, determining the first relative distance and the second relative distance through the first functional relationship and the second functional relationship, wherein the target relative distance is the average of the first relative distance and the second relative distance, includes:

[0027] The distance difference between the first relative distance and the second relative distance is determined based on the first mileage distance and the second mileage distance, and the horizon coordinates in the first monocular image and the second monocular image are determined based on the distance difference;

[0028] The first relative distance is determined based on the horizon coordinates and the first functional relationship, and the second relative distance is determined based on the horizon coordinates and the second functional relationship.

[0029] The target relative distance is determined based on the first relative distance and the second relative distance.

[0030] Preferably, controlling the sweeper's movement based on the relative distance to the target includes:

[0031] The preset motion control model is invoked, and the robot vacuum is controlled to move to the target stain based on the relative distance to the target stain and the offset angle of the target stain relative to the center of the monocular image.

[0032] A stain removal device for a sweeping machine is also provided, wherein the sweeping machine is equipped with a monocular camera whose optical axis is parallel to the ground, and the device includes:

[0033] The image receiving module is used to receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and to determine a first stain mark frame in the first monocular image and a second stain mark frame in the second monocular image.

[0034] The parameter acquisition module is used to acquire the first mileage distance and the second mileage distance when the robot acquires the first monocular image and the second monocular image, respectively, and to acquire the height value and focal length value of the monocular camera;

[0035] The distance calculation module is used to call a preset stain location calculation model and input the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model. The stain location calculation model outputs the target relative distance from the sweeper to the stain target point.

[0036] The action execution module is used to control the sweeper to move according to the target relative distance. When the sweeper moves to the target relative distance, it determines the stain target point and triggers the preset stain cleaning mode to clean the area around the stain target point.

[0037] A computer device is also provided, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described in any of the above descriptions.

[0038] A computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the method described in any of the above descriptions.

[0039] This application has the following advantages:

[0040] In the embodiments of this application, by receiving the first monocular image and the second monocular image continuously acquired by the monocular camera, a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image are determined; the first mileage distance and the second mileage distance of the robot when acquiring the first monocular image and the second monocular image are obtained respectively, and the height value and focal length value of the monocular camera are obtained; a preset stain location calculation model is called, and the position coordinate values ​​of the first stain marker frame, the position coordinate values ​​of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value are input to... In the stain location calculation model, the model outputs the target relative distance from the sweeper to the stain target point. The sweeper is controlled to move based on the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and a preset stain cleaning mode is triggered to clean the area around the stain target point. By acquiring monocular images of two adjacent frames, and combining the mileage distance, camera parameters, and position coordinates of the marker box in each image, the model outputs the target relative distance. This target relative distance is more accurate than the value calculated from a single image, which helps the sweeper reach the accurate ground position for effective cleaning. Attached Figure Description

[0041] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating the steps of a method for removing stains using a sweeper according to an embodiment of this application;

[0043] Figure 2 This is a structural block diagram of a stain removal device for a sweeper provided in one embodiment of this application;

[0044] Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0046] It should be noted that the sweeping machine of this application incorporates a stain recognition model, a stain location calculation model, and a motion control model. A monocular camera is installed at the front of the sweeping machine to capture images of the environment in front of it. The stain recognition model is mainly used to identify stain targets in the captured monocular images. The stain location calculation model is mainly used to calculate the relative distance between the sweeping machine and the stain target. The motion control model is mainly used to control the sweeping machine to move according to the relative distance, and then adjust the sweeping machine's cleaning mode to clean the stain target and its surroundings.

[0047] Reference Figure 1 This document illustrates a flowchart of a method for cleaning stains using a sweeping machine according to an embodiment of this application. The sweeping machine is equipped with a monocular camera whose optical axis is parallel to the ground. The method includes the following steps:

[0048] S110, receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and determine a first stain mark frame in the first monocular image and a second stain mark frame in the second monocular image;

[0049] S120, obtain the first mileage distance and the second mileage distance when the robot collects the first monocular image and the second monocular image respectively, and obtain the height value and focal length value of the monocular camera;

[0050] S130, call the preset stain location calculation model, and input the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value and the focal length value into the stain location calculation model, and output the target relative distance from the sweeper to the stain target point through the stain location calculation model;

[0051] S140, the sweeper is controlled to move according to the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and the preset stain cleaning mode is triggered to clean the area around the stain target point.

[0052] By receiving the first and second monocular images continuously acquired by the monocular camera, a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image are determined; the first and second mileage distances of the robot when acquiring the first and second monocular images are obtained, respectively, and the height and focal length values ​​of the monocular camera are obtained; a preset stain location calculation model is invoked, and the position coordinates of the first and second stain marker frames, the first and second mileage distances, the height and focal length values ​​are input to the stain location. In the calculation model, the target relative distance from the sweeper to the target stain point is output through the stain location calculation model. The sweeper is controlled to move according to the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and a preset stain cleaning mode is triggered to clean the area around the stain target point. By acquiring monocular images of two adjacent frames, and combining the mileage distance, camera parameters, and position coordinates of the marker box in each image, the target relative distance is output through the model. This target relative distance is more accurate than the value calculated from a single image, which helps the sweeper reach the accurate ground position for effective cleaning.

[0053] The following will further explain a method for treating stains using a sweeper in the above embodiments.

[0054] As described in step S110, a first monocular image and a second monocular image continuously acquired by the monocular camera are received, and a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image are determined.

[0055] In one embodiment of this application, the specific process of "receiving the first monocular image and the second monocular image continuously acquired by the monocular camera" in step S110 can be further explained in conjunction with the following description.

[0056] As described in the following steps, the first monocular image is received, and a preset stain recognition model is invoked to perform stain recognition on the first monocular image; when a stain target image is identified in the first monocular image by the stain recognition model, a first stain identification box is generated based on the stain target image, and the first bottom line ordinate value of the first stain identification box in the first monocular image is determined; the second monocular image is received, and the second stain identification box is generated by the stain recognition model, and the second bottom line ordinate value of the second stain identification box in the second monocular image is determined.

[0057] In one specific embodiment, when the first monocular image is received at time t1, the first monocular image is input into the stain recognition model. If the stain recognition model outputs the first stain identification box of the first monocular image and the bottom line ordinate of the first stain identification box, then the second monocular image at time t2 is acquired again, and the second stain identification box and the second bottom line ordinate of the second monocular image are determined.

[0058] In one embodiment of this application, the prerequisite for outputting the first stain identification box of the first monocular image is that the stain recognition model has been trained, and the training steps include:

[0059] Multiple stain training images are acquired, and stain features are extracted from each stain training image. The stain features include category features, ground texture features, and environmental features. Stain identification boxes are marked according to the location information of the stain features in the corresponding stain training images. The stain training images are used as training inputs to the stain recognition model, and the corresponding marked stain identification boxes are used as training outputs to train the target stain recognition model. The first monocular image is then input into the target stain recognition model for stain recognition.

[0060] It should be noted that the recognition model to be trained can directly adopt deep learning models for object detection, including but not limited to the YOLO series, SSD series, and CNN series. After the model training is completed, it is necessary to test it on metrics such as recall, accuracy, and precision, and to actually verify the model's effectiveness in detecting stains. Once the above requirements are met, the trained model can be applied to the real-time acquired monocular images for stain target recognition.

[0061] As described in step S120, the first mileage distance and the second mileage distance of the robot when acquiring the first monocular image and the second monocular image respectively are obtained, as well as the height value and focal length value of the monocular camera are obtained.

[0062] It should be noted that the distance between the robot vacuum and the target stain at time t1 is d1, and the distance between the robot vacuum and the target stain at time t2 is d2. The relative distance can be derived from d1 and d2 as follows:

[0063] Δd=d1-d2

[0064] Based on the actual movement of the sweeper, this application collects the first mileage distance at time t1 as O1 and the first mileage distance at time t2 as O2, and obtains the relative distance from O1 and O2.

[0065] Δd=O1-O2

[0066] As is known, the mileage distance can be directly obtained through the distance recording instrument built into the sweeper.

[0067] As described in step S130, a preset stain location calculation model is called, and the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value are input into the stain location calculation model. The target relative distance from the sweeper to the stain target point is output through the stain location calculation model.

[0068] In one embodiment of this application, the specific process of "outputting the target relative distance from the sweeper to the target stain point through the stain location calculation model" in step S130 can be further explained in conjunction with the following description.

[0069] As described in the following steps, the stain location calculation model is invoked; the first baseline ordinate value, the second baseline ordinate value, the first mileage distance, the second mileage distance, the height value, and the focal length value are input into the stain location calculation model, and the target relative distance from the sweeper to the stain target point is output through the stain location calculation model.

[0070] Specifically, a first functional relationship is constructed based on the first baseline ordinate value, the first mileage distance, the height value, and the focal length value to establish a first relative distance between the sweeping machine and the target stain point when acquiring the first monocular image; a second functional relationship is constructed based on the second baseline ordinate value, the second mileage distance, the height value, and the focal length value to establish a second relative distance between the sweeping machine and the target stain point when acquiring the second monocular image; the first relative distance and the second relative distance are determined through the first functional relationship and the second functional relationship, and the target relative distance is the average of the first relative distance and the second relative distance.

[0071] It should be noted that this solution is inspired by existing technologies that utilize conventional distance estimation methods in vision-based systems using monocular cameras. Under the condition that the roll angle and yaw angle are zero, the distance to the vehicle ahead on the actual road is estimated from the vehicle's position in the image using the camera's geometry. Based on the real-world application of this application in a sweeper, the distance calculation formula used in the aforementioned existing technology when the pitch angle is zero or negligible is as follows:

[0072]

[0073] In the above formula, F c It is the camera focal length, H c It is the camera height, y b It is the vertical coordinate of the vehicle's baseline in the image, y hd is the vertical coordinate of the horizon in the image, and d is the distance from the robot vacuum to the target stain.

[0074] In the above formula, the vertical coordinate of the baseline can be obtained by calculating the coordinates of the image frame that identifies the vehicle in the image. However, due to unevenness of the ground or camera installation deviation, changes in the horizontal position (the horizon may present different coordinates in two consecutive frames) lead to a large range error, making it difficult to directly identify the horizon through image recognition. Existing technologies extensively use the average width of a large number of target vehicles to estimate the value of the horizon in the image. However, in sweeping robots, the size of target objects such as stains varies greatly, making them difficult to use for horizon estimation. This application, assuming that the ground is flat in two adjacent frames, uses at least two consecutive frames of odometer data and image detection frames to estimate the value of the horizon.

[0075] Based on the distance calculation formula above, the formula for the first functional relationship between the first relative distance between the sweeping machine and the target stain at time t1 when the first monocular image is acquired is:

[0076]

[0077] The formula for the second function relationship of the second relative distance between the sweeper and the target stain at time t1 when the first monocular image is acquired is:

[0078]

[0079] Then, the mean of d1 and d2 is calculated as the estimated distance from the sweeper to the target stain at the current moment.

[0080] In one embodiment of this application, determining the first relative distance and the second relative distance through the first functional relationship and the second functional relationship, wherein the target relative distance is the average of the first relative distance and the second relative distance, includes:

[0081] The distance difference between the first relative distance and the second relative distance is determined based on the first mileage distance and the second mileage distance. The horizon coordinates in the first monocular image and the second monocular image are determined based on the distance difference. The first relative distance is determined based on the horizon coordinates and the first functional relationship. The second relative distance is determined based on the horizon coordinates and the second functional relationship. The target relative distance is determined based on the first relative distance and the second relative distance.

[0082] That is, the relative distance Δd between d1 and d2 can be transformed into:

[0083]

[0084] Since d1-d2=Δd=O1-O2、F c .H c and y b2 -y b1 Given all the conditions, the horizon y in the two images can be estimated. h The value of y, then the horizon y h Substituting the values ​​into the first and second functional relationships in the above equation, we obtain the values ​​of the first relative distance d1 and the second relative distance d2, and then obtain the final target relative distance.

[0085] It should be noted that the constraint equation for the horizon is:

[0086] h min ≤y h ≤h max

[0087] In this application, when the robot vacuum detects a stain, it adjusts its walking angle according to the position and angle of the stain in the current image, so that the robot vacuum's walking front corresponds to the stain point. This involves some control algorithms, including but not limited to fixed speed movement control, PID control, dynamic window method (DWA) control, etc.

[0088] As described in step S140, the sweeper is controlled to move according to the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and the preset stain cleaning mode is triggered to clean the area around the stain target point.

[0089] In one embodiment of this application, the specific process of "controlling the sweeper to move according to the relative distance to the target" in step S140 can be further explained in conjunction with the following description.

[0090] As described in the following steps, a preset motion control model is invoked, and the sweeping machine is controlled to move to the target stain based on the relative distance to the target stain and the offset angle of the target stain relative to the center of the monocular image.

[0091] Specifically, once the robot vacuum approaches the target stain, it begins focused cleaning, employing methods including, but not limited to, reciprocating cleaning, adjusting the cleaning intensity, and circular cleaning. It can clean the target stain in a small, arc-shaped motion.

[0092] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0093] Reference Figure 2 The diagram shows a structural block diagram of a stain removal device for a sweeper according to an embodiment of this application.

[0094] The device includes:

[0095] The image receiving module 110 is used to receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and to determine a first stain mark frame in the first monocular image and a second stain mark frame in the second monocular image.

[0096] The parameter acquisition module 120 is used to acquire the first mileage distance and the second mileage distance when the robot acquires the first monocular image and the second monocular image respectively, and to acquire the height value and focal length value of the monocular camera.

[0097] The distance calculation module 130 is used to call a preset stain location calculation model and input the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value and the focal length value into the stain location calculation model. The stain location calculation model outputs the target relative distance from the sweeper to the stain target point.

[0098] The action execution module 140 is used to control the sweeper to move according to the target relative distance. When the sweeper moves to the target relative distance, it determines the stain target point and triggers the preset stain cleaning mode to clean the area around the stain target point.

[0099] Reference Figure 3 The computer device illustrating a method for stain removal by a sweeping machine according to the present invention may specifically include the following:

[0100] The computer device 12 described above is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0101] Bus 18 refers to one or more of several types of bus 18 architectures, including memory bus 18 or memory controller, peripheral bus 18, graphics acceleration port, processor, or local bus 18 using any of the various bus 18 architectures. For example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus 18, Micro Channel Architecture (MAC) bus 18, Enhanced ISA bus 18, Audio / Video Electronics Standards Association (VESA) local bus 18, and Peripheral Component Interconnect (PCI) bus 18.

[0102] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0103] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Figure 3 Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42 configured to perform the functions of the embodiments of the present invention.

[0104] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory. Such program modules 42 include—but are not limited to—an operating system, one or more application programs, other program modules 42, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0105] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, camera, etc.), and with one or more devices that enable an operator to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 3Not shown, it can be combined with computer device 12 to use other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 16, external disk drive array, RAID system, tape drive and data backup storage system 34, etc.

[0106] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a stain removal method for a sweeper provided in an embodiment of the present invention.

[0107] That is, when the processing unit 16 executes the above program, it performs the following: receiving a first monocular image and a second monocular image continuously acquired by the monocular camera; determining a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image; obtaining a first mileage distance and a second mileage distance when the robot acquires the first monocular image and the second monocular image respectively; and obtaining the height value and focal length value of the monocular camera; calling a preset stain location calculation model and inputting the position coordinate values ​​of the first stain marker frame, the position coordinate values ​​of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model; outputting the target relative distance from the sweeper to the stain target point through the stain location calculation model; controlling the sweeper to walk according to the target relative distance; when the sweeper walks to the target relative distance, determining the stain target point and triggering a preset stain cleaning mode to clean the area around the stain target point.

[0108] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements a method for cleaning stains using a sweeping robot as provided in all embodiments of this application:

[0109] That is, when the program is executed by the processor, it implements the following: receiving a first monocular image and a second monocular image continuously acquired by the monocular camera; determining a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image; obtaining a first mileage distance and a second mileage distance of the robot when acquiring the first monocular image and the second monocular image, respectively; and obtaining the height value and focal length value of the monocular camera; calling a preset stain location calculation model, and inputting the position coordinate values ​​of the first stain marker frame, the position coordinate values ​​of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model; outputting the target relative distance from the sweeper to the stain target point through the stain location calculation model; controlling the sweeper to walk according to the target relative distance; when the sweeper walks to the target relative distance, determining the stain target point and triggering a preset stain cleaning mode to clean the area around the stain target point.

[0110] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0111] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0112] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as a standalone software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the operator's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0113] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0114] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0115] The above provides a detailed description of the method and apparatus for cleaning stains using a sweeping machine provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for stain removal by a sweeping machine, wherein the sweeping machine is equipped with a monocular camera whose optical axis is parallel to the ground, characterized in that, The method includes: Receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and determine a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image; The first mileage distance and the second mileage distance of the sweeping robot when acquiring the first monocular image and the second monocular image respectively are obtained, as well as the height value and focal length value of the monocular camera are obtained; A preset stain location calculation model is invoked, and the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value are input into the stain location calculation model. The stain location calculation model outputs the target relative distance from the sweeping machine to the stain target point. Specifically, the distance difference is determined based on the first mileage distance and the second mileage distance; the horizon coordinates are determined based on the distance difference, the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the height value, and the focal length value; and the target relative distance is determined based on the horizon coordinates. The sweeper is controlled to move according to the target relative distance. When the sweeper moves to the target relative distance, the stain target point is determined and the preset stain cleaning mode is triggered to clean the area around the stain target point.

2. The method according to claim 1, characterized in that, The step of receiving a first monocular image and a second monocular image continuously acquired by the monocular camera, and determining a first stain marker frame in the first monocular image and a second stain marker frame in the second monocular image, includes: Receive the first monocular image and call the preset stain recognition model to perform stain recognition on the first monocular image; When a stain target image is detected in the first monocular image by the stain recognition model, the first stain identification box is generated based on the stain target image and the first bottom line ordinate value of the first stain identification box in the first monocular image is determined. The second monocular image is received and the second stain identification box is generated through the stain recognition model, and the second bottom line ordinate value of the second stain identification box in the second monocular image is determined.

3. The method according to claim 2, characterized in that, The step of receiving the first monocular image and calling a preset stain recognition model to perform stain recognition on the first monocular image includes: Multiple stain training images are acquired and stain features are extracted from each stain training image. The stain features include category features, ground texture features, and environmental features. Based on the location information of the stain features in the corresponding stain training image, a stain identification box is marked; The stain training image is used as the training input of the stain recognition model, and the corresponding labeled stain identification box is used as the training output of the stain recognition model to train the target stain recognition model. Input the first monocular image into the target stain recognition model for stain recognition.

4. The method according to claim 2, characterized in that, The process involves calling a preset stain location calculation model and inputting the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model. The model then outputs the target relative distance from the sweeper to the stain target point, including: Call the stain location calculation model; Input the first baseline ordinate value, the second baseline ordinate value, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model, and output the target relative distance from the sweeper to the stain target point through the stain location calculation model.

5. The method according to claim 4, characterized in that, The step of outputting the target relative distance from the sweeper to the target stain point through the stain location calculation model includes: A first functional relationship is constructed based on the first baseline ordinate value, the first mileage distance, the height value, and the focal length value to establish a first relative distance between the sweeping machine and the stain target point when acquiring the first monocular image; A second functional relationship is constructed based on the second baseline ordinate value, the second mileage distance, the height value, and the focal length value, establishing a second relative distance between the sweeping machine and the stain target point when acquiring the second monocular image; The first relative distance and the second relative distance are determined by the first functional relationship and the second functional relationship, and the target relative distance is the average of the first relative distance and the second relative distance.

6. The method according to claim 5, characterized in that, The step of determining the first relative distance and the second relative distance through the first functional relationship and the second functional relationship, wherein the target relative distance is the average of the first relative distance and the second relative distance, includes: The distance difference between the first relative distance and the second relative distance is determined based on the first mileage distance and the second mileage distance, and the horizon coordinates in the first monocular image and the second monocular image are determined based on the distance difference; The first relative distance is determined based on the horizon coordinates and the first functional relationship, and the second relative distance is determined based on the horizon coordinates and the second functional relationship. The target relative distance is determined based on the first relative distance and the second relative distance.

7. A stain removal device for a sweeping machine, wherein the sweeping machine is equipped with a monocular camera whose optical axis is parallel to the ground, characterized in that, The device includes: The image receiving module is used to receive a first monocular image and a second monocular image continuously acquired by the monocular camera, and to determine a first stain mark frame in the first monocular image and a second stain mark frame in the second monocular image. The parameter acquisition module is used to acquire the first mileage distance and the second mileage distance when the sweeping robot collects the first monocular image and the second monocular image, respectively, and to acquire the height value and focal length value of the monocular camera; The distance calculation module is used to call a preset stain location calculation model and input the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the first mileage distance, the second mileage distance, the height value, and the focal length value into the stain location calculation model. The model then outputs the target relative distance from the sweeper to the stain target point. Specifically, the distance calculation module is used to determine the distance difference based on the first mileage distance and the second mileage distance, and to determine the horizon coordinates based on the distance difference, the position coordinates of the first stain marker frame, the position coordinates of the second stain marker frame, the height value, and the focal length value. Finally, it is used to determine the target relative distance based on the horizon coordinates. The action execution module is used to control the sweeper to move according to the target relative distance. When the sweeper moves to the target relative distance, it determines the stain target point and triggers the preset stain cleaning mode to clean the area around the stain target point.

8. A computer device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images