Cleaning device control method and apparatus, cleaning method and apparatus, and electronic device
By generating cleaning control commands from user image, voice, or text input, the problem of cleaning equipment having difficulty determining the target area is solved, enabling more convenient and precise cleaning operations.
Patent Information
- Application Number
- PCT/CN2026/083086
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-03-20
- Filing Date
- 2026-03-12
- Publication Date
- 2026-11-19
AI Technical Summary
Existing cleaning equipment has difficulty in conveniently identifying and indicating the location of the target area during the cleaning process, resulting in inaccurate cleaning operations.
By acquiring user input information, including images, voice, or text, cleaning control commands are generated to instruct cleaning equipment to perform cleaning operations on the target area.
It enables more convenient identification and indication of the target area for cleaning equipment, improving the accuracy and efficiency of cleaning operations.
Smart Images

Figure CN2026083086_19112026_PF_FP_ABST
Abstract
Description
Control methods, cleaning methods, devices and electronic equipment for cleaning equipment Cross-reference to related applications
[0001] This application claims priority to the application filed on March 20, 2025, with China National Intellectual Property Administration, application number 2025103373011, entitled "Control method, cleaning method, apparatus and electronic equipment for cleaning equipment", the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application belongs to the field of automatic control, and in particular relates to a control method, cleaning method, device and electronic equipment for cleaning equipment. Background Technology
[0003] Nowadays, cleaning devices with automatic cleaning capabilities are favored by most consumers. Take robotic vacuum cleaners as an example; they can automatically clean up trash in the user's home environment, improve home cleanliness, and free users from daily household cleaning.
[0004] During the cleaning process, cleaning equipment often needs to clean or avoid specific target areas. How to more conveniently indicate the location of the target area for the cleaning equipment is a problem that needs to be solved.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] The embodiments of this application provide a control method, cleaning method, apparatus, and electronic device for cleaning equipment, which can at least to some extent make it easier to determine the target area.
[0007] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.
[0008] According to a first aspect of the embodiments of this application, a control method for a cleaning device is provided, for controlling a terminal, including:
[0009] Obtain user input information, the input information including description information of the target area, the description information including at least one of image, voice or text;
[0010] Based on the input information, a cleaning control command associated with the target area is generated. The cleaning control command is used to instruct the cleaning equipment to perform a cleaning operation on the target area.
[0011] The cleaning control command is sent to the cleaning equipment.
[0012] According to a second aspect of the embodiments of this application, a cleaning method is provided for cleaning equipment, comprising:
[0013] Receive cleaning control instructions, wherein the cleaning control instructions are generated based on user input information, the input information including descriptive information of the target area, the descriptive information including at least one of image, voice or text;
[0014] According to the cleaning control command, the cleaning components of the cleaning equipment are controlled to perform cleaning operations on the target area.
[0015] According to a third aspect of the embodiments of this application, a control device for a cleaning equipment is provided, comprising:
[0016] A first receiver is configured to acquire user input information, the input information including descriptive information of a target area, the descriptive information including at least one of image, voice or text;
[0017] A processor is configured to generate cleaning control instructions associated with the target area based on the input information, the cleaning control instructions being used to instruct the cleaning equipment to perform cleaning operations on the target area;
[0018] A transmitter is used to send the cleaning control commands to the cleaning equipment.
[0019] According to a fourth aspect of the embodiments of this application, a cleaning apparatus is provided, comprising:
[0020] The second receiver is used to receive cleaning control instructions, wherein the cleaning control instructions are generated based on user input information, the input information including descriptive information of the target area, the descriptive information including at least one of image, voice or text;
[0021] The controller is used to control the cleaning components of the cleaning equipment to perform cleaning operations on the target area according to the cleaning control instructions.
[0022] According to a fifth aspect of the embodiments of this application, a cleaning control system is provided, comprising:
[0023] A control terminal is used to acquire user input information, generate cleaning control instructions associated with a target area based on the input information, and send the cleaning control instructions to the cleaning equipment. The input information includes description information of the target area, which includes at least one of images, voice, or text.
[0024] Cleaning equipment used to perform cleaning operations based on a target area in response to cleaning control commands.
[0025] According to a sixth aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the methods described in the above embodiments.
[0026] According to a seventh aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the method described in the above embodiments.
[0027] According to an eighth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the methods described in the embodiments above.
[0028] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:
[0030] Figure 1 is an application scenario diagram of a control method for a cleaning device provided in this application;
[0031] Figure 2 is an application scenario diagram of a control method for a cleaning device provided in this application;
[0032] Figure 3 is a flowchart of a control method for a cleaning device provided in an embodiment of this application;
[0033] Figure 4 is a schematic diagram of a control scheme for a cleaning device provided in an embodiment of this application;
[0034] Figure 5 is a schematic diagram of a control scheme for a cleaning device provided in an embodiment of this application;
[0035] Figure 6 is a schematic diagram of the target location information generation scheme provided in the embodiments of this application;
[0036] Figure 7 is a schematic diagram of the target location information generation scheme provided in the embodiments of this application;
[0037] Figure 8 is a schematic diagram of the scheme for generating target location information by the recognition model provided in the embodiment of this application;
[0038] Figure 9 is a flowchart of a cleaning method provided in an embodiment of this application;
[0039] Figure 10 is a schematic diagram of the structure of a control device for a cleaning equipment provided in an embodiment of this application;
[0040] Figure 11 is a schematic diagram of a cleaning device provided in an embodiment of this application;
[0041] Figure 12 is a schematic diagram of a cleaning control system provided in an embodiment of this application;
[0042] Figure 13 is a schematic diagram of the structure of an electronic device for controlling cleaning equipment provided in an embodiment of this application. Detailed Implementation
[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0044] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.
[0045] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0046] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.
[0047] It should also be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such uses of these terms can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described.
[0048] The technical solutions of this application and their effects are described below through several exemplary embodiments. It should be noted that the following embodiments can be referenced, borrowed from, or combined with each other. Identical terms, similar features, and similar implementation steps in different embodiments will not be repeated.
[0049] The following will illustrate the application scenarios of the cleaning equipment control method of this application with examples. As shown in Figure 1, in one example, the application scenario of the cleaning equipment control method of this application may include a control terminal 101 and a cleaning device 102. Specifically, the control terminal 101 acquires user input information and description information of the target area, wherein the description information includes at least one of image, voice, or text; the control terminal 101 generates a cleaning control command associated with the target area based on the input information, and sends the cleaning control command to the cleaning device 102, and the cleaning device 102 performs cleaning operations based on the target area.
[0050] Those skilled in the art will understand that the terminal can be a controller specifically configured for the cleaning equipment, or it can be a smartphone (such as an Android phone, iOS phone, etc.), tablet computer, laptop computer, digital broadcast receiver, MID (Mobile Internet Devices), PDA (Personal Digital Assistant), desktop computer, smart home appliance, vehicle terminal (such as a vehicle navigation terminal, vehicle computer, etc.), smart speaker, smartwatch, etc. that establishes a communication connection with the cleaning equipment. The control terminal and the cleaning equipment can be directly or indirectly connected through wired or wireless communication, but are not limited to this.
[0051] Those skilled in the art will understand that cleaning equipment can be any movable mechanical device used for cleaning, and can include, but is not limited to, cleaning equipment used in various industries such as municipal sanitation, airport terminals, commercial buildings, public places, manufacturing workshops, warehousing and logistics, hotels, medical and health institutions, retail, sports venues, etc. Cleaning equipment can include sweepers, floor scrubbers, sweeper trucks, dust mops, etc.
[0052] As shown in Figure 2, in one example, the application scenario of the cleaning equipment control method of this application may include a control terminal 201, a cleaning equipment 202, and a server 203. Specifically, the control terminal 201 acquires user input information, including descriptive information of the target area, which may include at least one of images, voice, or text; the control terminal 201 sends the input information to the server 203, the server 203 generates a cleaning control command associated with the target area based on the input information, and the server 203 sends the cleaning control command to the cleaning equipment 202, which then performs cleaning operations based on the target area.
[0053] In the application scenario shown in Figure 2, the server generates cleaning control instructions based on the input information and sends the cleaning control instructions directly to the cleaning equipment. In other application scenarios, the server may send the cleaning control instructions to the control terminal, and the control terminal may then send the cleaning control instructions to the cleaning equipment.
[0054] It should be noted that the above application scenarios are merely examples of the control method for the cleaning equipment of this application, and do not limit the application scenarios of the control method for the cleaning equipment of this application.
[0055] Those skilled in the art will understand that the terminal can be a controller specifically configured for the cleaning equipment, or a smartphone that establishes a communication connection with the cleaning equipment, etc. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.
[0056] As shown in Figure 3, in some possible implementations, this application provides a method for controlling a cleaning device. Taking the control terminal as the executing entity, the method may include the following steps:
[0057] Step S301: Obtain user input information.
[0058] The input information includes descriptive information about the target area. The descriptive information can be a description of the area, such as the type or location of the area, like a bedroom, living room, or dining room. The descriptive information can also include descriptions of reference objects in the area, such as a dining table, coffee table, or cat bed.
[0059] Specifically, the descriptive information may include at least one of images, audio, or text.
[0060] The target area can include areas to be cleaned or areas to be avoided.
[0061] Specifically, different types of descriptive information can correspond to different input methods. For example, when the descriptive information includes an image, the image can be directly acquired through the control terminal; when the descriptive information includes voice, the descriptive information can be obtained through voice input; when the descriptive information includes text, the descriptive information can be obtained through text input, or the input voice can be converted into text to obtain the descriptive information. The following will further elaborate on the acquisition process of different types of descriptive information.
[0062] Step S302: Generate a cleaning control instruction associated with the target area based on the input information. The cleaning control instruction is used to instruct the cleaning equipment to perform a cleaning operation on the target area.
[0063] Among them, cleaning control instructions can directly or indirectly indicate the location of the target area.
[0064] Specifically, cleaning control commands can directly indicate the location of the target area. This can be achieved by the control terminal determining the target location information corresponding to the target area based on the input information and generating the cleaning control command based on the target location information. Cleaning control commands can also indirectly indicate the location of the target area. This can be achieved by the control terminal sending the input information as a cleaning control command to the cleaning equipment, and the cleaning equipment determining the target location information of the target area based on the cleaning control command. The specific process of generating cleaning control commands will be described in detail below.
[0065] Step S303: Send the cleaning control command to the cleaning equipment.
[0066] It is understandable that the cleaning operation performed by the cleaning equipment corresponds to the description information. When the description information is a description of an area, the cleaning operation is for the treatment of that area; when the description information is a description of a reference object, the cleaning operation is for the treatment of an area within a preset distance of the reference object.
[0067] In the above embodiments, the user inputs information into the control terminal. This input information includes a description of the target area and can be in various forms such as voice, text, and images. The control terminal then generates a cleaning control command associated with the target area based on the input information and sends the command to the cleaning device. This allows for more convenient indication of the target area's location, enabling the cleaning device to perform cleaning operations based on that area. The cleaning operation can include cleaning and obstacle avoidance; that is, the cleaning device responds to the cleaning control command by cleaning the target area or avoiding it.
[0068] The following section will explain the specific process of obtaining input information by combining different types of descriptive information.
[0069] In some possible implementations, the descriptive information includes an image, and step S301 obtains user input information, including at least one of the following:
[0070] (1) The description information is generated based on the user's first selection operation for multiple real-time screens, wherein the real-time screens are real-time screens sent by the cleaning equipment.
[0071] In the specific implementation process, the control terminal can be equipped with a control application for communicating with the cleaning equipment. The cleaning equipment may continuously transmit real-time images to the control application on the control terminal and display them during the movement. The real-time images displayed in the control application may be constantly changing. The user can select a real-time image in the display interface as descriptive information through the first selection operation.
[0072] The first selection operation can be a click, long press, double-click, or other operation on the display interface of the control terminal that displays the real-time image; or it can be a selection trigger control displayed on the display interface used to receive the real-time image, and the real-time image can be selected by triggering the selection trigger control.
[0073] Specifically, the description information is generated based on the real-time screen corresponding to the first selection operation, and step S302 generates a cleaning control instruction for the target area based on the input information, which may include:
[0074] The area corresponding to the real-time image is taken as the target area, and cleaning control instructions associated with the target area are generated.
[0075] Specifically, if the user directly selects the real-time image of the cleaning equipment through the first selection operation and generates descriptive information, that is, the real-time image can indicate the target area, then the control terminal can directly use the target area corresponding to the real-time image as the target area and generate cleaning control instructions.
[0076] (2) The description information is generated based on the user's second selection operation for multiple candidate images, wherein the candidate images are images pre-stored in the control terminal.
[0077] Among them, the candidate image can be generated based on historical images collected during the historical cleaning process of the cleaning equipment. For example, if certain targets are found to need to be bypassed during the historical cleaning process, the image of the obstacle avoidance target that needs to be bypassed can be extracted from the historical image to generate the obstacle avoidance image; the candidate image can also be an image of the obstacle avoidance target directly provided in advance in the control terminal.
[0078] (3) Generate the description information based on the first image acquired by the control terminal.
[0079] Specifically, the control terminal can call the image acquisition module, such as the camera of the control terminal, through the control application to acquire images in real time and obtain descriptive information; or it can obtain images pre-stored in the control terminal through the control application to obtain descriptive information.
[0080] In other words, users can obtain descriptive information by capturing the first image in real time, or they can use the first image that has already been captured, i.e., the preset image, as descriptive information.
[0081] In the above embodiments, the user can directly select the real-time image of the cleaning equipment through the first selection operation and generate descriptive information. Then the control terminal can directly use the area corresponding to the real-time image as the target area and generate cleaning control instructions. The user can also generate descriptive information by selecting an image from candidate images. The user can also generate descriptive information from the first image collected in real time by the control terminal, or generate descriptive information from the first image already collected by the control terminal. The target area can be described in the form of images through multiple acquisition methods, which not only makes the operation more convenient, but also allows for more flexible indication of the target area.
[0082] In some possible implementations, the descriptive information includes at least one of voice and text, and step S303 obtains user input information, including any of the following:
[0083] (1) Receive description information input by the user;
[0084] (2) The description information is generated by receiving a third selection operation from the user for multiple reference description texts, wherein the reference description texts are description texts pre-stored in the control terminal.
[0085] Specifically, for the description information, the user can directly input it; or multiple reference description texts can be displayed on the control terminal's display interface to prompt the user, who can then input the description information based on these texts; or multiple reference description texts can be displayed on the control terminal's display interface as user candidates, from which the user can directly select the description information.
[0086] In practice, the methods also include any of the following:
[0087] Use the preset description text as the reference description text;
[0088] Generate reference description text based on historical images from the cleaning process of the cleaning equipment;
[0089] Use the user-defined description text as the reference description text.
[0090] In other words, the default description text can be used as the reference description text; the reference description text can be generated based on historical screens for users to refer to; or the user can customize the reference description text.
[0091] In the above embodiments, users can generate reference description text based on historical images, that is, generate reference description text based on the cleaning area corresponding to the historical cleaning process to prompt users to input description text; users can also customize the generated description text, which can more flexibly define target areas, more conveniently generate control cleaning instructions, and more accurately clean or avoid target areas.
[0092] The above embodiments illustrate the process of obtaining input information. The following will further illustrate the specific process of generating cleaning control instructions and performing cleaning in conjunction with the embodiments.
[0093] In some possible implementations, step S302, which generates a cleaning control instruction associated with the target area based on the input information, may include:
[0094] (1) When the input information includes description information of the target area and cleaning instructions, a first control instruction is generated, which is used to instruct the cleaning equipment to clean the target area;
[0095] (2) When the input information includes the description information of the target area and the obstacle avoidance instruction, a second control instruction is generated, which is used to instruct the cleaning equipment to avoid the target area.
[0096] In other words, when the input information includes a cleaning instruction to instruct the cleaning equipment to clean, the cleaning equipment will perform the corresponding cleaning operation on the target area; when the input information includes an obstacle avoidance instruction to instruct the cleaning equipment to avoid the target area, the cleaning equipment will perform the corresponding operation to avoid the target area.
[0097] In some implementations, the cleaning equipment may also have different types of cleaning modes, such as dry cleaning (e.g., sweeping, vacuuming, and dust collection) and wet cleaning (e.g., mopping or washing). In this case, the instruction information may also include cleaning mode commands, such as dry cleaning commands and wet cleaning commands. Dry cleaning commands instruct the cleaning equipment to perform dry cleaning on the target area. Wet cleaning commands instruct the cleaning equipment to perform wet cleaning on the target area.
[0098] In some possible implementations, determining the target location information of the target area based on the input information can be performed by the cleaning equipment.
[0099] Specifically, step S302, which generates cleaning control instructions associated with the target area based on the input information, may include:
[0100] The input information is used as a cleaning control command, which instructs the cleaning equipment to perform a recognition operation based on the real-time image and the input information, obtain the target location information of the target area, and perform a cleaning operation based on the target location information.
[0101] Specifically, the cleaning equipment can be equipped with a recognition model. The control terminal can send the input information as a cleaning control command to the cleaning equipment. The cleaning equipment uses the recognition model to perform recognition operations based on the real-time image and the input information to obtain the target location information of the target area.
[0102] As shown in Figure 4, the user inputs information into the control terminal, which then sends the input information to the cleaning equipment. The cleaning equipment performs a recognition operation based on the real-time image and the input information to obtain the target location information of the target area. The cleaning equipment then cleans the target area based on the target location information.
[0103] In some other possible implementations, the determination of the target location information of the target area based on the input information can be performed by the control terminal.
[0104] Step S302 generates cleaning control instructions associated with the target area based on the input information, which may include:
[0105] Acquire real-time images captured by cleaning equipment;
[0106] The recognition operation is performed based on real-time images and input information to obtain the target location information of the target area;
[0107] Cleaning control instructions are generated based on the target location information and associated with the target area.
[0108] Specifically, the control terminal can be equipped with a recognition model. The control terminal acquires real-time images collected by the cleaning equipment and performs recognition operations based on the real-time images and input information through the recognition model to obtain the target location information of the target area.
[0109] As shown in Figure 5, the cleaning equipment sends real-time images to the control terminal; the control terminal performs recognition operations based on the real-time images and input information to obtain the target location information of the target area; the control terminal generates cleaning control instructions for the target area based on the target location information; the control terminal sends the cleaning control instructions to the cleaning equipment, and the cleaning equipment performs cleaning operations based on the target area.
[0110] In other words, regardless of whether the recognition model is set on the control terminal or the cleaning equipment, the process is as shown in Figure 6. The target location information of the target area is determined by the recognition model based on the real-time images and input information collected by the cleaning equipment.
[0111] The following examples illustrate the specific process by which the recognition model determines the target location information.
[0112] In some possible implementations, a recognition operation is performed based on real-time video and input information to obtain target location information of the target area, including:
[0113] (1) Perform recognition operations based on real-time images and input information to determine the first location information of the target area in the real-time images;
[0114] (2) Obtain depth information from real-time images captured by the cleaning equipment;
[0115] (3) Based on the depth information, the first position information is transformed into coordinates to determine the target position information of the target area in the actual scene.
[0116] Among them, the first position information of the target area in the real-time image is two-dimensional position information, and the target position information of the target area in the actual scene is three-dimensional position information.
[0117] In the specific implementation process, the input information may include at least one of text, voice and image. When voice is included, the voice is converted into text. As shown in Figure 7, the input information, namely the text shown in the figure (including directly input text or text converted from voice) and the image are input into the recognition model. The recognition model detects the first position information, namely the position of the detection box shown in the figure, based on the real-time image, namely the real-scene image shown in the figure. Then, through the depth sensor of the cleaning device, namely the depth detection module shown in the figure, the depth information of each corresponding pixel in the real-time image can be obtained. Based on the depth information, the first position information is transformed into coordinates to obtain the target position information of the target area in the actual scene.
[0118] Specifically, the first location information can include the two-dimensional coordinates (u, v) of the detection box in the real-time image, which can be converted using the following formula:
[0119] Where (u,v) are the coordinate information corresponding to the first position information, x, y, z are the target position information corresponding to the first position information, i.e., the three-dimensional position information, and z is the depth information corresponding to each pixel in the image. Through the above formula (1), the target position information of the target area in the actual scene can be determined based on the first position information.
[0120] In some possible implementations, the identification operation includes:
[0121] Extracting input features from the input information;
[0122] Target detection is performed on the real-time image based on input features to determine the first position information of the target area in the real-time image.
[0123] Specifically, the recognition model can include an encoder and a recognition module, as shown in Figure 8. The encoder extracts the input features of the input information, and then the input features and the real-time image are input into the recognition module (the open object recognition model shown in the figure) to obtain the two-dimensional position of the target area in the real-time image, i.e., the detection box shown in the figure.
[0124] Specifically, the recognition module can be any module that can be used for open-world object detection. The recognition module can include, but is not limited to, the Randbox model. The Randbox model is an open-world object detection (OWOD) model based on the Fast R-CNN architecture, which aims to improve the detection capability of unknown objects. The Fast R-CNN architecture is a fast object detection method based on a region-based convolutional network.
[0125] The above embodiments illustrate the process of determining the target location information corresponding to the target area through a server set in a control terminal or cleaning equipment. The following will describe the process of determining the target location information through a server in conjunction with the embodiments.
[0126] In some possible implementations, step S302, which generates cleaning control instructions associated with the target area based on the input information, may include:
[0127] (1) Send the input information to the server so that the server can determine the target location information of the target area based on the historical images and input information of the historical cleaning process of the cleaning equipment.
[0128] Each historical scene has a corresponding historical description and historical location information.
[0129] In the specific implementation process, based on historical images and input information from the cleaning equipment's historical cleaning process, the target location information of the target area can be determined, which may include:
[0130] ① Obtain the target description text corresponding to the input information;
[0131] ② Match the target description text with multiple historical description texts to determine the historical location information corresponding to the historical description text that matches the target description text.
[0132] ③ Determine target location information based on matched historical location information
[0133] Specifically, during each historical cleaning process, the cleaning equipment can transmit the corresponding historical images and their corresponding historical location information to the server. Each time the server receives a historical image, it can generate a historical description text corresponding to that image.
[0134] Specifically, the server can use an image description model to generate location description information for historical images, thus obtaining historical description text.
[0135] The description information generation model may include an image feature extraction module and a text generation module. The image feature extraction module may include convolutional neural networks (CNNs), and the text generation module may include recurrent neural networks (RNNs) or long short-term memory networks (LSTMs), etc. This application does not limit the specific components of the model.
[0136] (2) Receive the target location information sent by the server and generate cleaning control instructions based on the target location information.
[0137] Specifically, the server can first send the target location information to the control terminal, and the control terminal can generate cleaning control instructions based on the target location information; alternatively, the server can directly send the target location information to the cleaning equipment, and the cleaning equipment can clean the target area based on the target location information.
[0138] In some possible implementations, after sending the cleaning control command to the cleaning device, the method further includes:
[0139] Create an operational environment map of the cleaning equipment.
[0140] Among them, the work environment map (e.g., SLAM map) is a digital map generated by cleaning equipment through real-time or offline scanning of the work environment (such as rooms, corridors, obstacle layout, etc.) using sensors and algorithms. The work environment map typically includes information such as environmental structure, obstacle location, and cleaned areas, which is used to guide the navigation and task planning of the cleaning equipment.
[0141] In related technologies, cleaning operations are performed after a work environment map is created; therefore, cleaning cannot be carried out before the map is created. In this application, the target area can be determined before the work environment map of the cleaning equipment is created, allowing for cleaning or obstacle avoidance of the target area, which improves the efficiency of selected area cleaning and enhances the user experience.
[0142] The aforementioned control method for the cleaning equipment involves the user inputting information into the control terminal. This input includes a description of the target area and can be in various formats such as voice, text, and images. The control terminal then generates cleaning control commands associated with the target area based on the input information and sends these commands to the cleaning equipment. This allows for more convenient indication of the target area's location, enabling the cleaning equipment to perform cleaning operations based on that area. The cleaning operation can include cleaning and obstacle avoidance; that is, the cleaning equipment responds to the cleaning control commands by cleaning the target area or avoiding it.
[0143] Furthermore, users can directly select the real-time image of the cleaning equipment through the first selection operation to generate descriptive information. The control terminal can then directly use the area corresponding to the real-time image as the target area to generate cleaning control instructions. Users can also generate descriptive information by selecting an image from candidate images. Users can also generate descriptive information from the first image captured in real time by the control terminal, or from the first image already captured by the control terminal. The target area can be described in the form of images through multiple acquisition methods, which not only makes the operation more convenient but also allows for more flexible indication of the target area.
[0144] Furthermore, users can generate reference description text based on historical images, that is, generate reference description text based on the cleaning area corresponding to the historical cleaning process to prompt users to enter description text; users can also customize the generated description text, which allows for more flexible definition of target areas, more convenient generation of control cleaning commands, and more precise cleaning of target areas or avoidance of target areas.
[0145] In some possible implementations, as shown in Figure 9, a cleaning method is provided, wherein the executing entity can be a cleaning device, specifically a controller of the cleaning device. Taking the controller of the cleaning device as an example, the method may include:
[0146] Step S901: Receive cleaning control command.
[0147] The cleaning control instructions are generated based on user input information, which includes a description of the target area, and the description information includes at least one of images, voice, or text.
[0148] Specifically, cleaning control commands can be sent from a control terminal.
[0149] Step S902: According to the cleaning control command, control the cleaning component of the cleaning equipment to perform cleaning operation on the target area.
[0150] In some possible implementations, the descriptive information includes images, and the descriptive information is acquired by the control terminal in any of the following ways:
[0151] The description information is generated based on the user's first selection operation for multiple real-time images, wherein the real-time images are those sent by the cleaning device.
[0152] The description information is generated based on a user's second selection operation on multiple candidate images, wherein the candidate images are images pre-stored in the control terminal;
[0153] The descriptive information is generated based on the first image acquired by the control terminal.
[0154] In some possible implementations, the descriptive information includes at least one of voice and text, and the descriptive information is obtained in any of the following ways:
[0155] Receive description information input by the user;
[0156] The description information is generated by receiving a third selection operation from a user for multiple reference description texts, wherein the reference description texts are description texts pre-stored in the control terminal.
[0157] In some possible implementations, the reference description text is generated by the control terminal based on any of the following methods:
[0158] Use the preset description text as the reference description text;
[0159] Generate reference description text based on historical images from the cleaning process of the cleaning equipment;
[0160] Use the user-defined description text as the reference description text.
[0161] In some possible implementations, controlling the cleaning components of the cleaning equipment to perform cleaning operations on the target area includes:
[0162] When the input information includes a description of the target area and a cleaning instruction, the cleaning device is controlled to clean the target area.
[0163] When the input information includes a description of the target area and an obstacle avoidance instruction, the cleaning component of the cleaning device is controlled to avoid the target area.
[0164] In some possible implementations, controlling the cleaning components of the cleaning equipment to perform cleaning operations on the target area includes:
[0165] Based on real-time images and input information, a recognition operation is performed to obtain the target location information of the target area, and the cleaning components of the cleaning equipment are controlled to clean the target area based on the target location information.
[0166] In some possible implementations, cleaning control commands are generated by the control terminal in the following manner:
[0167] Acquire real-time images captured by cleaning equipment;
[0168] Based on real-time images and input information, a recognition operation is performed to obtain the target location information of the target area;
[0169] Cleaning control instructions are generated based on the target location information and associated with the target area.
[0170] In some possible implementations, a recognition operation is performed based on real-time video and input information to obtain target location information of the target area, including:
[0171] Based on real-time images and input information, a recognition operation is performed to determine the first location information of the target area in the real-time images;
[0172] Acquire depth information from real-time images captured by cleaning equipment;
[0173] Based on depth information, the first location information is transformed into coordinates to determine the target location information of the target area in the actual scene.
[0174] In some possible implementations, the identification operation includes:
[0175] Extracting input features from the input information;
[0176] Target detection is performed on the real-time image based on input features to determine the first position information of the target area in the real-time image.
[0177] In some possible implementations, cleaning control commands are generated by the control terminal in the following manner:
[0178] The input information is sent to the server so that the server can determine the target location information of the target area based on the historical images and input information from the historical cleaning process of the cleaning equipment.
[0179] Receive target location information sent by the server and generate cleaning control instructions based on the target location information.
[0180] In some possible implementations, each historical image has corresponding historical description text and historical location information. Based on the historical images and input information from the historical cleaning process of the cleaning equipment, the target location information of the target area is determined, including:
[0181] Obtain the target description text corresponding to the input information;
[0182] The target description text is matched with multiple historical description texts to determine the historical location information corresponding to the historical description text that matches the target description text.
[0183] The target location information is determined based on the matched historical location information.
[0184] It should be noted that steps S901 and S902 can be performed before the cleaning equipment's operational environment map is created. For example, after step S902, the cleaning equipment can be controlled to perform a mapping operation. The mapping operation may include using various sensors to collect information such as passable areas, obstacle areas, and obstacle types in the operational environment, and then creating an operational environment map based on this information.
[0185] The cleaning methods and the control methods for the cleaning equipment described above are corresponding, the difference being the different entities performing the cleaning. Therefore, the details of the cleaning methods will not be elaborated here, but please refer to the control methods for the cleaning equipment described above.
[0186] In some possible implementations, as shown in FIG10, a control device 100 for a cleaning device is provided, comprising:
[0187] The first receiver 1001 is used to acquire user input information, the input information including description information of a target area, the description information including at least one of image, voice or text;
[0188] Processor 1002 is configured to generate cleaning control instructions associated with the target area based on the input information, the cleaning control instructions being used to instruct the cleaning equipment to perform cleaning operations on the target area;
[0189] Transmitter 1003 is used to send the cleaning control command to the cleaning equipment.
[0190] The first receiver 1001 is a hardware module in the control terminal used to acquire user input information (images, voice or text), and is responsible for converting physical input signals into digital signals for the processor to analyze.
[0191] Specifically, the first receiver may include a camera module or depth sensor for capturing images; it may also include a microphone array for capturing voice; and it may also include a touch screen or a physical / virtual keyboard for capturing text.
[0192] Among them, the processor 1002 is the core computing unit of the control terminal, responsible for parsing input information (images, voice, text) and generating cleaning control commands.
[0193] Specifically, the processor 1002 may include a main control SoC (System on Chip), etc.
[0194] Among them, transmitter 1003 is a hardware module in the control terminal that transmits cleaning control commands to the cleaning equipment, and it needs to support high-speed and stable wireless communication.
[0195] Specifically, the transmitter 1003 may include a Wi-Fi module, a Bluetooth module, or an NFC (Near Field Communication) chip, etc.
[0196] In some possible embodiments of this application, the descriptive information includes an image, and the first receiver 1001 is used for:
[0197] The description information is generated based on the user's first selection operation for multiple real-time images, wherein the real-time images are those sent by the cleaning device; or,
[0198] The description information is generated based on a user's second selection operation on multiple candidate images, wherein the candidate images are images pre-stored in the control terminal; or,
[0199] The descriptive information is generated based on the first image acquired by the control terminal.
[0200] In some possible embodiments of this application, the descriptive information includes at least one of voice and text, and the first receiver 1001 is used for:
[0201] Receive description information input by the user; or,
[0202] The description information is generated by receiving a third selection operation from a user for multiple reference description texts, wherein the reference description texts are description texts pre-stored in the control terminal.
[0203] In some possible embodiments of this application, the first receiver 1001 is further configured to:
[0204] Use the preset description text as the reference description text; or,
[0205] The reference description text is generated based on historical images from the cleaning process of the cleaning equipment; or...
[0206] Use the user-defined description text as the reference description text.
[0207] In some possible implementations of this application, the processor 1002 is used for:
[0208] When the input information includes description information of the target area and cleaning instructions, a first control instruction is generated, which is used to instruct the cleaning equipment to clean the target area.
[0209] When the input information includes a description of the target area and an obstacle avoidance instruction, a second control instruction is generated, which instructs the cleaning device to avoid the target area.
[0210] In some possible implementations of this application, the processor 1002 is used for:
[0211] The input information is used as a cleaning control command, which instructs the cleaning equipment to perform a recognition operation based on the real-time image and the input information to obtain the target location information of the target area, and to perform a cleaning operation based on the target location information.
[0212] In some possible implementations of this application, the processor 1002 is used for:
[0213] Acquire real-time images captured by the cleaning equipment;
[0214] Based on the real-time image and the input information, a recognition operation is performed to obtain the target location information of the target area;
[0215] Cleaning control instructions associated with the target area are generated based on the target location information.
[0216] In some possible implementations of this application, the processor 1002 is used for:
[0217] Based on the real-time image and the input information, a recognition operation is performed to determine the first location information of the target area in the real-time image;
[0218] Obtain depth information from the real-time images captured by the cleaning equipment;
[0219] Based on the depth information, the first location information is transformed into coordinates to determine the target location information of the target area in the actual scene.
[0220] In some possible embodiments of this application, the processor 1002 is used for:
[0221] Extract the input features of the input information;
[0222] Based on the input features, target detection is performed on the real-time image to determine the first position information of the target region in the real-time image.
[0223] In some possible embodiments of this application, the processor 1002 is used for:
[0224] The input information is sent to the server so that the server can determine the target location information of the target area based on the historical images of the cleaning process of the cleaning equipment and the input information.
[0225] Receive the target location information sent by the server;
[0226] The cleaning control command is generated based on the target location information.
[0227] In some possible embodiments of this application, the processor 1002 is used for:
[0228] Obtain the target description text corresponding to the input information;
[0229] The target description text is matched with multiple historical description texts to determine the historical location information corresponding to the historical description text that matches the target description text.
[0230] The target location information is determined based on the matched historical location information.
[0231] In some possible embodiments of this application, after the transmitter 1003 sends the cleaning control command to the cleaning device, the processor 1002 is further configured to:
[0232] Create an operational environment map of the cleaning equipment.
[0233] In some possible embodiments of this application, as shown in FIG11, a cleaning device 1100 is provided, comprising:
[0234] The second receiver 1101 is used to receive cleaning control instructions, wherein the cleaning control instructions are generated based on user input information, the input information including descriptive information of the target area, the descriptive information including at least one of image, voice or text;
[0235] The controller 1102 is used to control the cleaning components of the cleaning equipment to perform cleaning operations on the target area according to the cleaning control command.
[0236] The second receiver 1101 is a hardware module in the cleaning equipment used to receive cleaning control commands sent by a control terminal (such as a mobile phone). It needs to support multi-protocol communication to ensure reliable reception and parsing of commands.
[0237] Specifically, the second receiver 1101 may include a wireless communication module, such as a WIFI dual-mode chip, a Bluetooth dual-mode chip, etc.; the second receiver 1101 may also include a cellular network module, such as a 4G or 5G module; the second receiver 1101 may also include a near-field communication module, such as an NFC card reader, etc.
[0238] Among them, the controller 1102 is the core computing unit of the cleaning equipment, which is responsible for parsing the received instructions.
[0239] Specifically, the processor 1002 may include the main control SoC, etc.
[0240] In some possible implementations of this application, the controller is used for:
[0241] Based on the real-time image and the input information, a recognition operation is performed to obtain the target location information of the target area, and the cleaning components of the cleaning equipment are controlled to perform cleaning operations based on the target location information.
[0242] In some possible implementations of this application, the controller is used for:
[0243] Based on the real-time image and the input information, a recognition operation is performed to determine the first location information of the target area in the real-time image;
[0244] Obtain depth information from the real-time images captured by the cleaning equipment;
[0245] Based on the depth information, the first location information is transformed into coordinates to determine the target location information of the target area in the actual scene.
[0246] In some possible implementations of this application, the controller is used for:
[0247] Extract the input features of the input information;
[0248] Based on the input features, target detection is performed on the real-time image to determine the first position information of the target region in the real-time image.
[0249] In some possible implementations of this application, the controller is used for:
[0250] When the input information includes a description of the target area and a cleaning instruction, the cleaning device is controlled to clean the target area.
[0251] When the input information includes a description of the target area and an obstacle avoidance instruction, the cleaning component of the cleaning equipment is controlled to avoid the target area.
[0252] As shown in FIG12, in some possible embodiments of this application, a cleaning control system 1200 is provided, comprising:
[0253] The control terminal 1201 is used to acquire user input information, generate cleaning control instructions associated with the target area based on the input information, and send the cleaning control instructions to the cleaning equipment. The input information includes description information of the target area, and the description information includes at least one of image, voice or text.
[0254] Cleaning equipment 1202 is used to perform cleaning operations based on a target area in response to cleaning control commands.
[0255] In one optional embodiment, an electronic device is provided, as shown in FIG13. The electronic device 4000 shown in FIG13 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.
[0256] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.
[0257] Bus 4002 may include a pathway for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in Figure 13, but this does not indicate that there is only one bus or one type of bus.
[0258] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.
[0259] The memory 4003 stores computer programs that execute embodiments of this application, and its execution is controlled by the processor 4001. The processor 4001 executes the computer programs stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.
[0260] For example, the electronic device can be a control terminal or a cleaning device. When the electronic device is a control terminal, the processor 4001 is configured to execute steps S301 and S302. When the electronic device is a cleaning device, the processor 4001 is configured to execute steps S901 and S902.
[0261] This application provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it can implement the steps and corresponding content of the aforementioned method embodiments.
[0262] This application also provides a computer program product, including a computer program that, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.
[0263] The beneficial effects of the technical solutions provided in this application are:
[0264] Users input information into the control terminal, including a description of the area to be cleaned. This input can be in various formats, such as voice, text, and images. The control terminal then generates cleaning control commands associated with the target area based on the input information and sends these commands to the cleaning equipment. This allows for more convenient indication of the target area's location, enabling the cleaning equipment to perform cleaning operations based on that area. Cleaning operations can include cleaning and obstacle avoidance; that is, the cleaning equipment responds to the cleaning control commands by cleaning the target area or avoiding it.
[0265] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.
[0266] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.
Claims
1. A control method of a cleaning apparatus, wherein, A control terminal for controlling a cleaning device, comprising: obtaining input information of a user, the input information comprising description information of a target area, the description information comprising at least one of an image, a voice or a text; generating a cleaning control instruction associated with the target area based on the input information, the cleaning control instruction being used to instruct the cleaning device to perform a cleaning operation on the target area; sending the cleaning control instruction to the cleaning device.
2. The method of claim 1, wherein, The description information comprises an image, and the obtaining of the input information of the user comprises: generating the description information according to a first selection operation of the user on a plurality of real-time pictures, the real-time pictures being real-time pictures sent by the cleaning device; or generating the description information according to a second selection operation of the user on a plurality of candidate images, the candidate images being images pre-stored in the control terminal; or generating the description information according to a first image collected by the control terminal.
3. The method of claim 1, wherein, The description information comprises at least one of a voice and a text, and the obtaining of the input information of the user comprises: receiving the description information input by the user; or generating the description information according to a third selection operation of the user on a plurality of reference description texts, the reference description texts being description texts pre-stored in the control terminal.
4. The method of claim 3, wherein, The method further comprises: taking a preset description text as the reference description text; or generating the reference description text based on historical pictures in a historical cleaning process of the cleaning device; or taking a description text set by the user as the reference description text.
5. The method of claim 1, wherein, The generating of the cleaning control instruction associated with the target area based on the input information comprises: when the input information comprises description information of the target area and a cleaning instruction, generating a first control instruction, the first control instruction being used to instruct the cleaning device to clean the target area; or when the input information comprises description information of the target area and an obstacle avoidance instruction, generating a second control instruction, the second control instruction being used to instruct the cleaning device to avoid the target area.
6. The method of claim 1, wherein, The generating of the cleaning control instruction associated with the target area based on the input information comprises: taking the input information as the cleaning control instruction, the cleaning control instruction being used to instruct the cleaning device to perform an identification operation based on a real-time picture and the input information to obtain target position information of the target area, and perform a cleaning operation based on the target position information.
7. The method of claim 1, wherein, The generating of the cleaning control instruction associated with the target area based on the input information comprises: obtaining a real-time picture collected by the cleaning device; performing an identification operation based on the real-time picture and the input information to obtain target position information of the target area; and generating the cleaning control instruction associated with the target area based on the target position information.
8. The method of claim 6 or 7, wherein, The performing of the identification operation based on a real-time picture and the input information to obtain target position information of the target area comprises: performing the identification operation based on the real-time picture and the input information to determine first position information of the target area in the real-time picture; obtaining depth information of the real-time picture collected by the cleaning device; and The first position information is coordinate-converted based on the depth information to determine the target position information of the target area in an actual scene.
9. The method of claim 8, wherein, The identification operation includes: extracting input features of the input information; performing target detection on the real-time image based on the input features to determine first position information of the target area in the real-time image.
10. The method of claim 1, wherein, The generating of the cleaning control instruction associated with the target area based on the input information includes: sending the input information to a server to enable the server to determine target position information of the target area based on historical pictures in a historical cleaning process of the cleaning device and the input information; receiving the target position information sent by the server; generating the cleaning control instruction based on the target position information.
11. The method of claim 10, wherein, The determination of the target position information of the target area based on the historical pictures in the historical cleaning process of the cleaning device and the input information includes: obtaining a target description text corresponding to the input information; matching the target description text with a plurality of historical description texts respectively to determine historical position information corresponding to the historical description text matched with the target description text; determining the target position information based on the matched historical position information.
12. The method of claim 1, wherein, After the cleaning control instruction is sent to the cleaning device, the method further includes: establishing a working environment map of the cleaning device.
13. A cleaning method wherein, A cleaning device includes: receiving a cleaning control instruction, wherein the cleaning control instruction is generated based on input information of a user, and the input information includes description information of a target area, and the description information includes at least one of an image, a voice, or a text; controlling a cleaning component of the cleaning device to perform a cleaning operation on the target area according to the cleaning control instruction.
14. The method of claim 13, wherein, The cleaning control instruction includes the input information, and the controlling of the cleaning component of the cleaning device to perform the cleaning operation on the target area according to the cleaning control instruction includes: performing an identification operation based on a real-time image and the input information to obtain target position information of the target area, and controlling the cleaning component of the cleaning device to perform a cleaning operation based on the target position information.
15. The method of claim 14, wherein, The performing of the identification operation based on the real-time image and the input information to obtain the target position information of the target area includes: performing an identification operation based on the real-time image and the input information to determine first position information of the target area in the real-time image; obtaining depth information of the real-time image collected by the cleaning device; coordinate-converting the first position information based on the depth information to determine the target position information of the target area in an actual scene.
16. The method of claim 15, wherein, The identification operation includes: extracting input features of the input information; performing target detection on the real-time image based on the input features to determine first position information of the target area in the real-time image.
17. The method of claim 13, wherein, The controlling of the cleaning component of the cleaning device to perform the cleaning operation based on the target area includes: when the input information includes description information and a cleaning instruction of the target area, controlling the cleaning component of the cleaning device to clean the target area; When the input information comprises the description information of the target area and the obstacle avoidance instruction, the cleaning part of the cleaning device is controlled to avoid the target area.
18. A control device for a cleaning apparatus, wherein, Comprise: The first receiver is configured to obtain input information of a user, the input information comprising description information of a target area, the description information comprising at least one of an image, a voice or a text; The processor is configured to generate a cleaning control instruction associated with the target area based on the input information, the cleaning control instruction being used to instruct the cleaning device to perform a cleaning operation on the target area; The transmitter is configured to send the cleaning control instruction to the cleaning device.
19. The apparatus of claim 18, wherein, The description information comprises an image, and the first receiver is configured to: generate the description information according to a first selection operation of the user on a plurality of real-time pictures, the real-time pictures being real-time pictures sent by the cleaning device; or, generate the description information according to a second selection operation of the user on a plurality of candidate images, the candidate images being images pre-stored in the control terminal; or, generate the description information according to a first image collected by the control terminal.
20. The apparatus of claim 18, wherein, The description information comprises at least one of a voice and a text, and the first receiver is configured to: receive the description information input by the user; or, generate the description information according to a third selection operation of the user on a plurality of reference description texts, the reference description texts being description texts pre-stored in the control terminal.
21. The apparatus of claim 20, wherein, The first receiver is further configured to: use a preset description text as the reference description text; or, generate the reference description text based on historical pictures in a historical cleaning process of the cleaning device; or, use a description text set by the user as the reference description text.
22. The apparatus of claim 18, wherein, The processor is configured to: generate a first control instruction when the input information comprises the description information of the target area and a cleaning instruction, the first control instruction being used to instruct the cleaning device to clean the target area; generate a second control instruction when the input information comprises the description information of the target area and an obstacle avoidance instruction, the second control instruction being used to instruct the cleaning device to avoid the target area.
23. The apparatus of claim 18, wherein, The processor is configured to: use the input information as a cleaning control instruction, the cleaning control instruction being used to instruct the cleaning device to perform an identification operation based on a real-time picture and the input information, to obtain target position information of a target area, and perform a cleaning operation based on the target position information.
24. The apparatus of claim 18, wherein, The processor is configured to: obtain a real-time picture collected by the cleaning device; perform an identification operation based on the real-time picture and the input information, to obtain target position information of a target area; generate a cleaning control instruction associated with the target area based on the target position information.
25. The apparatus of claim 23 or 24, wherein, The processor is configured to: perform an identification operation based on the real-time picture and the input information, to determine first position information of the target area in the real-time picture; obtain depth information of the real-time picture collected by the cleaning device; perform coordinate conversion on the first position information based on the depth information, to determine the target position information of the target area in an actual scene.
26. The apparatus of claim 25, wherein, The processor is configured to: extract input features of the input information; perform target detection on the real-time picture based on the input feature, and determine first position information of the target region in the real-time picture.
27. The apparatus of claim 18, wherein, The processor is configured to: send the input information to a server, so that the server determines target position information of the target region based on historical pictures in a historical cleaning process of the cleaning device and the input information; receive the target position information sent by the server; generate the cleaning control instruction based on the target position information.
28. The apparatus of claim 27, wherein, The processor is configured to: obtain a target description text corresponding to the input information; match the target description text with a plurality of historical description texts respectively, to determine historical position information corresponding to a historical description text matched with the target description text; determine the target position information based on the matched historical position information.
29. The apparatus of claim 18, wherein, After the transmitter sends the cleaning control instruction to the cleaning device, the processor is further configured to: establish a working environment map of the cleaning device.
30. A cleaning device, wherein, comprise: a second receiver configured to receive a cleaning control instruction, wherein the cleaning control instruction is generated based on input information of a user, the input information comprising description information of a target region, the description information comprising at least one of an image, a voice, or a text; a controller configured to control a cleaning component of a cleaning device to perform a cleaning operation on a target region according to the cleaning control instruction.
31. The apparatus of claim 30, wherein, The cleaning control instruction comprises the input information, and the controller is configured to: perform an identification operation based on the real-time picture and the input information, to obtain target position information of the target region, and control the cleaning component of the cleaning device to perform a cleaning operation based on the target position information.
32. The apparatus of claim 31, wherein, The controller is configured to: perform an identification operation based on the real-time picture and the input information, to determine first position information of the target region in the real-time picture; obtain depth information of the real-time picture collected by the cleaning device; perform coordinate conversion on the first position information based on the depth information, to determine the target position information of the target region in an actual scene.
33. The apparatus of claim 32, wherein, The controller is configured to: extract input features of the input information; perform target detection on the real-time picture based on the input feature, and determine first position information of the target region in the real-time picture.
34. The apparatus of claim 30, wherein, The controller is configured to: when the input information comprises description information of the target region and a cleaning instruction, control a cleaning component of a cleaning device to clean the target region; when the input information comprises description information of the target region and an obstacle avoidance instruction, control a cleaning component of a cleaning device to avoid the target region.
35. A cleaning control system, wherein, comprise: a control terminal configured to obtain input information of a user, generate a cleaning control instruction associated with a target region based on the input information, and send the cleaning control instruction to a cleaning device, wherein the input information comprises description information of a target region, and the description information comprises at least one of an image, a voice, or a text; a cleaning device configured to perform a cleaning operation based on the target region in response to the cleaning control instruction.
36. An electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein, The processor executes the computer program to implement the steps of the method of any one of claims 1-12 or 13-17.
37. A computer readable storage medium having stored thereon a computer program, wherein, The computer program, which when executed by the processor implements the steps of the method of any one of claims 1-12 or 13-17.
38. A computer program product comprising a computer program, wherein, The computer program, which when executed by the processor implements the steps of the method of any one of claims 1-12 or 13-17.
