Method and apparatus for operating cleaning device, storage medium and electronic device

Abstract

The application provides a cleaning device operation control method and device, a storage medium and an electronic device. The method comprises the following steps: obtaining virtual forbidden area information corresponding to a target area map, the target area map being an area map to which a to-be-cleaned area to be cleaned by a cleaning device belongs, and the virtual forbidden area information being used for indicating a virtual forbidden area in the area map; performing target detection through a first sensor on the cleaning device to obtain target object information, the target object information being used for representing a target scene object in a current area where the cleaning device is located and matching the virtual forbidden area; in a situation that the cleaning device is trapped during cleaning of the to-be-cleaned area, performing a target escape operation on the cleaning device according to the virtual forbidden area information and the target object information, and the cleaning device after escaping being located outside the virtual forbidden area. Through the application, the problem that the cleaning device is easily trapped due to virtual forbidden area establishment errors in the related art is solved.

Description

[Technical Field]

[0001] This application relates to the field of smart homes, and more specifically, to a method and apparatus for controlling the operation of a cleaning device, a storage medium, and an electronic device. [Background Technology]

[0002] Currently, users' devices can run applications compatible with cleaning equipment (e.g., cleaning robots). Users can configure virtual restricted areas on a map displayed in the application's configuration interface to set areas where the cleaning equipment is allowed and prohibited from cleaning. Furthermore, cleaning equipment can also create virtual restricted areas based on its history of being trapped.

[0003] For example, users can set up virtual walls between rooms to restrict which areas are allowed to be cleaned by cleaning robots and which are not. When a cleaning robot gets stuck under furniture due to a stuck surface, it can designate the area where the furniture is located as a virtual no-go zone based on its history of being stuck.

[0004] However, the area map displayed on the terminal device differs significantly in scale from the actual room scene, resulting in errors in the established virtual no-go zones. Furthermore, the virtual no-go zones created by the cleaning device itself also contain errors due to limited information. These errors in the establishment of virtual no-go zones make the cleaning device prone to getting stuck, thus reducing the user experience.

[0005] Therefore, it can be seen that the operation control methods of cleaning equipment in related technologies have the problem of cleaning equipment being easily trapped due to errors in the establishment of virtual restricted areas. [Summary of the Invention]

[0006] The purpose of this application is to provide a method and apparatus for controlling the operation of cleaning equipment, a storage medium and an electronic device, so as to at least solve the problem that the operation control methods of cleaning equipment in the related art are prone to getting the cleaning equipment trapped due to errors in the establishment of virtual restricted areas.

[0007] The purpose of this application is to achieve the following technical solution:

[0008] According to one aspect of the embodiments of this application, a method for controlling the operation of a cleaning device is provided, comprising: acquiring virtual restricted area information corresponding to a target area map, wherein the target area map is an area map to which the cleaning device belongs, and the virtual restricted area information is used to indicate virtual restricted areas in the area map; performing target detection through a first sensor on the cleaning device to obtain target object information, wherein the target object information is used to represent a target scene object in the current area where the cleaning device is located that matches the virtual restricted area; and, if the cleaning device becomes trapped during the cleaning of the area to be cleaned, controlling the cleaning device to perform a target escape operation according to the virtual restricted area information and the target object information, wherein the cleaning device, after escaping, is outside the virtual restricted area.

[0009] In an exemplary embodiment, the step of detecting a target using a first sensor on the cleaning device to obtain target object information includes: performing target recognition on point cloud data collected by the first sensor to obtain the target object information, wherein the target object information is the object point cloud of the target scene object.

[0010] In an exemplary embodiment, the step of performing target recognition on the point cloud data collected by the first sensor to obtain the target object information includes: performing target recognition on the point cloud data collected by the first sensor to obtain candidate object information, wherein the candidate object information is the object point cloud of candidate objects contained in the current area; and selecting the target scene object that matches the virtual restricted area from the candidate objects according to the location information of the candidate objects and the virtual restricted area information to obtain the target object information.

[0011] In an exemplary embodiment, controlling the cleaning device to perform a target escape operation based on the virtual restricted area information and the target object information includes: when the target scene object is determined to be a scene object that the cleaning device is allowed to cross or overcome obstacles based on the target object information, and the cleaning device passes through the virtual wall of the virtual restricted area and enters an area in the target area map other than the area to be cleaned, controlling the cleaning device to cross the target scene object and enter the area to be cleaned.

[0012] In an exemplary embodiment, controlling the cleaning device to perform a target escape operation based on the virtual restricted area information and the target object information includes: when the target scene object is determined to be a scene object of a target type based on the target object information, and the cleaning device has passed through the virtual wall of the virtual restricted area and entered the target object area where the target scene object is located, controlling the cleaning device to perform a target escape operation matching the target type, wherein the target type includes at least one of the following: a type where the bottom does not allow the cleaning device to pass through, or a type where the distance between the device and the wall is less than or equal to a distance threshold.

[0013] In an exemplary embodiment, controlling the cleaning device to perform the target escape operation matching the target type includes: when the target type includes a type where the bottom does not allow the cleaning device to pass, acquiring point cloud data through a second sensor on the cleaning device to obtain target point cloud data; identifying an exit matching the movement trajectory of the cleaning device entering the target object area based on the target point cloud data, wherein the size of the exit allows the cleaning device to pass; controlling the cleaning device to move out of the target object area from the exit along the movement trajectory; when the target type includes a type where the distance between the target and the wall is less than or equal to a distance threshold, controlling the cleaning device to move along a target boundary detected by the distance sensor of the cleaning device until it moves out of the target object area, wherein the target boundary is at least one of the following: the wall, the boundary of the target scene object.

[0014] In an exemplary embodiment, controlling the cleaning device to perform a target escape operation based on the virtual restricted area information and the target object information includes: when the cleaning device is trapped due to detecting a cliff or detecting that the wheels of the cleaning device are in a falling state, controlling the cleaning device to perform a first escape operation, wherein the first escape operation is used to control the cleaning device to leave the detected cliff or control the wheels to escape the falling state, and during the execution of the first escape operation, the cleaning device ignores the virtual restricted area; after the first escape operation is completed, if it is detected that the cleaning device has passed through the virtual wall of the virtual restricted area, controlling the cleaning device to perform a second escape operation, wherein the second escape operation is used to control the cleaning device to leave the range of the virtual restricted area.

[0015] In one exemplary embodiment, after target detection via a first sensor on the cleaning device, the method further includes: if the boundary of the target scene object is within the area to be cleaned, controlling the cleaning device to clean along the boundary of the target scene object; and if the boundary of the target scene object is outside the area to be cleaned, controlling the cleaning device to clean along the virtual wall of the virtual restricted area.

[0016] According to another aspect of the embodiments of this application, an operation control device for a cleaning device is also provided, comprising: a first acquisition unit, configured to acquire virtual restricted area information corresponding to a target area map, wherein the target area map is an area map to which the area to be cleaned by the cleaning device belongs, and the virtual restricted area information is used to indicate virtual restricted areas in the area map; a detection unit, configured to perform target detection through a first sensor on the cleaning device to obtain target object information, wherein the target object information is used to represent a target scene object in the current area where the cleaning device is located that matches the virtual restricted area; and a first control unit, configured to control the cleaning device to perform a target escape operation according to the virtual restricted area information and the target object information when the cleaning device is trapped during the cleaning of the area to be cleaned, wherein the cleaning device after escape is outside the virtual restricted area.

[0017] In an exemplary embodiment, the detection unit includes: a first identification module, configured to perform target identification on the point cloud data collected by the first sensor to obtain the target object information, wherein the target object information is the object point cloud of the target scene object.

[0018] In an exemplary embodiment, the first identification module includes: an identification submodule, configured to perform target identification on point cloud data collected by the first sensor to obtain candidate object information, wherein the candidate object information is an object point cloud of candidate objects contained in the current area; and a selection submodule, configured to select the target scene object matching the virtual restricted area from the candidate objects according to the location information of the candidate objects and the virtual restricted area information, to obtain the target object information.

[0019] In an exemplary embodiment, the first control unit includes: a first control module, configured to control the cleaning device to cross the target scene object and enter the area to be cleaned when the target scene object is determined to be a scene object that the cleaning device is allowed to cross based on the target object information, and the cleaning device passes through the virtual wall of the virtual restricted area and enters an area in the target area map other than the area to be cleaned.

[0020] In an exemplary embodiment, the first control unit includes: a second control module, configured to control the cleaning device to perform a target escape operation matching the target type when the target scene object is determined to be a scene object of a target type based on the target object information, and the cleaning device has passed through the virtual wall of the virtual restricted area and entered the target object area where the target scene object is located, wherein the target type includes at least one of the following: a type where the bottom does not allow the cleaning device to pass through, and a type where the distance between the device and the wall is less than or equal to a distance threshold.

[0021] In an exemplary embodiment, the first control unit includes: a data acquisition module, configured to acquire point cloud data from a second sensor on the cleaning device to obtain target point cloud data when the target type includes a type where the bottom does not allow the cleaning device to pass; a second identification module, configured to identify an exit matching the movement trajectory of the cleaning device entering the target object area based on the target point cloud data, wherein the size of the exit allows the cleaning device to pass; a third control module, configured to control the cleaning device to move out of the target object area from the exit along the movement trajectory; and a fourth control module, configured to control the cleaning device to move along a target boundary detected by a distance sensor on the cleaning device until it moves out of the target object area when the target type includes a type where the distance between the target and a wall is less than or equal to a distance threshold, wherein the target boundary is at least one of the following: the wall, or the boundary of the target scene object.

[0022] In one exemplary embodiment, the first control unit includes: a fifth control module, configured to control the cleaning device to perform a first escape operation when the cleaning device is trapped due to detecting a cliff or detecting that the wheels of the cleaning device are in a falling state, wherein the first escape operation is used to control the cleaning device to leave the detected cliff or control the wheels to escape the falling state, and during the execution of the first escape operation, the cleaning device ignores the virtual restricted area; and a sixth control module, configured to control the cleaning device to perform a second escape operation after the first escape operation is completed and the cleaning device is detected to have passed through the virtual wall of the virtual restricted area, wherein the second escape operation is used to control the cleaning device to leave the area of ​​the virtual restricted area.

[0023] In one exemplary embodiment, the apparatus further includes: a second control unit, configured to control the cleaning device to clean along the boundary of the target scene object when the boundary of the target scene object is located within the area to be cleaned after target detection by the first sensor on the cleaning device; and a third control unit, configured to control the cleaning device to clean along the virtual wall of the virtual restricted area when the boundary of the target scene object is located outside the area to be cleaned.

[0024] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program, and the computer program is configured to execute the test method of the above-described interface at runtime.

[0025] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the test method of the interface described above through the computer program.

[0026] In this embodiment, a method combining real-world escape scenarios with virtual restricted area information is employed. Virtual restricted area information corresponding to a target area map is obtained. The target area map is the map of the area to be cleaned by the cleaning equipment, and the virtual restricted area information indicates the virtual restricted areas within the area map. Target detection is performed using a first sensor on the cleaning equipment to obtain target object information, which represents a target scene object within the current area of ​​the cleaning equipment that matches the virtual restricted area. If the cleaning equipment becomes trapped during the cleaning process, it is controlled to perform a target escape operation based on the virtual restricted area information and the target object information. After escaping, the cleaning equipment is outside the virtual restricted area. By combining virtual restricted areas and real-world scenarios for escape, the escape process becomes more intelligent, achieving the technical effect of reducing the entrapment rate of the cleaning equipment and improving the user experience. This solves the problem in related technologies where the cleaning equipment's operation control methods are prone to entrapment due to errors in establishing virtual restricted areas. [Attached Image Description]

[0027] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0028] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the hardware environment of an optional cleaning equipment operation control method according to an embodiment of this application;

[0030] Figure 2 This is a flowchart illustrating an optional operation control method for a cleaning device according to an embodiment of this application;

[0031] Figure 3 This is a schematic diagram of an optional virtual restricted area according to an embodiment of this application;

[0032] Figure 4 This is a schematic diagram of another optional virtual restricted area according to an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of yet another optional virtual restricted area according to an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of an optional slide rail according to an embodiment of this application;

[0035] Figure 7 This is a structural block diagram of an optional operation control device for a cleaning equipment according to an embodiment of this application;

[0036] Figure 8 This is a structural block diagram of an optional electronic device according to an embodiment of this application.

Detailed Implementation Methods

[0037] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0038] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0039] According to one aspect of the embodiments of this application, a method for controlling the operation of a cleaning device is provided. Optionally, in this embodiment, the above-described method for controlling the operation of a cleaning device can be applied to, for example... Figure 1 The hardware environment shown consists of terminal device 102, cleaning device 104, and server 106. For example... Figure 1As shown, terminal device 102 can connect to cleaning device 104 and / or server 106 (e.g., IoT platform or cloud server) via a network to control cleaning device 104, such as binding to cleaning device 104 and configuring the cleaning functions of cleaning device 104. Cleaning device 104 may include a host and a base station (e.g., a sweeper and a base station, a cleaning machine and a base). The host and the base station can connect via a network to determine the current status of the other end (e.g., battery status, working status, location information, etc.).

[0040] The aforementioned network may include, but is not limited to, at least one of the following: wired network, wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity), Bluetooth, infrared. The network used by the terminal device 102 to communicate with the cleaning device 104 and / or the server 106 may be the same as or different from the network used by the cleaning device 104 to communicate with the server 106. The terminal device 102 is not limited to PCs, mobile phones, tablets, etc. The cleaning device 104 may include, but is not limited to, self-cleaning robots, such as automatic mop-washing robots, sweeping robots, etc. The server 106 may be a server of an IoT platform.

[0041] The operation control method for the cleaning equipment in this embodiment can be executed by a single terminal device 102, cleaning device 104, or server 106, or by at least two of them. Alternatively, the terminal device 102 or cleaning device 104 can execute the operation control method for the cleaning equipment in this embodiment by a client installed on it.

[0042] Taking the cleaning equipment 104 as an example to execute the operation control method of the cleaning equipment in this embodiment, Figure 2 This is a flowchart illustrating an optional operation control method for cleaning equipment according to an embodiment of this application, as shown below. Figure 2 As shown, the process of this method may include the following steps:

[0043] Step S202: Obtain virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs, and the virtual restricted area information is used to indicate the virtual restricted areas in the area map.

[0044] The cleaning equipment operation control method in this embodiment can be applied to the following scenarios: during the cleaning process of the cleaning equipment on the area to be cleaned in the target area, the operation control of the cleaning equipment is combined with the real scene and the virtual restricted area to reduce the probability of the cleaning equipment getting stuck. The target area can be an indoor area in a home, or other areas such as a restaurant, office, factory workshop, or other areas that can be cleaned by the cleaning equipment. The cleaning equipment can be a smart vacuum cleaner, a smart sweeping robot, a smart sweeping and mopping robot, or other robots with cleaning functions; this embodiment does not limit the scope of the application.

[0045] The area currently to be cleaned by the cleaning equipment is the area to be cleaned. Before or after starting area cleaning, the cleaning equipment can obtain a target area map corresponding to the target area. The target area map is the area map to which the area to be cleaned belongs, and the area to be cleaned can be all or part of the target area. Based on the set cleaning area information, the cleaning equipment can determine the area to be cleaned. The cleaning area information can be default area information, such as defaulting to cleaning all or a specific part of the target area, or it can be cleaning area information generated by the terminal device in response to detecting a selection operation performed on the target area map. The cleaning area information can be sent to the cleaning equipment through the server. The selection operation can be the operation of selecting the sub-area to be cleaned from multiple sub-areas contained in the target area.

[0046] The target area map can be a map formed by area images or area point clouds acquired by the cleaning equipment using image acquisition components (such as cameras or laser sensors), or it can be a map obtained through other means, such as a map received from the terminal device, which can be a map formed by area images or area point clouds acquired by image acquisition components on other devices. This embodiment does not limit this.

[0047] The target area map may also include virtual restricted area information, which the cleaning equipment can also access. This virtual restricted area information is used to indicate virtual restricted areas on the target area map. A virtual restricted area refers to an area marked by virtual walls, restricted area lines, etc., that prohibits the cleaning equipment from entering. Here, the virtual walls and restricted area lines are virtual area boundaries, and the cleaning equipment can pass through the virtual walls and restricted area lines to enter the virtual restricted area. The virtual restricted area can be a restricted area established by the user on the target area map displayed on their terminal device, or it can be a restricted area established by the cleaning equipment based on historical entrapment records. In this embodiment, the method of establishing virtual restricted areas is not limited.

[0048] For example, to prevent cleaning equipment from being damaged or unable to continue working due to falls or getting stuck under furniture, virtual restricted areas can be set up at stairwells or furniture locations on the room area map.

[0049] Step S204: Target detection is performed using the first sensor on the cleaning equipment to obtain target object information, wherein the target object information is used to represent the target scene object that matches the virtual restricted area within the current area where the cleaning equipment is located.

[0050] To avoid the cleaning equipment getting trapped due to discrepancies between the established virtual restricted area and the real scene, the cleaning equipment can use its first sensor to detect targets and determine the scene objects (i.e., scene objects) located within the current area where the cleaning equipment is located. Based on the location information of the virtual restricted area and the location information of the detected scene objects, the target scene object matching the virtual restricted area can be determined, and thus the object information of the target scene object, i.e., the target object information, can be obtained.

[0051] The first sensor can be an image acquisition device or other devices with object recognition capabilities. For example, the first sensor can be an LDS (Laser Direct Structuring) laser rangefinder, which can acquire point cloud data by emitting lasers and determine the target object information based on the acquired point cloud data. The first sensor can be deployed on the outside (e.g., front, left, right, rear, etc.), top, or bottom of the cleaning equipment, and there can be one or more of them. One or more sensors can be deployed on different sides of the cleaning equipment.

[0052] The current area where the cleaning device is located is the collection area of ​​the first sensor, which represents the range of areas it can collect. It can belong to the area to be cleaned (at this time, the cleaning device can be located in the area to be cleaned), or it can belong to other areas in the target area other than the area to be cleaned (at this time, the cleaning device can enter other areas through virtual restricted areas, etc.), or it can belong partly to the area to be cleaned and partly to other areas. This embodiment does not limit this.

[0053] In this embodiment, the cleaning device can perform target recognition (scene object recognition) on the scene data collected by the first sensor to determine the detected scene objects. The cleaning device can match the collected scene data using the reference object features of various reference objects and virtual restricted area information to determine the target scene objects that match various reference objects, thereby obtaining the object information of all target scene objects in the current area, i.e., the target object information.

[0054] It should be noted that multiple reference objects to be identified (i.e., the objects referenced for target identification) and reference object features for each reference object can be pre-configured. Each reference object can be a scene object of a predetermined object type, and its corresponding reference object features can be pre-stored in the cleaning equipment or on the server side for target identification by the cleaning equipment or the server side. For different area types, the corresponding predetermined object types can be the same or different. This embodiment does not impose any limitations on this.

[0055] For example, a robot vacuum cleaner can use its sensors to detect obstacles and identify those that match a virtual no-go zone, such as steps, tracks, or dark carpets.

[0056] Step S206: If the cleaning equipment gets stuck during the cleaning of the area to be cleaned, the cleaning equipment is controlled to perform a target escape operation based on the virtual restricted area information and the target object information. After escaping, the cleaning equipment is outside the virtual restricted area.

[0057] During the cleaning process, the cleaning equipment may become stuck in a certain location due to obstacle avoidance (i.e., avoiding obstacles). This location can be within the area to be cleaned or in a different location within the target area. For example, the cleaning equipment might pass through a virtual wall into an area other than the area to be cleaned, get stuck under furniture, trigger a downward view, or fall. Once stuck, the cleaning equipment can use its configured escape strategy to free itself.

[0058] Due to the lack of integration and correction between the virtual restricted areas and the real-world scenario, the machine (i.e., the cleaning equipment) cannot understand the true purpose of establishing the virtual restricted areas. Furthermore, in complex scenarios composed of real escape scenarios (obstacles, steps, narrow areas, etc.) combined with virtual restricted areas, on the one hand, the machine perceives the virtual restricted areas as real walls and cannot escape from narrow areas; on the other hand, escaping in complex scenarios may cause the machine to mistakenly enter or pass through restricted areas, triggering an alarm indicating it is trapped. Therefore, the escape solutions for cleaning equipment in related technologies cannot identify escape scenarios in both real-world complex scenarios and virtual restricted areas, and they also lack corresponding escape methods and remedial solutions.

[0059] It is evident that in complex escape scenarios comprised of real-world escape scenarios and virtual restricted areas, the cleaning equipment's escape strategies and post-entrapment recovery plans are inadequate, exhibiting a lack of overall intelligence and exhibiting issues with misidentification of escape scenarios (e.g., accidental triggering of downward view / fall). In this embodiment, during escape, the cleaning equipment can combine real-world scenario information with virtual restricted area information to comprehensively determine the purpose of establishing the virtual restricted area, identify the current escape scenario, and employ corresponding escape strategies for towing. This improves the escape effectiveness against virtual restricted areas, making escape more intelligent, reducing the cleaning equipment's entrapment rate, and ultimately enhancing the user experience.

[0060] In this embodiment, the cleaning device can determine the current trapped scene type based on virtual restricted area information and target object information, and then control the cleaning device to execute an escape operation matching the current trapped scene type, i.e., a target escape operation. After executing the target escape operation, the cleaning device can escape the trapped scene, and the escaped cleaning device will be outside the virtual restricted area on the map.

[0061] Optionally, the cleaning equipment can combine real-world scene information (e.g., items in the scene) and virtual restricted area information to comprehensively determine the purpose of establishing the virtual restricted area and identify the escape scenario corresponding to the established virtual restricted area. The escape scenario may include, but is not limited to, at least one of the following:

[0062] Scenario 1: A scenario where the machine (cleaning equipment) can easily overcome obstacles, but the actual restricted area prevents it from doing so. That is, a scenario where the machine can easily overcome obstacles, but the virtual restricted area prevents the machine from overcoming obstacles.

[0063] Scenario 2: The upper surface is stuck or LDS collision can be used to escape, while also considering the scenario of being far away from the virtual wall. That is, the upper surface of the machine may be stuck in the virtual restricted area or can be escaped by LDS collision. At the same time, the scenario of escaping should be far away from the virtual restricted area.

[0064] Escape Scenario 3: Prioritize handling scenarios triggered by looking down or falling, then handle virtual restricted areas (e.g., virtual walls within virtual restricted areas), and finally handle other obstacles.

[0065] Optionally, based on the identified escape scenario, the cleaning device can determine the escape operation to be performed. For example, moving away from the virtual restricted area; or ignoring the virtual restricted area first, then moving away from it after escaping from the obstacle or passing through the virtual restricted area into the area to be cleaned; or ignoring the virtual restricted area first to handle a downward view or fall, then moving away from it, etc. This embodiment does not limit this.

[0066] Through steps S202 to S206, virtual restricted area information corresponding to the target area map is obtained. The target area map is the area map of the area to be cleaned by the cleaning equipment, and the virtual restricted area information indicates the virtual restricted areas in the area map. Target detection is performed using the first sensor on the cleaning equipment to obtain target object information, which represents the target scene object within the current area where the cleaning equipment is located that matches the virtual restricted area. If the cleaning equipment becomes trapped during the cleaning process, it is controlled to perform a target escape operation based on the virtual restricted area information and the target object information. This solves the problem of cleaning equipment becoming trapped due to errors in establishing virtual restricted areas in related technologies, reducing the trapping rate of the cleaning equipment and improving the user experience.

[0067] In one exemplary embodiment, target detection is performed using a first sensor on the cleaning device to obtain target object information, including:

[0068] S11, target recognition is performed on the point cloud data collected by the first sensor to obtain target object information, wherein the target object information is the object point cloud of the target scene object.

[0069] In this embodiment, the first sensor can be a point cloud sensor, that is, a sensor used to collect point cloud data. During the cleaning process, the cleaning facility can use the point cloud sensor to collect point data, and the collected point cloud data may include point cloud data of obstacles within the collection range.

[0070] Optionally, the data acquisition by the first sensor (e.g., a point cloud sensor) can be performed periodically. For example, the first sensor can perform data acquisition in real time, that is, once every target duration (e.g., 1 second). Alternatively, it can be triggered by events, such as when a collision is detected with the cleaning equipment or when the distance between the cleaning equipment and the virtual restricted area is less than or equal to a first distance threshold.

[0071] After collecting the point cloud data, the cleaning equipment can perform target identification. For example, the cleaning equipment can match the collected point cloud data with reference object point clouds of various reference objects and virtual restricted area information to determine the target scene objects that match various reference objects, obtain the object point clouds of all target scene objects in the current area, and thus obtain the target object information.

[0072] This embodiment demonstrates how using point cloud sensors for scene object recognition can improve the accuracy and convenience of scene object recognition.

[0073] In one exemplary embodiment, target identification is performed on the point cloud data collected by the first sensor to obtain target object information, including:

[0074] S21, target recognition is performed on the point cloud data collected by the first sensor to obtain candidate object information, wherein the candidate object information is the object point cloud of the candidate objects contained in the current area;

[0075] S22, Based on the location information of the candidate objects and the virtual restricted area information, select the target scene object that matches the virtual restricted area from the candidate objects to obtain the target object information.

[0076] In this embodiment, when performing target recognition, the cleaning device can first perform target recognition on the collected point cloud data to obtain candidate object information. The candidate object information is the object information of the candidate objects contained in the current area. Here, the candidate object can be a scene object of a predetermined object type, and the object information of the candidate object can be the object point cloud of the candidate object.

[0077] Optionally, in order to identify candidate objects, the cleaning device can first determine the object to be identified through contour detection, that is, the detected object that may be the required scene object; then, the object information of the object to be identified (e.g., object point cloud) is matched with the object information of the reference object, and the object information of the object to be identified that matches the object information of the reference object is determined as the object information of the candidate object, that is, the object point cloud of the candidate object.

[0078] The cleaning equipment can determine the target scene object that matches the virtual restricted area based on the location information of the virtual restricted area and the object information of the candidate object, and then obtain the target object information, which can be the object point cloud of the target object.

[0079] Optionally, the cleaning equipment can determine the target scene object matching the virtual restricted area based on the location information of the virtual restricted area and the location information of the candidate objects. The location information of the candidate objects can be included in the object information of the candidate objects. The methods for determining the target scene object matching the virtual restricted area may include, but are not limited to, at least one of the following:

[0080] Candidate objects located within the virtual restricted area are identified as target scene objects that match the virtual restricted area;

[0081] Candidate objects whose distance from the boundary of the virtual restricted area is less than or equal to a first distance threshold are identified as target scene objects that match the virtual restricted area;

[0082] Candidate objects whose shapes match the boundary shapes of the virtual restricted area are identified as target scene objects that match the virtual restricted area.

[0083] For example, at the sliding rail or steps between the living room and the balcony, calculate whether the virtual restricted area is nearly parallel to the steps and has a similar width. If they are nearly parallel and have a similar width, the scene object that matches the virtual restricted area can be determined to be the sliding rail or steps.

[0084] This embodiment improves the efficiency of determining scene objects that match the virtual restricted area by first identifying candidate objects in the scene and then determining scene objects that match the virtual restricted area based on the object information of the identified candidate objects and the location information of the virtual restricted area.

[0085] In one exemplary embodiment, based on virtual restricted area information and target object information, the cleaning equipment is controlled to perform a target escape operation, including:

[0086] S31, if the target scene object is determined to be a scene object that the cleaning device is allowed to cross and overcome obstacles based on the target object information, and the cleaning device passes through the virtual wall of the virtual restricted area and enters an area in the target area map other than the area to be cleaned, control the cleaning device to cross the target scene object and enter the area to be cleaned.

[0087] In this embodiment, if the current escape scenario is determined to be the aforementioned escape scenario 1 based on the target object information, that is, a scenario where the machine can easily identify obstacles and the actual restricted area blocks the way, the cleaning device can calculate the distance between the cleaning device and the virtual restricted area, and then determine which side of the restricted area the cleaning device is located on based on the target area map and the target object information.

[0088] If the cleaning device is located within the area to be cleaned and has not crossed the virtual wall of the virtual restricted area, the cleaning device can adopt a strategy of moving away from the virtual wall. The cleaning device can also clean along specific boundaries. If the cleaning device crosses the virtual wall of the virtual restricted area and enters an area on the target area map other than the area to be cleaned, the cleaning device can be controlled to cross the target scene object and enter the area to be cleaned. After entering the area to be cleaned, the cleaning device can adopt a strategy of moving away from the virtual wall, or it can clean along specific boundaries.

[0089] It should be noted that the specific boundary is the boundary of the virtual restricted area and the boundary of the target scene object that is closer to the area to be cleaned. That is, when the target scene object is located in the virtual restricted area, the specific boundary is the boundary of the virtual restricted area; when the target scene object is located outside the virtual restricted area, the specific boundary is the boundary of the target scene object.

[0090] For example, at the sliding rail or steps between the living room and balcony, if the virtual restricted area is calculated to be nearly parallel to the steps and of similar width, the escape scenario corresponding to the established virtual restricted area can be identified as a scenario where the machine (a robot vacuum cleaner, an example of the aforementioned cleaning equipment) can easily overcome the obstacle, while the actual restricted area obstructs its passage. The robot vacuum cleaner can calculate the distance between the machine and the restricted area and, based on map information and real-world escape scenario information (an example of target object information), determine which side of the restricted area the machine is on. If the machine is inside the room, a strategy of moving away from the virtual wall can be adopted; if the machine has already crossed the virtual restricted area, it can ignore the virtual restricted area, use an obstacle-crossing strategy to cross the steps or sliding rail and enter the room, and then adopt a strategy of moving away from the virtual wall.

[0091] As an example, such as Figure 3 As shown, during area cleaning, the robot vacuum cleaner crosses the no-go zone line and enters the inside of the clothes rack. The robot vacuum cleaner can identify the bottom of the clothes rack as the scene object matching the virtual no-go zone, and determine that the current escape scenario is escape scenario 1, and that it has already passed through the virtual no-go zone. At this point, the robot vacuum cleaner can use the obstacle-crossing strategy to cross the rail and enter the living room, and then adopt the strategy of moving away from the virtual wall.

[0092] For example, such as Figure 4 As shown, the area to be cleaned by the robot vacuum is the living room. If the robot vacuum passes through the virtual restricted area and enters the balcony, it can identify the scene object matching the virtual restricted area as the sliding rail between the living room and the balcony, and determine that the current escape scenario is escape scenario 1, and that it has already passed through the virtual restricted area. At this time, the robot vacuum can use the obstacle-crossing strategy to cross the sliding rail and enter the living room, and then adopt the strategy of moving away from the virtual wall.

[0093] Through this embodiment, for escape scenarios where the machine can easily identify obstacles or where actual restricted areas block the way, controlling the cleaning equipment to ignore the virtual restricted area and proceed to the area to be cleaned after the cleaning equipment has already passed through the virtual restricted area can improve the convenience and success rate of the cleaning equipment's escape.

[0094] In one exemplary embodiment, based on virtual restricted area information and target object information, the cleaning equipment is controlled to perform a target escape operation, including:

[0095] S41, if the target scene object is determined to be a scene object of the target type based on the target object information, and the cleaning equipment has passed through the virtual wall of the virtual restricted area and entered the target object area where the target scene object is located, control the cleaning equipment to perform a target escape operation that matches the target type, wherein the target type includes at least one of the following: a type where the bottom does not allow the cleaning equipment to pass through, or a type where the distance between the cleaning equipment and the wall is less than or equal to a distance threshold.

[0096] In this embodiment, if the current escape scenario is determined to be the aforementioned escape scenario 2 based on the target object information, i.e., escape from a stuck upper surface or LDS collision, while also considering a scenario far from the virtual wall, the cleaning device can determine the object type (i.e., the target type) of the target scene object. The object type of the target scene object can be a type whose bottom does not allow the cleaning device to pass through, such as furniture with a bottom that is too low, or a type whose distance from the wall is less than or equal to a distance threshold (which can be a second distance threshold), such as a bed that is close to the wall.

[0097] If the cleaning device has passed through the virtual wall of the virtual restricted area and entered the target object area, it can execute a target escape operation matching the object type of the target scene object, thereby controlling the cleaning device to escape. For different object types, the cleaning device can execute different escape operations. These target escape operations can be pre-configured, i.e., corresponding escape operations are configured for different object types, or the cleaning device can try multiple escape operations and determine the escape operation based on the results. These multiple escape operations can include, but are not limited to, at least one of the following: obstacle crossing operation, local navigation operation, and edge-following operation. This embodiment does not limit this.

[0098] For example, for the bottom outer contour edges of various furniture pieces, the robot vacuum can calculate the point cloud obstacle and no-go zone locations identified by the sensors on them, and identify the current escape scenario as: easy to get stuck on the top surface or escape from LDS collision, while also considering scenarios far away from virtual walls. Based on the point cloud contour, the robot vacuum can determine the furniture type, combine the no-go zone range, and use escape strategies in conjunction with local navigation, edge-following, and other modules to help the robot vacuum escape from obstacles.

[0099] This embodiment demonstrates how, by using the corresponding escape operation based on the object type of the scene object matching the virtual restricted area and the restricted area range of the virtual restricted area, the success rate of cleaning equipment escaping from trouble can be improved.

[0100] In one exemplary embodiment, controlling a cleaning device to perform a target escape operation matching the target type includes at least one of the following:

[0101] S51, when the target type includes a type where the bottom does not allow the cleaning equipment to pass through, point cloud data is collected by the second sensor on the cleaning equipment to obtain target point cloud data; based on the target point cloud data, an exit matching the movement trajectory of the cleaning equipment entering the target object area is identified, wherein the size of the exit allows the cleaning equipment to pass through; the cleaning equipment is controlled to move out of the target object area from the exit along the movement trajectory.

[0102] S52, when the target type includes a type where the distance between the target and the wall is less than or equal to a distance threshold, control the cleaning device to move along the target boundary detected by the distance sensor of the cleaning device until it moves out of the target object area, wherein the target boundary is at least one of the following: a wall, the boundary of the target scene object.

[0103] If the target type includes areas where the cleaning device cannot pass underneath (e.g., a wardrobe with a high perimeter and a low center, preventing a robot vacuum from passing underneath), the cleaning device can perform local navigation, returning to the area to be cleaned either along a historical trajectory or by replanning a new trajectory. During trajectory planning, the cleaning device can acquire point cloud data using a second sensor to obtain target point cloud data. The second sensor can be the same as the first sensor (e.g., a point cloud sensor) or a different sensor.

[0104] For target point cloud data, the cleaning equipment can identify the target point cloud data, determine a movement path that allows the cleaning equipment to pass through, and move out of the target object area along the determined movement trajectory. Optionally, the cleaning equipment can return to the area to be cleaned along a historical movement trajectory: identify the target point cloud data, identify an exit that matches the movement trajectory of the cleaning equipment entering the target object area (i.e., the historical movement trajectory), and the size of the identified exit is sufficient for the cleaning equipment to pass through; move out of the target object area from the identified exit along the historical movement trajectory.

[0105] For example, after a robot vacuum cleaner enters the bottom of furniture, it can identify the exit or movement channel within a local area based on the point cloud data collected by the point cloud sensor, and then move out of the bottom of the furniture from the identified exit or movement channel.

[0106] If the target type includes those whose distance from the wall is less than or equal to a distance threshold, the space allowed for the cleaning device to move is limited, similar to a narrow passage (i.e., a narrow area). Edge detection can be used to control the cleaning device to move along the wall until it moves out of the target area (into the area to be cleaned). Optionally, the cleaning device can be equipped with distance sensors (e.g., LDS laser rangefinders), which can detect obstacles, such as walls, by emitting detection signals. The cleaning device can then move along the walls detected by the distance sensors.

[0107] Optionally, if the bottom of the target scene object is low, the boundary of the target scene object can also be detected by the distance sensor, and the cleaning device can move along the boundary of the target scene object detected by the distance sensor until it moves out of the target object area. The distance sensor mentioned above can be the same sensor as the first sensor and / or the second sensor mentioned above, or it can be a different sensor.

[0108] In this embodiment, the cleaning equipment can be controlled to get out of trouble by using local navigation and / or edge movement based on the type of scene object, which can improve the success rate of cleaning equipment getting out of trouble.

[0109] In one exemplary embodiment, based on virtual restricted area information and target object information, the cleaning equipment is controlled to perform a target escape operation, including:

[0110] S61, when the cleaning equipment is trapped due to the detection of a cliff or the detection that the wheels of the cleaning equipment are in a falling state, the cleaning equipment is controlled to perform a first escape operation. The first escape operation is used to control the cleaning equipment to leave the detected cliff or control the wheels to get out of the falling state. During the execution of the first escape operation, the cleaning equipment ignores the virtual restricted area.

[0111] S62, after performing the first escape operation, if it is detected that the cleaning equipment has passed through the virtual wall of the virtual restricted area, the cleaning equipment is controlled to perform a second escape operation, wherein the second escape operation is used to control the cleaning equipment to leave the range of the virtual restricted area.

[0112] The cleaning equipment may be equipped with a downward-facing sensor and / or a drop sensor. The downward-facing sensor is used for cliff detection, and the drop sensor is used for drop detection, that is, to detect the falling state of the cleaning equipment's wheels. The cleaning equipment can use the downward-facing sensor and / or drop sensor to perform downward-facing detection and / or drop detection to determine whether a downward-facing detection (the downward-facing sensor is triggered) or a drop (the drop sensor is triggered, at which point the cleaning equipment's wheels are in a falling state) is triggered.

[0113] If a downward view or fall is triggered, the current escape scenario can be identified as escape scenario 3 mentioned above. That is, the scenario where the triggered downward view or fall is dealt with first, followed by the virtual restricted area, and finally other obstacles are dealt with. The cleaning device can prioritize dealing with the downward view or fall, in which case the cleaning device ignores the virtual restricted area. When dealing with the downward view or fall, the cleaning device can execute the first escape operation, which is used to control the cleaning device to leave the detected cliff or control the wheels to escape the fall state.

[0114] The first escape operation can be an escape operation matched with a downward view or a fall. If a downward view is triggered, the cleaning equipment can perform escape operations such as forward movement, backward movement, and rotation (rotating left, rotating right, or rotating in place, etc.). If a fall is triggered, the cleaning equipment can perform escape operations such as backward movement and rotation. In this embodiment, the first escape operation is not limited.

[0115] In some scenarios, downward-looking or falling actions may be falsely triggered. For example, some dark-colored objects (such as dark carpets) may be mistakenly identified as triggering downward-looking actions due to their weak light reflection. Similarly, objects with narrow grooves (such as sliding rails) may also falsely trigger downward-looking or falling actions. To improve extrication efficiency, the cleaning device can determine whether a downward-looking or falling action has been falsely triggered. If it is determined to be a false trigger, the cleaning device can ignore the triggered action. If it is not a false trigger, the cleaning device can perform the first extrication operation.

[0116] Optionally, the amplitude of the reflected ultrasonic waves differs for cliffs and dark objects. If a downward view is triggered, the cleaning device can identify the object by sending ultrasonic waves, and determine whether the downward view was mistakenly triggered based on the reflected ultrasonic waves. If a fall is triggered, the cleaning device can determine the outline and shape of the scene object that triggered the downward view based on the point cloud data collected by the point cloud sensor, and determine whether a fall was mistakenly triggered. If the scene object that triggered the fall is determined to be an obstacle that the cleaning device can cross, then a fall has been mistakenly triggered; otherwise, it has been determined that it was not a fall that was mistakenly triggered.

[0117] For example, when cleaning the bathroom, when moving to such Figure 5 When the location shown is reached, the robot vacuum triggers the downward view. At this time, the robot vacuum can ignore the virtual no-go zone and prioritize the downward view processing.

[0118] For example, when the sweeper moves to such a position... Figure 6 When the robot vacuum is positioned at the slide rail shown, the depth of the groove in the middle of the slide rail may trigger a downward view or cause the wheels to fall. The robot vacuum can use ultrasonic waves or the sensed point cloud to determine if a downward view or a fall has been mistakenly triggered.

[0119] After performing the first escape operation, the cleaning equipment can determine its current location. Based on this location and the location of the virtual restricted area, it determines the positional relationship between the cleaning equipment and the virtual restricted area. If it is determined that the cleaning equipment has passed through the virtual wall of the virtual restricted area (potentially entering the virtual restricted area, or possibly performing a second escape operation, which can be used to control the cleaning equipment to leave the area of ​​the virtual restricted area), the cleaning equipment can determine the current escape scenario (e.g., escape scenario 1, escape scenario 2) and perform the corresponding escape operation based on the determined escape scenario, i.e., the second escape operation.

[0120] For example, on dark carpets or in recessed tracks, the robot vacuum may accidentally trigger the downward-facing camera or fall. The robot vacuum can prioritize handling downward-facing camera / fall scenarios, then virtual walls, and finally other obstacle avoidance scenarios. It can first determine if a downward-facing camera / fall scenario has been accidentally triggered; if so, ignore it; otherwise, prioritize handling the downward-facing camera / fall scenario. After successfully overcoming the obstacle, it can then design an escape strategy based on the no-go zone area.

[0121] This embodiment improves the safety of equipment operation and the efficiency of equipment extrication by prioritizing the handling of downward view / fall scenarios, followed by virtual walls, and finally other extrication scenarios.

[0122] In one exemplary embodiment, after target detection via a first sensor on the cleaning device, the method further includes:

[0123] S71, when the boundary of the target scene object is located within the area to be cleaned, control the cleaning device to clean along the boundary of the target scene object;

[0124] S72, when the boundary of the target scene object is outside the area to be cleaned, control the cleaning device to clean along the virtual wall of the virtual restricted area.

[0125] In this embodiment, after target detection via the first sensor, the cleaning device can clean the area to be cleaned based on the positional relationship between the area to be cleaned, the virtual restricted area, and the target scene object. During area cleaning, the cleaning device can adopt a strategy of moving away from the virtual wall and cleaning along a specific boundary.

[0126] The specific boundary is the boundary of the virtual restricted area or the boundary of the target scene object, whichever is closer to the area to be cleaned. If the boundary of the target scene object is within the area to be cleaned, the specific boundary is the boundary of the target scene object, and the cleaning device can clean the area along the boundary of the target scene object; if the boundary of the target scene object is outside the area to be cleaned, the specific boundary is the boundary of the virtual restricted area, which can be a virtual wall of the virtual restricted area, and the cleaning device can clean the area along the virtual wall of the virtual restricted area.

[0127] This embodiment uses the positional relationship between the area to be cleaned, the virtual restricted area, and the target scene object to clean the area, which can improve the rationality of area cleaning and reduce the probability of equipment getting stuck.

[0128] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0129] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0130] According to another aspect of the embodiments of this application, an operation control device for cleaning equipment for implementing the above-described operation control method for cleaning equipment is also provided. Figure 7 This is a structural block diagram of an optional operation control device for cleaning equipment according to an embodiment of this application, such as... Figure 7 As shown, the device may include:

[0131] The first acquisition unit 702 is used to acquire virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs to the area to be cleaned, and the virtual restricted area information is used to indicate the virtual restricted area in the area map;

[0132] The detection unit 704 is connected to the first acquisition unit 702 and is used to perform target detection through the first sensor on the cleaning equipment to obtain target object information, wherein the target object information is used to represent the target scene object that matches the virtual restricted area in the current area where the cleaning equipment is located;

[0133] The first control unit 706, connected to the detection unit 704, is used to control the cleaning equipment to perform a target escape operation based on virtual restricted area information and target object information when the cleaning equipment is trapped during the cleaning process of the area to be cleaned. The cleaning equipment is outside the virtual restricted area after escaping the trap.

[0134] It should be noted that the first acquisition unit 702 in this embodiment can be used to perform the above step S202, the detection unit 704 in this embodiment can be used to perform the above step S204, and the first control unit 706 in this embodiment can be used to perform the above step S206.

[0135] Through the aforementioned modules, virtual restricted area information corresponding to the target area map is obtained. The target area map is the map of the area to be cleaned by the cleaning equipment, and the virtual restricted area information indicates the virtual restricted areas within the area map. Target detection is performed using the first sensor on the cleaning equipment to obtain target object information, which represents the target scene object within the current area where the cleaning equipment is located that matches the virtual restricted area. If the cleaning equipment becomes trapped during the cleaning process, it is controlled to perform a target escape operation based on the virtual restricted area information and the target object information. This solves the problem of cleaning equipment easily becoming trapped due to errors in establishing virtual restricted areas in related technologies, reducing the trapping rate of cleaning equipment and improving the user experience.

[0136] In one exemplary embodiment, the detection unit includes:

[0137] The first recognition module is used to perform target recognition on the point cloud data collected by the first sensor to obtain target object information, wherein the target object information is the object point cloud of the target scene object.

[0138] In one exemplary embodiment, the first identification module includes:

[0139] The identification submodule is used to identify targets in the point cloud data collected by the first sensor and obtain candidate object information, wherein the candidate object information is the object point cloud of the candidate objects contained in the current area;

[0140] The selection submodule is used to select the target scene object that matches the virtual restricted area from the candidate objects based on the location information and virtual restricted area information, and obtain the target object information.

[0141] In one exemplary embodiment, the first control unit includes:

[0142] The first control module is used to control the cleaning equipment to cross the target scene object and enter the area to be cleaned when the target scene object is determined to be a scene object that the cleaning equipment is allowed to cross, and the cleaning equipment passes through the virtual wall of the virtual restricted area and enters an area in the target area map other than the area to be cleaned.

[0143] In one exemplary embodiment, the first control unit includes:

[0144] The second control module is used to control the cleaning equipment to perform a target escape operation that matches the target type when the target scene object is determined to be a scene object of the target type based on the target object information and the cleaning equipment has passed through the virtual wall of the virtual restricted area and entered the target object area where the target scene object is located. The target type includes at least one of the following: a type where the bottom does not allow the cleaning equipment to pass through, or a type where the distance between the cleaning equipment and the wall is less than or equal to a distance threshold.

[0145] In one exemplary embodiment, the first control unit includes:

[0146] The acquisition module is used to acquire point cloud data through the second sensor on the cleaning device when the target type includes types where the bottom does not allow the cleaning device to pass, and obtain target point cloud data; the second identification module is used to identify the exit that matches the movement trajectory of the cleaning device entering the target object area based on the target point cloud data, wherein the size of the exit allows the cleaning device to pass through; the third control module is used to control the cleaning device to move out of the target object area from the exit along the movement trajectory.

[0147] The fourth control module is used to control the cleaning device to move along the target boundary detected by the distance sensor of the cleaning device until it moves out of the target object area when the target type includes the type where the distance between the target and the wall is less than or equal to a distance threshold. The target boundary is at least one of the following: a wall, or the boundary of the target scene object.

[0148] In one exemplary embodiment, the first control unit includes:

[0149] The fifth control module is used to control the cleaning equipment to perform a first escape operation when the cleaning equipment is trapped due to the detection of a cliff or the detection that the wheels of the cleaning equipment are in a falling state. The first escape operation is used to control the cleaning equipment to leave the detected cliff or control the wheels to get out of the falling state. During the execution of the first escape operation, the cleaning equipment ignores the virtual restricted area.

[0150] The sixth control module is used to control the cleaning equipment to perform a second escape operation after the first escape operation is completed and the cleaning equipment is detected to have passed through the virtual wall of the virtual restricted area. The second escape operation is used to control the cleaning equipment to leave the area of ​​the virtual restricted area.

[0151] In one exemplary embodiment, the above-described apparatus further includes:

[0152] The second control unit is used to control the cleaning device to clean along the boundary of the target scene object after the target is detected by the first sensor on the cleaning device and the boundary of the target scene object is located within the area to be cleaned.

[0153] The third control unit is used to control the cleaning equipment to clean along the virtual wall of the virtual restricted area when the boundary of the target scene object is outside the area to be cleaned.

[0154] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in the above embodiments.

[0155] It should be noted that the above-mentioned module, as part of the device, can operate in, for example... Figure 1 The hardware environment shown can be implemented through software or hardware, and the hardware environment includes the network environment.

[0156] According to another aspect of the embodiments of this application, a storage medium is also provided. Optionally, in this embodiment, the storage medium can be used to execute program code for the operation control method of any of the cleaning devices described in the embodiments of this application.

[0157] Optionally, in this embodiment, the storage medium may be located on at least one of the network devices in the network shown in the above embodiment.

[0158] Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps:

[0159] S1, Obtain the virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs, and the virtual restricted area information is used to indicate the virtual restricted area in the area map;

[0160] S2, target detection is performed through the first sensor on the cleaning equipment to obtain target object information, wherein the target object information is used to represent the target scene object that matches the virtual restricted area within the current area where the cleaning equipment is located;

[0161] S3, if the cleaning equipment gets stuck during the cleaning of the area to be cleaned, control the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information, wherein the cleaning equipment is outside the virtual restricted area after escaping the obstacle.

[0162] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated in this embodiment.

[0163] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0164] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described operation control method for cleaning equipment is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0165] Figure 8 This is a structural block diagram of an optional electronic device according to an embodiment of this application, such as... Figure 8 As shown, it includes a processor 802, a communication interface 804, a memory 806, and a communication bus 808. The processor 802, communication interface 804, and memory 806 communicate with each other via the communication bus 808.

[0166] Memory 806 is used to store computer programs;

[0167] When processor 802 executes a computer program stored in memory 806, it performs the following steps:

[0168] S1, Obtain the virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs, and the virtual restricted area information is used to indicate the virtual restricted area in the area map;

[0169] S2, target detection is performed through the first sensor on the cleaning equipment to obtain target object information, wherein the target object information is used to represent the target scene object that matches the virtual restricted area within the current area where the cleaning equipment is located;

[0170] S3, if the cleaning equipment gets stuck during the cleaning of the area to be cleaned, control the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information, wherein the cleaning equipment is outside the virtual restricted area after escaping the obstacle.

[0171] Optionally, in this embodiment, the communication bus can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, Figure 8 The symbol is represented by a single thick line, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned electronic device and other devices.

[0172] The aforementioned memory may include RAM, or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0173] As an example, the memory 806 described above may include, but is not limited to, the first acquisition unit 702, the detection unit 704, and the first control unit 706 of the control device of the aforementioned device. Furthermore, it may include, but is not limited to, other module units of the control device of the aforementioned device, which will not be elaborated upon in this example.

[0174] The processors mentioned above can be general-purpose processors, including but not limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; they can also be DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0175] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0176] Those skilled in the art will understand that Figure 8 The structure shown is for illustrative purposes only. The device implementing the above-described cleaning equipment operation control method can be a terminal device, such as a smartphone (e.g., Android phone, iOS phone), tablet computer, PDA, mobile Internet Devices (MID), PAD, etc. Figure 8 This does not limit the structure of the aforementioned electronic device. For example, the electronic device may also include components that are more... Figure 8 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 8 The different configurations shown.

[0177] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, ROM, RAM, disk or optical disk, etc.

[0178] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0179] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0180] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0181] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between units or modules, and may be electrical or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the solution provided in this embodiment, depending on actual needs.

[0183] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0184] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for controlling the operation of cleaning equipment, characterized in that, include: Obtain virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs, and the virtual restricted area information is used to indicate the virtual restricted area in the area map; Target detection is performed using the first sensor on the cleaning device to obtain target object information, wherein the target object information is used to represent a target scene object that matches the virtual restricted area within the current area where the cleaning device is located; If the cleaning equipment gets stuck during the cleaning of the area to be cleaned, the cleaning equipment is controlled to perform a target escape operation based on the virtual restricted area information and the target object information, wherein the cleaning equipment is outside the virtual restricted area after escaping the trap; The step of controlling the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information includes: determining the current trapped scene type based on the virtual restricted area information and the target object information, and controlling the cleaning equipment to perform a target escape operation matching the scene type based on the scene type.

2. The method according to claim 1, characterized in that, The step of obtaining target object information by detecting the target through the first sensor on the cleaning equipment includes: Target recognition is performed on the point cloud data collected by the first sensor to obtain the target object information, wherein the target object information is the object point cloud of the target scene object.

3. The method according to claim 2, characterized in that, The step of performing target identification on the point cloud data collected by the first sensor to obtain the target object information includes: Target recognition is performed on the point cloud data collected by the first sensor to obtain candidate object information, wherein the candidate object information is the object point cloud of candidate objects contained in the current area; Based on the location information of the candidate objects and the virtual restricted area information, the target scene object that matches the virtual restricted area is selected from the candidate objects to obtain the target object information.

4. The method according to claim 1, characterized in that, The step of controlling the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information includes: If, based on the target object information, the target scene object is determined to be a scene object that the cleaning device is allowed to pass through, and the cleaning device passes through the virtual wall of the virtual restricted area and enters an area in the target area map other than the area to be cleaned, the cleaning device is controlled to pass through the target scene object and enter the area to be cleaned.

5. The method according to claim 1, characterized in that, The step of controlling the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information includes: If the target scene object is determined to be a scene object of the target type based on the target object information, and the cleaning device has passed through the virtual wall of the virtual restricted area and entered the target object area where the target scene object is located, the cleaning device is controlled to perform the target escape operation that matches the target type. The target type includes at least one of the following: a type where the bottom does not allow the cleaning device to pass through, or a type where the distance between the device and the wall is less than or equal to a distance threshold.

6. The method according to claim 5, characterized in that, The control of the cleaning equipment to perform the target escape operation matching the target type includes: In cases where the target type includes a type where the bottom does not allow the cleaning equipment to pass through, point cloud data is collected by a second sensor on the cleaning equipment to obtain target point cloud data; based on the target point cloud data, an exit matching the movement trajectory of the cleaning equipment entering the target object area is identified, wherein the size of the exit allows the cleaning equipment to pass through; the cleaning equipment is controlled to move out of the target object area from the exit along the movement trajectory. When the target type includes types where the distance between the target and the wall is less than or equal to a distance threshold, the cleaning device is controlled to move along the target boundary detected by the distance sensor of the cleaning device until it moves out of the target object area, wherein the target boundary is at least one of the following: the wall, the boundary of the target scene object.

7. The method according to claim 1, characterized in that, The scenario types include cliffs and vehicles falling; the step of controlling the cleaning equipment to perform target escape operations based on the virtual restricted area information and the target object information includes: When the cleaning device becomes trapped due to detecting a cliff or the wheels of the cleaning device being in a falling state, the cleaning device is controlled to perform a first escape operation. The first escape operation is used to control the cleaning device to leave the detected cliff or control the wheels to escape the falling state. During the execution of the first escape operation, the cleaning device ignores the virtual restricted area. After the first escape operation is completed, if it is detected that the cleaning device has passed through the virtual wall of the virtual restricted area, the cleaning device is controlled to perform a second escape operation, wherein the second escape operation is used to control the cleaning device to leave the area of ​​the virtual restricted area.

8. The method according to any one of claims 1 to 7, characterized in that, After target detection via a first sensor on the cleaning device, the method further includes: When the boundary of the target scene object is located within the area to be cleaned, the cleaning device is controlled to clean along the boundary of the target scene object; When the boundary of the target scene object is outside the area to be cleaned, the cleaning device is controlled to clean along the virtual wall of the virtual restricted area.

9. An operation control device for cleaning equipment, characterized in that, include: The first acquisition unit is used to acquire virtual restricted area information corresponding to the target area map, wherein the target area map is the area map to which the cleaning equipment belongs, and the virtual restricted area information is used to indicate the virtual restricted area in the area map; The detection unit is used to perform target detection through the first sensor on the cleaning equipment to obtain target object information, wherein the target object information is used to represent the target scene object that matches the virtual restricted area within the current area where the cleaning equipment is located; A first control unit is configured to, when the cleaning equipment is trapped during the cleaning of the area to be cleaned, control the cleaning equipment to perform a target escape operation based on the virtual restricted area information and the target object information, wherein the cleaning equipment after escaping the trap is outside the virtual restricted area; The first control unit is specifically used to determine the current trapped scene type based on the virtual restricted area information and the target object information, and to control the cleaning equipment to perform a target escape operation that matches the scene type based on the scene type.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program, when executed, performs the method of any one of claims 1 to 7.

11. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to execute the method of any one of claims 1 to 7 through the computer program.

Images