Cleaning control method, device, medium and electronic device of cleaning equipment

Abstract

The application provides a cleaning control method and device of a cleaning device, a medium and an electronic device. The method determines the correlation degree of the following instruction correlation feature of the candidate guide and the following trigger instruction based on the following instruction correlation feature of the candidate guide; and / or determines the confidence of the biological feature based on the biological feature; and / or determines the action feature sequence based on the action feature; determines the target guide based on the correlation degree of the following instruction correlation feature of the candidate guide and the following trigger instruction, the confidence of the biological feature and / or the action feature sequence; acquires the dynamic tracking data of the target guide in real time, controls the cleaning device to follow the target guide based on the dynamic tracking data; and executes the cleaning task corresponding to the cleaning instruction when detecting the cleaning instruction issued by the target guide. The application guides the cleaning device to the cleaning area in the form of human following.

Description

Cleaning control methods, devices, media and electronic equipment for cleaning equipment Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a cleaning control method, apparatus, medium, and electronic device for cleaning equipment. Background Technology

[0002] Cleaning equipment (such as cleaning robots) has become an important tool for maintaining the modern home environment. Users have put forward higher requirements for the convenience of cleaning operations, especially when dealing with specific scenarios such as local dirt. How to quickly and intuitively guide the equipment to complete the fixed-point cleaning has become a key technical direction for improving the user experience.

[0003] In related technologies, the main ways to enable cleaning equipment to reach dirty areas and perform cleaning tasks include: the user manually picking up the equipment and placing it near the area to be cleaned; or manually specifying the movement path of the cleaning equipment in a mobile app and controlling the cleaning equipment to move autonomously to the area to be cleaned according to that path.

[0004] Therefore, existing solutions for locating cleaning areas and moving cleaning equipment to the area to be cleaned require users to perform complex operations to guide the equipment to the area to be cleaned, which cannot achieve an efficient and precise cleaning experience. Summary of the Invention

[0005] The purpose of this application is to provide a cleaning control method, apparatus, medium, and electronic device for cleaning equipment, which can solve the technical problem that cleaning equipment cannot autonomously follow the user to the user-designated location to perform cleaning tasks. The specific solution is as follows:

[0006] According to a specific embodiment of this application, in a first aspect, this application provides a cleaning control method for a cleaning device, the method comprising:

[0007] In response to the follow trigger command, search for candidate guides in the target scene and extract the follow command association features, biometric features and / or action features of each candidate guide;

[0008] Based on the follow instruction association features of the candidate guides, determine the correlation degree between the follow instruction association features and the follow triggering command; and / or based on the biometric features, determine the confidence level of the biometric features; and / or based on the action features, determine the action feature sequence;

[0009] The target guide is determined based on the correlation between the follow instruction association features and the follow trigger instruction of the candidate guide, the confidence level of the biometric features and / or the action feature sequence;

[0010] The system acquires real-time dynamic tracking data of the target guide and controls the cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics.

[0011] When a cleaning instruction is detected from the target guide, the cleaning task corresponding to the cleaning instruction is executed.

[0012] In some embodiments, the search for a guide within the target scene includes:

[0013] Control the cleaning equipment to move within the target scene and collect environmental data in real time;

[0014] When the environmental data includes human body contour features, candidate guides are determined based on the human body contour features.

[0015] In some embodiments, it also includes:

[0016] When the environmental data does not include human outline features, the cleaning device is controlled to output a camera entry reminder message at a preset time interval;

[0017] When human body contour features are detected, candidate guides are determined based on the human body contour features;

[0018] When the number of times the on-camera reminder is output reaches the preset limit, the search for a guide is deemed to have failed.

[0019] The image capture notification information is used to remind the user to move into the image acquisition range of the cleaning equipment.

[0020] In some embodiments, determining the target guide based on the correlation between the follow instruction association features and the follow trigger instruction of the candidate guide, the confidence level of the biometric features, and / or the action feature sequence includes:

[0021] Based on a pre-defined guide selection model, the following probability of each candidate guide is determined according to the correlation degree between the follow instruction association features and the follow trigger instruction, the confidence level of the biometric features, and / or the action feature sequence.

[0022] The target facilitator is determined based on the following probability of each candidate facilitator.

[0023] In some embodiments, the facilitator selection model is a pre-trained machine learning model or a weighted computation model, wherein the weighted computation model includes preset weight values ​​for the correlation, confidence, and action feature sequences.

[0024] In some embodiments, determining the target facilitator based on the follower probability of each of the candidate facilitators includes:

[0025] When the following probability of at least two of the candidate guides is greater than a preset probability threshold, the cleaning device is controlled to output interactive reminder information; wherein, the interactive reminder information is used to instruct the candidate guide to initiate an identity verification response;

[0026] Based on the identity verification response obtained from the candidate guide, the target guide is determined.

[0027] In some embodiments, the identity verification response is at least one of the following:

[0028] The candidate guide's body parts and / or predetermined actions; wherein, the body parts are feet or legs, and the predetermined actions are foot or leg movements;

[0029] Voice commands or identity verification commands based on terminal devices.

[0030] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0031] If no identity verification response is received within a preset time, or if the unique target guide cannot be selected based on the received identity verification response, the cleaning device is controlled to perform an end-of-screening operation; wherein, the end-of-screening operation includes: outputting a failure prompt message, exiting the follow mode, or selecting the candidate guide with the highest follow probability as the target guide.

[0032] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0033] If no identity verification response is received within a preset time, or if the unique target guide cannot be selected based on the received identity verification response, the cleaning device will output interactive reminder information a second time based on preset interactive reminder information rules; wherein, the interactive reminder information is used to instruct the candidate guide to initiate a second identity verification response;

[0034] The target guide is determined based on the secondary identity verification response output by the candidate guide.

[0035] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0036] Within a preset time period after the cleaning device outputs interactive reminder information, the cleaning device is controlled to collect voice data of each candidate guide;

[0037] Based on the speech data, extract the voiceprint features, sound source spatial information and / or semantic information of the speech data of the candidate guide;

[0038] The target guide is determined based on the voiceprint features, the spatial information of the sound source, and / or the semantic information of the speech data.

[0039] In some embodiments, the follow-triggered instruction is at least one of the following:

[0040] Control commands issued by the terminal device application;

[0041] Preset voice control commands;

[0042] Control commands based on a video interactive interface.

[0043] In some embodiments, the method further includes:

[0044] In response to the received follow-trigger command, if the cleaning device is not located in the target scene, the cleaning device is controlled to move to the target scene.

[0045] In some embodiments, acquiring real-time dynamic tracking data of the target guide and controlling the cleaning device to follow the target guide's movement based on the dynamic tracking data includes:

[0046] If at least one interfering object is detected during the tracking process, the following operations are performed:

[0047] Extract similar characteristics between the target guide and the interference object, including biological and / or non-biological characteristics;

[0048] The extracted similar features are compared with the pre-stored benchmark feature library of the target facilitator;

[0049] The cleaning equipment is controlled to continue following the object with the highest similarity to the benchmark feature library.

[0050] In some embodiments, acquiring real-time dynamic tracking data of the target guide and controlling the cleaning device to follow the target guide's movement based on the dynamic tracking data includes:

[0051] When the real-time dynamic tracking data is a local component of the preset biometrics of the target guide, the missing parts of the preset biometrics are supplemented based on the local component.

[0052] Based on the completed biometric and / or non-biometric features, the real-time acquired dynamic tracking data is matched with the pre-stored baseline feature library of the target guide.

[0053] When the real-time acquired dynamic tracking data matches the dynamic tracking data, based on the completed biometric and / or non-biometric features, the cleaning device is controlled to follow the target guide so that the distance between the cleaning device and the target guide is a preset distance.

[0054] When the real-time acquired dynamic tracking data does not match the dynamic tracking data, a preset target guide search operation is performed.

[0055] In some embodiments, when a cleaning instruction issued by the target guide is detected, a cleaning task corresponding to the cleaning instruction is executed, including:

[0056] When a cleaning instruction issued by the target guide is detected to match a preset cleaning instruction, the cleaning task corresponding to the cleaning instruction is executed.

[0057] In some embodiments, the cleaning instruction is the behavior of the target guide staying in a preset area;

[0058] The execution of the cleaning task corresponding to the cleaning instruction includes:

[0059] Based on the duration of the target guide's stay, the cleaning mode and / or cleaning area used for the cleaning task are determined.

[0060] In some embodiments, when a cleaning instruction is detected issued by the target guide, executing a cleaning task corresponding to the cleaning instruction includes:

[0061] Upon detecting a cleaning instruction issued by the target guide, the cleaning device outputs a task confirmation request;

[0062] Upon receiving a task confirmation response from the target guide, the cleaning task corresponding to the cleaning instruction is executed.

[0063] In some embodiments, the follow instruction associated features of the candidate facilitator include at least one of the following:

[0064] The distance between the candidate guide and the sound source of the follow-triggered command;

[0065] The distance between the candidate guide and the cleaning equipment;

[0066] The azimuth angle of the candidate guide relative to the direction of travel of the cleaning equipment;

[0067] The time difference between the time the follow-triggered instruction is received and the time the candidate guide is identified;

[0068] The matching result between the candidate guide's identity information and the pre-stored priority follow permissions.

[0069] According to a specific embodiment of this application, in a second aspect, this application also provides a cleaning control device for cleaning equipment, the device comprising:

[0070] The target search unit is configured to respond to a follow trigger command, search for candidate guides in the target scene, and extract the follow command association features, biometric features, and / or action features of each candidate guide;

[0071] The feature processing unit is configured to determine the correlation between the follow instruction association features and the follow triggering instruction based on the follow instruction association features of the candidate facilitator; and / or determine the confidence level of the biometric features based on the biometric features; and / or determine the action feature sequence based on the action features.

[0072] The target locking unit is configured to determine the target guide based on the correlation between the follow instruction association features and the follow trigger instruction of the candidate guide, the confidence level of the biometric features and / or the action feature sequence;

[0073] A dynamic following unit is configured to acquire dynamic tracking data of the target guide in real time, and control the cleaning device to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics.

[0074] The cleaning execution unit is configured to execute a cleaning task corresponding to the cleaning instruction issued by the target guide when the cleaning instruction is detected.

[0075] According to a specific embodiment of this application, in a third aspect, this application also provides a cleaning device, the device comprising:

[0076] Cleaning equipment body,

[0077] A cleaning control device is disposed on the main body of the cleaning equipment; the cleaning control device is configured as follows:

[0078] In response to the follow trigger command, search for candidate guides in the target scene and extract the follow command association features, biometric features and / or action features of each candidate guide;

[0079] Based on the follow instruction association features of the candidate guides, determine the correlation degree between the follow instruction association features and the follow triggering command; and / or based on the biometric features, determine the confidence level of the biometric features; and / or based on the action features, determine the action feature sequence;

[0080] The target guide is determined based on the correlation between the follow instruction association features and the follow trigger instruction of the candidate guide, the confidence level of the biometric features and / or the action feature sequence;

[0081] The system acquires real-time dynamic tracking data of the target guide and controls the cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics.

[0082] When a cleaning instruction is detected from the target guide, the cleaning task corresponding to the cleaning instruction is executed.

[0083] According to a specific embodiment of this application, in a fourth aspect, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any of the preceding claims.

[0084] According to a specific embodiment of this application, in a fifth aspect, this application also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in any of the preceding claims.

[0085] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects:

[0086] Through a three-layer collaborative discrimination mechanism based on scene relevance, biometric confidence, and action sequence, the system can accurately locate the real user issuing commands in complex home environments with multiple users and pets, significantly reducing false identification and misfollowing rates. Users only need to guide the cleaning equipment through natural walking behavior to allow it to actively follow and autonomously plan its path, quickly reaching the area to be cleaned. This eliminates reliance on mobile apps, remote controls, or manual handling, enabling the equipment to move and arrive autonomously with the user. During the following process, the system continuously verifies the target's identity and status through the fusion of biometric and non-biometric features, maintaining stable tracking even under interference such as occlusion and changes in lighting. Upon arrival, the system can trigger the corresponding cleaning mode through preset actions. Therefore, the entire solution integrates multiple processes including environmental perception, decision-making, motion control, and task execution, not only improving the accuracy of individual stages but also enhancing the overall practicality and reliability of the cleaning equipment in real-world, complex home scenarios. Attached Figure Description

[0087] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0088] Figure 1 illustrates a scenario diagram of a cleaning control method for a cleaning device provided in this application;

[0089] Figure 2 shows a schematic flowchart of a cleaning control method for a cleaning device provided in this application;

[0090] Figure 3 shows another schematic flowchart of a cleaning control method for a cleaning device provided in this application;

[0091] Figure 4 shows a schematic diagram of the structure of a cleaning control device for a cleaning equipment provided in this application;

[0092] Figure 5 shows a schematic diagram of the structure of an electronic device provided in this application. Detailed Implementation

[0093] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0094] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0095] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0096] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0097] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0098] In existing technologies, the need for cleaning equipment to reach dirty areas and perform cleaning tasks (such as spot cleaning) can be achieved through the following main solutions:

[0099] Firstly, users view a pre-built map in a mobile app and manually specify the target location of the cleaning device (moving to that location and cleaning) by designating the area to be cleaned. The cleaning device then autonomously moves to the area based on map navigation. In this solution, the cleaning device's path planning and movement rely entirely on the map pre-stored in the app. If the map is not updated in a timely manner or deviates from the real environment, it will directly cause path planning errors, the appearance of unrecorded obstacles on the path, etc., preventing the cleaning robot from moving smoothly to the target location, or even causing collisions and damage during the movement.

[0100] Secondly, users can control the cleaning equipment in real time using the directional keys or virtual joystick within the app, gradually moving it to the area to be cleaned (or nearby). In this solution, users need to remotely control the equipment through a two-dimensional interface. On the one hand, the operational precision of a two-dimensional interface is limited; small deviations in the user's operation can be amplified into significant deviations in the cleaning equipment's movement, leading to an increase in invalid movement paths. On the other hand, this operation method is highly susceptible to network latency or blind spots, causing the equipment to collide with surrounding obstacles.

[0101] Third, users manually pick up the device and place it near the area to be cleaned. However, this approach has several drawbacks. First, manually moving the cleaning device is a physical burden for elderly, young, and users with disabilities. Second, during transport, the user's limbs directly contact the device, making it prone to dust and other dirt accumulation, posing a hygiene risk. The overall interaction method is clumsy and inefficient, failing to meet the user's need for convenient cleaning.

[0102] To address the aforementioned issues, Figure 1 illustrates a scenario diagram of a cleaning control method for a cleaning device provided in this application. As shown in Figure 1, the cleaning device 02 moves to the area to be cleaned by following the user (target guide) 01. The method includes:

[0103] In response to the follow trigger command, search for candidate guides in the target scene and extract the follow command association features, biometric features and / or action features of each candidate guide;

[0104] Based on the follow instruction association features of the candidate guides, determine the correlation degree between the follow instruction association features and the follow triggering command; and / or based on the biometric features, determine the confidence level of the biometric features; and / or based on the action features, determine the action feature sequence;

[0105] The target guide is determined based on the correlation between the follow instruction association features and the follow trigger instruction of the candidate guide, the confidence level of the biometric features and / or the action feature sequence;

[0106] The system acquires real-time dynamic tracking data of the target guide and controls the cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics.

[0107] When a cleaning instruction is detected from the target guide, the cleaning task corresponding to the cleaning instruction is executed.

[0108] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0109] Figure 2 shows a flowchart illustrating a cleaning control method for a cleaning device according to this application. As shown in Figure 2, a cleaning control method for a cleaning device includes:

[0110] S201. Respond to the follow trigger command, search for candidate guides in the target scene, and extract the follow command association features, biometric features and / or action features of each candidate guide;

[0111] The follow-trigger command instructs the cleaning device to enter follow mode, i.e., to follow the target guide. The follow-instruction associated features are a set of real-time environmental and behavioral data acquired by the cleaning device through its sensing module, used to determine which user in the current environment is most likely to become the target guide. Biometric features are the user's skeletal features collected by the cleaning device through its sensing module; motion features, given the hardware structure of the cleaning device, mainly refer to the motion features of the lower body, such as the feet and legs.

[0112] S202. Based on the follow instruction association features of candidate facilitators, determine the correlation between the follow instruction association features and the follow triggering instructions; and / or based on biometric features, determine the confidence level of biometric features; and / or based on action features, determine the action feature sequence.

[0113] Among them, the correlation between the follow instruction association features and the follow trigger instruction describes the degree of association between the follow instruction association features and the follow trigger instruction. This refers to the degree of matching between the behavioral characteristics of the currently detected candidate guide in the target scene, quantified by the cleaning device through an algorithm, and the trigger instruction received by the cleaning device. Different instructions correspond to different behavioral characteristics of the target guide. For example, if a user says "come here," then the user is near the device, facing the device, and has a guiding intention. The confidence level of biometric features refers to the extent to which the target detected by the cleaning device can be confirmed as a biologically human being. In multi-person scenarios, the confidence level also reflects the relative competitiveness score of each individual. The action feature sequence refers to the technical process by which the cleaning device analyzes a series of continuous actions of the candidate guide. It is not merely about recognizing a single static posture, but about understanding the complete intent expressed by the action flow.

[0114] S203. Based on the correlation between the candidate facilitator's follow instruction association features and the follow trigger instruction, the confidence level of biometric features and / or the sequence of action features, determine the target facilitator;

[0115] One approach is the simple "choose one" judgment mode, which uses multiple dimensions to improve accuracy and avoid misjudgments. The "choose one" mode is suitable for scenarios with few candidates, offering fast response and low computational cost. It selects any one of the following indicators: biometric confidence, correlation, or action feature sequence; if the corresponding threshold is met, the target is directly identified. The multi-dimensional judgment mode is suitable for scenarios with many candidates. It weights and fuses the correlation, confidence, and action feature sequence matching scores to calculate a comprehensive adaptation score, sorts the scores, and filters targets based on thresholds. An outlier removal mechanism is introduced to prevent misjudgments caused by anomalies in a single dimension.

[0116] S204. Acquire dynamic tracking data of the target guide in real time, and control the cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics;

[0117] During the following phase, the cleaning equipment needs to continuously lock onto and track the target guide. In this context, non-biological characteristics refer to visual identifiers that are not dependent on human physiological structures but are stable over a short period and can be continuously observed by the equipment, such as clothing color, shoe color, and style.

[0118] S205. When a cleaning instruction is detected from the target guide, execute the cleaning task corresponding to the cleaning instruction.

[0119] Once the robot vacuum cleaner arrives at the target location with the user, the user can directly issue a cleaning command, and the robot will then begin to perform the specific cleaning task. The cleaning command can be a voice command, a button command, or a motion command.

[0120] This embodiment utilizes a collaborative discrimination mechanism combining scene relevance, biometric confidence, and action sequence to accurately pinpoint the real user issuing commands in complex home environments with multiple users and pets, significantly reducing false identification and misfollowing rates. Users only need to guide the cleaning equipment through natural walking behavior to allow it to actively follow and autonomously plan its path, quickly reaching the cleaning area. This eliminates reliance on mobile apps, remote controls, or manual handling, enabling the equipment to move and arrive autonomously with the user. During the following process, the system continuously verifies the target's identity and status through the fusion of biometric and non-biometric features, maintaining stable tracking even under interference such as occlusion and changes in lighting. Upon arrival, the corresponding cleaning mode can be triggered based on the cleaning command. Therefore, the entire solution integrates multiple processes—following recognition, following lock, motion control, and cleaning task execution—not only improving the accuracy of individual stages but also enhancing the overall practicality and reliability of the cleaning equipment in real, complex home scenarios.

[0121] Figure 3 shows another schematic flowchart of a cleaning control method for a cleaning device provided in this application. As shown in Figure 3, a cleaning control method for a cleaning device includes:

[0122] S301. In response to the received follow-trigger command, if the cleaning equipment is not located in the target scene, control the cleaning equipment to move to the target scene.

[0123] The target scenario is the scene implicitly or explicitly specified by the follow-triggered command (e.g., if a user issues the command "follow me" in the living room, the target scenario is the living room; or if the command explicitly states "follow me to the bedroom," the target scenario is the bedroom). In some scenarios, the cleaning robot obtains its current position through a positioning module (such as an indoor positioning sensor) and compares it with the preset position information of the target scenario to determine whether it is already within the target scenario. If the current position is within the target scenario, it directly proceeds to the next step; if it is not within the target scenario, it first performs a scene movement task, and only starts following after arriving at the target scenario, ensuring that the following action is triggered in the correct scenario.

[0124] In some embodiments, the follow-up trigger instruction is at least one of the following:

[0125] Control commands issued by the terminal device application; for example, the user can manually click the "Follow" button or send a preset command (such as voice input "Start Follow" in the APP) through the cleaning device on the mobile phone / tablet, and the terminal will transmit the command to the cleaning device to trigger the follow process (suitable for scenarios where the user is not near the device but needs to start the follow remotely).

[0126] Pre-set voice control commands; for example, the user can directly issue a pre-stored voice keyword (such as "follow me" or "follow me"), and the cleaning device will trigger the follow function after the audio acquisition module recognizes and matches the keyword.

[0127] Control commands based on a video interactive interface; for example, the cleaning equipment captures real-time images through its own camera and transmits them to the video interactive interface of the terminal APP. The user clicks on their own position in the image or triggers the "follow the target in the image" virtual button in the interface, and the device receives the command and starts following.

[0128] S302. Search for candidate guides in the target scene and extract the follow instruction association features, biometric features and / or action features of each candidate guide;

[0129] Among these, searching for and selecting guides within the target scenario includes:

[0130] Control the movement of cleaning equipment within the target environment and collect environmental data in real time;

[0131] When the environmental data includes human body contour features, candidate guides are determined based on these features.

[0132] Understandably, upon arriving at the target scene, the cleaning equipment will not simply stand still and wait. Instead, it will perform predetermined actions (such as cruising along the edge of the scene or circling 360° to locate itself), while simultaneously collecting real-time environmental data (including image data, contour data, and motion trajectory data) from the scene using visual sensors (cameras) and infrared sensors. From the collected environmental data, image recognition algorithms will extract contour features to filter out objects that match human contours, excluding non-target objects such as furniture, pets, and clutter. In other words, a preliminary screening using human contours reduces the collection and calculation of features for invalid objects, improving overall following response efficiency and avoiding misclassifying non-human objects as candidates. For example, if a user issues the command "Follow me" in the bedroom (target scene), the cleaning robot will move from the living room to the bedroom (target scene). After entering the bedroom, the robot will slowly cruise along the bedroom wall, collecting real-time image data of the bedroom environment through cameras. From the image data, contour features will be extracted, identifying one human contour template and excluding non-human contours such as beds, wardrobes, and chairs in the bedroom. This object will be marked as the unique candidate guide, and the following operation will begin.

[0133] In some embodiments, it also includes:

[0134] When the environmental data does not include human silhouette features, the cleaning equipment is controlled to output a camera entry reminder message at preset time intervals.

[0135] When human body contour features are detected, candidate guides are determined based on these features.

[0136] When the number of times the on-camera notification is displayed reaches the preset limit, the search for a guide is deemed to have failed.

[0137] The "appearance alert" message is used to remind users to move into the image capture range of the cleaning equipment.

[0138] Understandably, once the cleaning equipment reaches the target scene, if it doesn't capture any people, it will periodically remind the user to enter the shooting range; if no one appears after enough reminders, it will determine that no candidate guide can be found and stop searching; once a human silhouette is captured, the candidate will be identified according to the previous logic, and the subsequent follow-up process will be initiated.

[0139] In this embodiment, by providing a reminder message, the user is clearly guided into the image acquisition range, which solves the problem of not being able to find people in the target scene and improves the success rate of following the user. It also avoids blind searching and wasted computing power by setting an upper limit on the number of reminders and automatically determining failure if the timeout is exceeded, thus preventing the robot from continuously moving ineffectively in unmanned scenes and reducing energy consumption and computing power consumption.

[0140] S303. Based on the follow instruction association features of candidate facilitators, determine the correlation between the follow instruction association features and the follow triggering instructions; and / or based on biometric features, determine the confidence level of biometric features; and / or based on action features, determine the action feature sequence.

[0141] Among them, the follow instruction association features of the candidate facilitator include at least one of the following:

[0142] The distance between the candidate facilitator and the sound source that triggers the follow-up command;

[0143] The distance between the candidate guide and the cleaning equipment;

[0144] The azimuth angle of the candidate guide relative to the direction of travel of the cleaning equipment;

[0145] The time difference between the time the trigger command is received and the time the candidate leader is identified;

[0146] The matching results of the candidate guide's identity information with the pre-stored priority follow permissions.

[0147] Based on the follow instruction association characteristics of candidate facilitators, the correlation between the follow instruction association characteristics and the follow trigger instruction is determined, including:

[0148] The features associated with each following instruction are standardized. For example, features such as sound source distance, device distance, azimuth angle, time difference, and permission matching results are standardized and uniformly mapped to the [0,1] interval.

[0149] Based on the priority assigned to each follow instruction's associated features, weights are distributed, and the association degree is obtained by weighted summation. The higher the association degree, the stronger the fit between the candidate and the follow trigger instruction.

[0150] Based on action features, determine the action feature sequence, including:

[0151] Extract action features of candidate guides from multiple frames of images within a continuous time period;

[0152] Multiple action features arranged in chronological order constitute a temporal action feature sequence;

[0153] The temporal action feature sequence is matched with a preset action sequence template to determine whether it is the target guide.

[0154] Among them, the temporal action feature sequence can reflect the user's movement trajectory and action trend (such as walking direction and speed), which can not only be used to determine the target guide, but also provide a reference for subsequent following path planning (such as adjusting the robot's following speed according to the user's walking action).

[0155] S304. Based on a preset facilitator selection model, determine the following probability of each candidate facilitator according to the correlation between the following instruction association features and the following trigger instruction, the confidence level of biometric features, and / or the action feature sequence;

[0156] The facilitator selects either a pre-trained machine learning model or a weighted computation model. The weighted computation model includes preset weight values ​​for relevance, confidence, and action feature sequences.

[0157] S305. Based on the following probability of each candidate facilitator, determine the target facilitator.

[0158] Understandably, the facilitator selection model is a pre-built and validated algorithmic model whose function is to integrate scattered, multi-dimensional evaluation metrics (relevance, confidence, action feature sequences) into a single probability value (0-1 range). The specific implementations of the two models are as follows:

[0159] Firstly, the weighted calculation model is suitable for basic cleaning equipment with low hardware costs and low computing power. Specifically, the calculation process uses correlation ([0,1]), confidence ([0,1]), and action feature sequence matching score (e.g., perfect match = 1, partial match = 0.3-0.8, no match = 0) as input parameters. The model has built-in preset weight values ​​(total weight = 1), allocated according to indicator priority (e.g., confidence weight 0.4, correlation weight 0.3, action matching score weight 0.3), and the weights can be adjusted according to the actual scenario. The following probability is directly calculated using a weighted summation formula: Following probability = Correlation × First weight + Confidence × Second weight + Action matching score × Third weight. For example, candidate A: correlation 0.8, confidence 0.9, action matching score 1.0, weighted calculation model (weights 0.3 / 0.4 / 0.3): Following probability = 0.8 × 0.3 + 0.9 × 0.4 + 1.0 × 0.3 = 0.9.

[0160] Secondly, the pre-trained machine learning model is adapted to cleaning equipment with high-performance hardware. The preferred machine learning models are logistic regression, random forest, and lightweight neural networks, which are pre-trained and optimized using massive amounts of sample data (association, confidence, action sequences, and whether corresponding labels represent the target guide in different scenarios). During use, the feature vectors of association, confidence, and action sequence are used as model input; the model outputs the following probability of candidates (in the range of 0-1), with the probability value directly reflecting the model's likelihood that a candidate is the target guide. After calculating the following probability of each candidate guide through the model, the candidate with the highest probability and ≥ a preset threshold (e.g., 0.7) is selected; if only one candidate has a probability ≥ the threshold, it is directly identified as the target guide, avoiding misjudgment or missed judgment. For example, candidate A has a relevance score of 0.8, a confidence score of 0.9, and an action matching score of 1.0; candidate B has a relevance score of 0.6, a confidence score of 0.7, and an action matching score of 0.5. The probability of candidate A is 0.92, and the probability of candidate B is 0.58. In the end, candidate A, who has a higher probability, is chosen.

[0161] In some embodiments, determining a target facilitator based on the follower probability of each candidate facilitator includes:

[0162] When the following probability of at least two candidate guides is greater than a preset probability threshold, the cleaning device is controlled to output interactive reminder information; wherein, the interactive reminder information is used to instruct the candidate guides to initiate an identity verification response;

[0163] Based on the identity verification responses obtained from the candidate facilitators, the target facilitator is determined.

[0164] Understandably, when the following probability of at least two candidate guides exceeds a preset threshold (such as 0.7), it means that the target guide cannot be clearly distinguished by the device's own model calculation alone. At this time, interactive reminders are used to encourage the candidate to respond actively, and the real target guide is locked in by combining the identity confirmation results.

[0165] For example, the cleaning equipment outputs interactive reminders through voice (such as asking the user who needs to follow to say "confirm follow"), flashing lights, or push notifications from the terminal APP, clearly instructing the candidate to initiate an identity verification response; the user responds in a preset way (such as saying a specified voice keyword, performing a preset action, or clicking "confirm" on the APP); the cleaning equipment receives and verifies the response information, and identifies the candidate who makes a valid response as the target guide; if only one candidate responds, it is directly locked; if multiple candidates respond, further filtering can be done based on response priority (such as pre-stored permission levels).

[0166] In some embodiments, the identity verification response is at least one of the following:

[0167] The candidate facilitator's body parts and / or planned movements; wherein the body parts are feet or legs, and the planned movements are foot or leg movements;

[0168] Voice commands or identity verification commands based on terminal devices.

[0169] Understandably, foot / leg movements do not require users to bend over or approach the device, which aligns with the characteristics of cleaning robots working on the ground. Furthermore, the foot / leg movements are simple and intuitive, requiring no additional learning. Voice and terminal commands are commonly used interaction methods, requiring no additional adaptation from users and reducing usage costs.

[0170] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0171] Within a preset time period after the cleaning equipment outputs interactive reminder information, the cleaning equipment is controlled to collect voice data from each candidate guide;

[0172] Based on speech data, extract the voiceprint features, spatial information of the sound source, and / or semantic information of the speech data of the candidate guides;

[0173] The target guide is identified based on voiceprint features, spatial information of the sound source, and / or semantic information of the speech data.

[0174] This embodiment integrates voiceprint features (unique identity), sound source spatial information, and semantic information to locate the target from three dimensions, avoiding misjudgments caused by single voice recognition. Voice response requires no additional learning of complex operations by the user, conforms to daily communication habits, and supports scenarios where physical operation is inconvenient, such as at a distance or when holding objects with both hands, thus improving ease of use.

[0175] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0176] If no identity verification response is received within the preset time, or if the received identity verification response cannot be used to select a unique target guide, the cleaning device will be controlled to perform an end-screening operation. The end-screening operation includes: outputting a failure message, exiting follow mode, or selecting the candidate guide with the highest follow probability as the target guide.

[0177] It is understandable that setting a preset time threshold and automatically ending the screening process when no identity verification response is received can prevent the cleaning equipment from being stuck in the verification process for a long time. Ending the screening process in a timely manner avoids continuously consuming computing power and electricity for identity verification, thus balancing the accuracy of target tracking with energy consumption costs.

[0178] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0179] If no identity verification response is received within the preset time, or if the received identity verification response cannot be used to filter out a unique target guide, the cleaning device will output a second interactive reminder message based on the preset interactive reminder message rules; the interactive reminder message is used to instruct the candidate guide to initiate a second identity verification response.

[0180] The target facilitator is determined based on the secondary identity verification response output by the obtained candidate facilitators.

[0181] The secondary identity verification response can be initiated multiple times, and the content and mode of each response can be the same or different.

[0182] Understandably, providing users with two (or even multiple) opportunities to respond avoids screening failures due to unclear hearing or operational delays, thus increasing the success rate of identifying the target guide. The content / mode of each response can be flexibly adjusted (e.g., the initial voice prompt "speak to confirm," the second "raise your foot to confirm") to adapt to different user operating habits and lower the response threshold. Furthermore, combining multiple rounds of identity verification results eliminates interference and reduces the risk of misjudgment. Of course, the number of reminder responses can also be adjusted based on the device's remaining battery power; for example, initiating one secondary identity verification response when the battery is low, and multiple secondary identity verification responses when the battery is high.

[0183] S306. Real-time acquisition of dynamic tracking data of the target guide, and control of cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics;

[0184] In some embodiments, real-time dynamic tracking data of the target guide is acquired, and the cleaning equipment is controlled to follow the target guide's movement based on the dynamic tracking data, including:

[0185] If at least one interfering object is detected during the tracking process, the following operations are performed:

[0186] Extract similar characteristics between the target guide and the interference object, including biological and / or non-biological characteristics;

[0187] The extracted similar features are compared with the pre-stored benchmark feature library of the target facilitator;

[0188] The cleaning equipment is controlled to continue following the object with the highest similarity to the baseline feature library.

[0189] In this embodiment, if an interfering object (such as another person) is detected during the follow-up process, similar features (biological features such as the lower body and non-biological features such as clothing color) of the target guide and the interfering object are extracted and compared with the target baseline feature library. The device is then controlled to continue following the object with the highest similarity, avoiding being led astray by the interfering object. In this way, by comparing similar features, the target and the interfering object are accurately distinguished. Even in interference scenarios, the target can still be locked based on feature matching, ensuring that the follow-up process is uninterrupted and adapting to complex scenarios such as families with multiple people and multiple pets.

[0190] In some embodiments, real-time dynamic tracking data of the target guide is acquired, and the cleaning equipment is controlled to follow the target guide's movement based on the dynamic tracking data, including:

[0191] When the real-time dynamic tracking data is a local component of the target guide's preset biometric features, the missing parts of the preset biometric features are supplemented based on the local component features;

[0192] Based on the completed biometric and / or non-biometric features, match the real-time acquired dynamic tracking data with the pre-stored benchmark feature library of the target guide.

[0193] When the real-time acquired dynamic tracking data matches the dynamic tracking data, based on the completed biometric and / or non-biometric features, the cleaning equipment is controlled to follow the target guide so that the distance between the cleaning equipment and the target guide is a preset distance.

[0194] When the real-time dynamic tracking data does not match the target guide search operation, a preset target guide search operation is performed.

[0195] The system acquires dynamic tracking data of the target in real time during the follow-up process. If the data is only a partial component of the preset biometric features (such as part of a limb), the missing part is filled in by the algorithm. The completed biometric features or non-biometric features (such as clothing color) are then compared with the pre-stored target benchmark feature library. If the comparison matches, the device is controlled to maintain a preset distance for following. If the comparison does not match (such as the target is lost or mistakenly following another person), a preset target search operation is initiated.

[0196] This embodiment completes local biometric features, avoiding tracking failures due to feature loss caused by occlusion or shooting angle issues, and is suitable for complex mobile scenarios in the home.

[0197] S307. When a cleaning instruction issued by the target guide is detected to match a preset cleaning instruction, the cleaning task corresponding to the cleaning instruction is executed.

[0198] In some embodiments, the cleaning instruction is the behavior of the target guide staying in a preset area;

[0199] Perform cleaning tasks corresponding to the cleaning instructions, including:

[0200] Based on the duration of the target facilitator's stay, determine the cleaning pattern and / or cleaning area to be used for the cleaning task.

[0201] In some embodiments, when a cleaning instruction issued by a target facilitator is detected, a cleaning task corresponding to the cleaning instruction is performed, including:

[0202] Upon detecting a cleaning instruction issued by the target guide, the cleaning device outputs a task confirmation request;

[0203] After receiving a task confirmation response from the target facilitator, execute the cleaning task corresponding to the cleaning instruction.

[0204] Understandably, as the cleaning device follows the user, it continuously detects whether the user has issued a cleaning instruction via audio, visual, or terminal communication modules. Upon detecting an instruction, the device doesn't execute it directly. Instead, it sends a task confirmation request to the user via voice (e.g., "Do you want to clean the current area?"), flashing lights, or a push notification from the terminal, clearly informing them of the upcoming cleaning task. The device only executes the cleaning task upon receiving a valid confirmation response from the user (e.g., voice confirmation or app confirmation); otherwise, it doesn't execute the task. This avoids erroneous execution and improves accuracy.

[0205] This application also provides apparatus embodiments that follow the above embodiments, for implementing the method steps of the above embodiments. The interpretation of the same names is the same as that of the above embodiments, and they have the same technical effects as those of the above embodiments, so they will not be repeated here.

[0206] As shown in Figure 4, this application provides a cleaning control device for cleaning equipment, the device comprising:

[0207] The target search unit 401 is configured to respond to a follow trigger command, search for candidate guides in the target scene, and extract the follow command association features, biometric features and / or action features of each candidate guide;

[0208] The feature processing unit 402 is used to determine the correlation between the follow instruction association features and the follow triggering instructions based on the follow instruction association features of the candidate facilitator; and / or to determine the confidence level of the biometric features based on the biometric features; and / or to determine the action feature sequence based on the action features.

[0209] The target locking unit 403 is configured to determine the target guide based on the correlation between the follow instruction association features of the candidate guide and the follow trigger instruction, the confidence level of the biometric features and / or the action feature sequence.

[0210] The dynamic following unit 404 is configured to acquire dynamic tracking data of the target guide in real time, and control the cleaning equipment to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics.

[0211] The cleaning execution unit 405 is configured to execute a cleaning task corresponding to a cleaning instruction issued by a target guide when a cleaning instruction is detected.

[0212] In some embodiments, the target locking unit 403 is used to control the cleaning equipment to move within the target scene and to collect environmental data in real time;

[0213] When the environmental data includes human body contour features, candidate guides are determined based on these features.

[0214] In some embodiments, the target search unit 401 is further configured to control the cleaning device to output a camera entry reminder message at a preset time interval when the environmental data does not include human body contour features;

[0215] When human body contour features are detected, candidate guides are determined based on these features.

[0216] When the number of times the on-camera notification is displayed reaches the preset limit, the search for a guide is deemed to have failed.

[0217] The "appearance alert" message is used to remind users to move into the image capture range of the cleaning equipment.

[0218] In some embodiments, the target locking unit 403 is used to determine the following probability of each candidate guide based on a preset guide selection model, according to the correlation between the follow instruction association features and the follow trigger instruction, the confidence level of biometric features, and / or the action feature sequence of each candidate guide;

[0219] The target facilitator is determined based on the follow probability of each candidate facilitator.

[0220] In some embodiments, the facilitator selects a pre-trained machine learning model or a weighted computation model, wherein the weighted computation model includes preset weight values ​​for relevance, confidence, and action feature sequences.

[0221] In some embodiments, the target locking unit 403 is used to control the cleaning device to output interactive reminder information when the following probability of at least two candidate guides is greater than a preset probability threshold; wherein, the interactive reminder information is used to instruct the candidate guides to initiate an identity verification response;

[0222] Based on the identity verification responses obtained from the candidate facilitators, the target facilitator is determined.

[0223] In some embodiments, the identity verification response is at least one of the following:

[0224] The candidate facilitator's body parts and / or planned movements; wherein the body parts are feet or legs, and the planned movements are foot or leg movements;

[0225] Voice commands or identity verification commands based on terminal devices.

[0226] In some embodiments, determining the target facilitator based on the obtained identity verification response of the candidate facilitator includes:

[0227] If no identity verification response is received within the preset time, or if the received identity verification response cannot be used to select a unique target guide, the cleaning device will be controlled to perform an end-screening operation. The end-screening operation includes: outputting a failure message, exiting follow mode, or selecting the candidate guide with the highest follow probability as the target guide.

[0228] In some embodiments, the target locking unit 403 is configured to control the cleaning device to output interactive reminder information a second time based on preset interactive reminder information rules if no identity verification response is received within a preset time, or if a unique target guide cannot be selected based on the received identity verification response; wherein the interactive reminder information is used to instruct the candidate guide to initiate a second identity verification response;

[0229] The target facilitator is determined based on the secondary identity verification response output by the obtained candidate facilitators.

[0230] In some embodiments, the target locking unit 403 is used to control the cleaning device to collect voice data of each candidate guide within a preset time period after the cleaning device outputs interactive reminder information;

[0231] Based on speech data, extract the voiceprint features, spatial information of the sound source, and / or semantic information of the speech data of the candidate guides;

[0232] The target guide is identified based on voiceprint features, spatial information of the sound source, and / or semantic information of the speech data.

[0233] In some embodiments, the follow-up trigger instruction is at least one of the following:

[0234] Control commands issued by the terminal device application;

[0235] Preset voice control commands;

[0236] Control commands based on a video interactive interface.

[0237] In some embodiments, the method further includes:

[0238] In response to the received follow-trigger command, if the cleaning equipment is not located in the target scene, control the cleaning equipment to move to the target scene.

[0239] In some embodiments, the dynamic tracking unit is configured to perform the following operations if at least one interfering object is detected during the tracking process:

[0240] Extract similar characteristics between the target guide and the interference object, including biological and / or non-biological characteristics;

[0241] The extracted similar features are compared with the pre-stored benchmark feature library of the target facilitator;

[0242] The cleaning equipment is controlled to continue following the object with the highest similarity to the baseline feature library.

[0243] In some embodiments, the dynamic tracking unit is used to supplement the missing parts of the preset biometrics based on the local component features when the real-time obtained dynamic tracking data is a local component feature of the preset biometrics of the target guide.

[0244] Based on the completed biometric and / or non-biometric features, match the real-time acquired dynamic tracking data with the pre-stored benchmark feature library of the target guide.

[0245] When the real-time acquired dynamic tracking data matches the dynamic tracking data, based on the completed biometric and / or non-biometric features, the cleaning equipment is controlled to follow the target guide so that the distance between the cleaning equipment and the target guide is a preset distance.

[0246] When the real-time dynamic tracking data does not match the target guide search operation, a preset target guide search operation is performed.

[0247] In some embodiments, the cleaning execution unit 405 is configured to execute a cleaning task corresponding to a cleaning instruction when it detects that a cleaning instruction issued by a target guide matches a preset cleaning instruction.

[0248] In some embodiments, the cleaning instruction is the behavior of the target guide staying in a preset area;

[0249] The cleaning execution unit 405 is used to determine the cleaning mode and / or cleaning area to be used for the cleaning task based on the duration of the target guide's stay.

[0250] In some embodiments, the cleaning execution unit 405 is configured to output a task confirmation request when a cleaning instruction is detected from a target guide;

[0251] After receiving a task confirmation response from the target facilitator, execute the cleaning task corresponding to the cleaning instruction.

[0252] In some embodiments, the follow instruction associated features of the candidate facilitator include at least one of the following:

[0253] The distance between the candidate facilitator and the sound source that triggers the follow-up command;

[0254] The distance between the candidate guide and the cleaning equipment;

[0255] The azimuth angle of the candidate guide relative to the direction of travel of the cleaning equipment;

[0256] The time difference between the time the trigger command is received and the time the candidate leader is identified;

[0257] The matching results of the candidate guide's identity information with the pre-stored priority follow permissions.

[0258] As shown in Figure 4, this embodiment provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by a processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method steps of the above embodiment.

[0259] This application provides a non-volatile computer storage medium storing computer-executable instructions that can execute the method steps of the above embodiments.

[0260] Referring now to Figure 5, a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of this application is shown. The terminal devices in the embodiments of this application may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in Figure 5 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0261] As shown in Figure 5, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0262] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5 shows an electronic device with various devices, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0263] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of the embodiments of this application.

[0264] It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0265] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0266] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0267] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0268] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

Claims

1. A cleaning control method for cleaning equipment, characterized in that, The method includes: responding to a follow-triggered command, searching for candidate guides within a target scene, and extracting follow-instruction association features, biometric features, and / or action features of each candidate guide; determining the correlation degree between the follow-instruction association features and the follow-triggered command based on the follow-instruction association features of the candidate guides; and / or determining the confidence level of the biometric features based on the biometric features; and / or determining the action feature sequence based on the action features; determining a target guide based on the correlation degree between the follow-instruction association features and the follow-triggered command, the confidence level of the biometric features, and / or the action feature sequence of the candidate guides; acquiring dynamic tracking data of the target guide in real time, and controlling the cleaning device to follow the target guide based on the dynamic tracking data; wherein the dynamic tracking data includes the biometric features and / or non-biological features of the target guide; and executing a cleaning task corresponding to the cleaning command when a cleaning command issued by the target guide is detected.

2. The method according to claim 1, characterized in that, The process of searching for and selecting guides within the target scene includes: controlling the cleaning equipment to move within the target scene and collecting environmental data in real time; when the environmental data includes human contour features, determining candidate guides based on the human contour features.

3. The method according to claim 2, characterized in that, Also includes: When the environmental data does not include human silhouette features, the cleaning device is controlled to output a camera entry reminder message at a preset time interval; when human silhouette features are detected, a candidate guide is determined based on the human silhouette features; when the number of times the camera entry reminder is output reaches a preset limit, the search for a guide is deemed to have failed; wherein, the camera entry reminder message is used to remind the user to move into the image acquisition range of the cleaning device.

4. The method according to claim 1, characterized in that, The step of determining the target guide based on the correlation degree between the follow instruction association features and the follow trigger instruction of the candidate guides, the confidence level of the biometric features, and / or the action feature sequence includes: determining the follow probability of each candidate guide based on a preset guide selection model, according to the correlation degree between the follow instruction association features and the follow trigger instruction of each candidate guide, the confidence level of the biometric features, and / or the action feature sequence; and determining the target guide based on the follow probability of each candidate guide.

5. The method according to claim 4, characterized in that, The facilitator selection model is a pre-trained machine learning model or a weighted computation model, wherein the weighted computation model includes preset weight values ​​for the correlation, confidence, and action feature sequences.

6. The method according to claim 4, characterized in that, The step of determining the target guide based on the following probability of each of the candidate guides includes: when the following probability of at least two of the candidate guides is greater than a preset probability threshold, controlling the cleaning device to output interactive reminder information; wherein, the interactive reminder information is used to instruct the candidate guide to initiate an identity verification response; and determining the target guide based on the obtained identity verification response of the candidate guide.

7. The method according to claim 6, characterized in that, The identity verification response is at least one of the following: a body part and / or a predetermined action of the candidate guide; wherein the body part is a foot or leg, and the predetermined action is a foot or leg action; a voice command or an identity verification command based on a terminal device.

8. The method according to claim 6, characterized in that, The step of determining the target guide based on the obtained identity verification response of the candidate guide includes: if no identity verification response is received within a preset time, or if the unique target guide cannot be selected based on the received identity verification response, then controlling the cleaning device to perform an end-screening operation; wherein, the end-screening operation includes: outputting a failure prompt message, exiting the follow mode, or selecting the candidate guide with the highest follow probability as the target guide.

9. The method according to claim 7, characterized in that, The step of determining the target guide based on the obtained identity verification response of the candidate guide includes: if no identity verification response is received within a preset time, or if the unique target guide cannot be selected based on the received identity verification response, then controlling the cleaning device to output interactive reminder information a second time based on preset interactive reminder information rules; wherein, the interactive reminder information is used to instruct the candidate guide to initiate a second identity verification response; and determining the target guide based on the obtained second identity verification response output by the candidate guide.

10. The method according to claim 6, characterized in that, The step of determining the target guide based on the obtained identity verification response of the candidate guide includes: controlling the cleaning device to collect voice data of each candidate guide within a preset time period after the cleaning device outputs interactive reminder information; extracting the voiceprint features, sound source spatial information and / or semantic information of the voice data of the candidate guide based on the voice data; and determining the target guide based on the voiceprint features, the sound source spatial information and / or the semantic information of the voice data.

11. The method according to claim 1, characterized in that, The follow-trigger instruction is at least one of the following: a control instruction issued by a terminal device application; a preset voice control instruction; or a control instruction based on a video interactive interface.

12. The method according to claim 1, characterized in that, The method further includes: responding to the received follow-trigger command, if the cleaning device is not located in the target scene, controlling the cleaning device to move to the target scene.

13. The method according to claim 1, characterized in that, The real-time acquisition of dynamic tracking data of the target guide, and the control of the cleaning device to follow the target guide based on the dynamic tracking data, includes: during the following process, if at least one interfering object is detected, the following operations are performed: extracting similar features of the target guide and the interfering object, the similar features including biological features and / or non-biological features; comparing the extracted similar features with a pre-stored benchmark feature library of the target guide; and controlling the cleaning device to continue following the object with the highest similarity to the benchmark feature library.

14. The method according to claim 1, characterized in that, The real-time acquisition of dynamic tracking data of the target guide, and the control of the cleaning device to follow the target guide based on the dynamic tracking data, includes: when the real-time acquired dynamic tracking data is a local component of the target guide's preset biometric features, supplementing the missing parts of the preset biometric features based on the local component features; matching the real-time acquired dynamic tracking data with a pre-stored baseline feature library of the target guide based on the supplemented biometric features and / or non-biometric features; when the real-time acquired dynamic tracking data matches the dynamic tracking data, controlling the cleaning device to follow the target guide based on the supplemented biometric features and / or non-biometric features, so that the distance between the cleaning device and the target guide is a preset distance; when the real-time acquired dynamic tracking data does not match the dynamic tracking data, performing a preset target guide search operation.

15. The method according to claim 1, characterized in that, When a cleaning instruction issued by the target guide is detected, a cleaning task corresponding to the cleaning instruction is executed, including: when a cleaning instruction issued by the target guide is detected to match a preset cleaning instruction, a cleaning task corresponding to the cleaning instruction is executed.

16. The method according to claim 15, characterized in that, The cleaning instruction refers to the target guide's behavior of staying in the preset area; The execution of the cleaning task corresponding to the cleaning instruction includes: determining the cleaning mode and / or cleaning area to be used for the cleaning task based on the duration of the target guide's stay.

17. The method according to claim 1, characterized in that, The step of executing a cleaning task corresponding to a cleaning instruction issued by the target guide when the cleaning instruction is detected includes: after the cleaning instruction issued by the target guide is detected, the cleaning device outputs a task confirmation request; after receiving a task confirmation response from the target guide, the cleaning task corresponding to the cleaning instruction is executed.

18. The method according to any one of claims 1 to 17, characterized in that, The following instruction association features of the candidate guide include at least one of the following: the sound source distance between the candidate guide and the follow trigger instruction; the distance between the candidate guide and the cleaning equipment; the azimuth angle of the candidate guide relative to the direction of travel of the cleaning equipment; and the time difference between the time of receiving the follow trigger instruction and the time when the candidate guide is identified. The matching result between the candidate guide's identity information and the pre-stored priority follow permissions.

19. A cleaning control device for a cleaning equipment, characterized in that, The device includes: a target search unit configured to respond to a follow-triggered command, search for candidate guides within a target scene, and extract follow-instruction association features, biometric features, and / or action features of each candidate guide; a feature processing unit configured to determine the correlation between the follow-instruction association features and the follow-triggered command based on the follow-instruction association features of the candidate guides; and / or determine the confidence level of the biometric features based on the biometric features; and / or determine the action feature sequence based on the action features; a target locking unit configured to determine a target guide based on the correlation between the follow-instruction association features and the follow-triggered command, the confidence level of the biometric features, and / or the action feature sequence of the candidate guides; a dynamic following unit configured to acquire dynamic tracking data of the target guide in real time, and control the cleaning device to follow the target guide based on the dynamic tracking data; wherein the dynamic tracking data includes the biometric features and / or non-biological features of the target guide; and a cleaning execution unit configured to execute a cleaning task corresponding to the cleaning command when a cleaning command issued by the target guide is detected.

20. A cleaning device, characterized in that, include: The main body of the cleaning equipment and the cleaning control device are mounted on the main body of the cleaning equipment. The cleaning control device is configured to: respond to a follow-triggered command, search for candidate guides within a target scene, and extract follow-triggered command association features, biometric features, and / or action features of each candidate guide; determine the correlation between the follow-triggered command association features and the follow-triggered command based on the follow-triggered command association features of the candidate guides; and / or determine the confidence level of the biometric features based on the biometric features; and / or determine the action feature sequence based on the action features; and determine the target guide based on the correlation between the follow-triggered command association features and the follow-triggered command, the confidence level of the biometric features, and / or the action feature sequence of the candidate guides. The system acquires real-time dynamic tracking data of the target guide and controls the cleaning device to follow the target guide's movement based on the dynamic tracking data; wherein, the dynamic tracking data includes the target guide's biometrics and / or non-biometrics; when a cleaning instruction is detected from the target guide, the system executes the cleaning task corresponding to the cleaning instruction.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 18.

22. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 18.

Images