Job order determination method and apparatus, electronic device, and storage medium

Abstract

This disclosure provides a method, apparatus, electronic device, and storage medium for determining the work sequence. The method includes: determining the task priority of each work area based on the number of work units in each work area and the work task information of each work unit in the work area; determining the location priority of each work area based on its location information; and determining the work sequence of the multiple work areas based on the task priority and location priority of each work area. Based on this implementation, the robot can plan the work sequence according to the actual situation of the area to be cleaned, allowing the robot's work route planning to fully consider the actual situation of the environment, thus improving the rationality of the robot's work sequence and work path planning.

Description

Technical Field

[0001] This disclosure relates to the field of robot operation sequence technology, and in particular to operation sequence determination methods, devices, electronic devices and storage media. Background Technology

[0002] Typically, due to logical and spatial reasons, the target cleaning area of ​​a cleaning robot is divided into multiple sub-areas. For example, for a robotic vacuum cleaner operating in a home environment, bedrooms, living rooms, dining rooms, and studies naturally form multiple working areas. Similarly, for commercial cleaning robots operating in a factory environment, consumable storage rooms, production workshops, and unloading areas also form multiple working areas.

[0003] In related technologies, cleaning robots typically rely on a user-preset sequence to mechanically clean multiple target cleaning areas. However, such solutions cannot meet users' increasingly higher demands for quality of life, resulting in a poor user experience. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, electronic device and storage medium for determining the order of operations.

[0005] The first aspect of this disclosure provides a method for determining a job sequence, the method comprising:

[0006] The task priority of each waiting area is determined based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area.

[0007] Based on the location information of each area to be worked on, the location priority of the area to be worked on is determined;

[0008] The order of operations for the multiple areas to be worked on is determined based on the task priority and location priority of each area.

[0009] Optionally, the location information includes at least one parameter value represented numerically, and determining the location priority of each area to be worked on based on the location information includes:

[0010] For each parameter value of the location information, the parameter values ​​of the multiple areas to be worked are normalized.

[0011] For each area to be worked on, the sum of the normalized results of each parameter value of the location information of the area to be worked on is used as the location priority of the area to be worked on.

[0012] Optionally, determining the work order of the plurality of work areas based on the task priority and location priority of each work area includes:

[0013] The order of operations for the multiple areas to be worked is determined based on the task priority and location priority of each area to be worked, as well as the weights corresponding to the task priority and location priority.

[0014] Optionally, the task information of the unit to be worked on is used at least to determine the work actions of the robot when performing work on the unit to be worked on;

[0015] And / or, the location information includes at least one of the following:

[0016] The distance between the work area and the robot's starting point, the area of ​​the work area, and the geometric features of the work area.

[0017] Optionally, the task information of the unit to be worked on includes at least one of the following:

[0018] The type of dirt in the unit to be worked on, the area of ​​the unit to be worked on, the location characteristics of the unit to be worked on, the ground material covered by the unit to be worked on, and whether there are animals in the work area corresponding to the unit to be worked on.

[0019] Optionally, the type of dirt in the unit to be worked on includes at least one of the following:

[0020] Solid dirt, liquid dirt, stubborn dirt;

[0021] And / or, the positional characteristics of the unit to be worked on are used to characterize:

[0022] Whether the distance between the edge of the area corresponding to the unit to be worked and the obstacle is less than a preset threshold.

[0023] Optionally, determining the task priority of a task-to-work area based on the number of task-to-work units in each of the multiple task-to-work areas and the task information of each task-to-work unit in the task-to-work area includes:

[0024] For each of the multiple pending task areas:

[0025] The task information of each unit to be worked in the work area is combined according to a preset rule to obtain combined text information;

[0026] The combined text information is processed based on a preset feature representation method to obtain initial features;

[0027] The initial features are subjected to dimensionality reduction processing to obtain a task feature vector of a preset dimension;

[0028] The task feature vector of the region to be worked on is input into a pre-trained model to determine the task priority of the region to be worked on.

[0029] Optionally, the method further includes:

[0030] The robot is controlled to cruise within a preset area, and visual information of the preset area is collected based on the vision sensors mounted on the robot, wherein the preset area covers the multiple areas to be worked on;

[0031] Based on the visual information, the units to be worked in the preset area and the task information of each unit to be worked are determined.

[0032] Optionally, the method further includes:

[0033] Each preset number of work areas corresponding to the work units to be done within the preset area is taken as one of the multiple work areas to be done.

[0034] A second aspect of this disclosure provides a work sequence determination apparatus, the apparatus comprising:

[0035] The task priority module is used to determine the task priority of the waiting area based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area.

[0036] The location priority module is used to determine the location priority of each area to be worked on based on its location information.

[0037] The sequence determination module is used to determine the operation sequence of the multiple areas to be worked based on the task priority and location priority of each area to be worked.

[0038] Optionally, the location information includes at least one parameter value represented numerically, and the location priority module is used to determine the location priority of each work area based on its location information, specifically for:

[0039] For each parameter value of the location information, the parameter values ​​of the multiple areas to be worked are normalized.

[0040] For each area to be worked on, the sum of the normalized results of each parameter value of the location information of the area to be worked on is used as the location priority of the area to be worked on.

[0041] Optionally, when the sequence determination module determines the work order of the multiple work areas based on the task priority and location priority of each work area, it is specifically used for:

[0042] The order of operations for the multiple areas to be worked is determined based on the task priority and location priority of each area to be worked, as well as the weights corresponding to the task priority and location priority.

[0043] Optionally, the task information of the unit to be worked on is used at least to determine the robot's actions when performing work on the unit to be worked on; and / or,

[0044] The location information includes at least one of the following:

[0045] The distance between the work area and the robot's starting point, the area of ​​the work area, and the geometric features of the work area.

[0046] Optionally, the task information of the unit to be worked on includes at least one of the following:

[0047] The type of dirt in the unit to be worked on, the area of ​​the unit to be worked on, the location characteristics of the unit to be worked on, the ground material covered by the unit to be worked on, and whether there are animals in the work area corresponding to the unit to be worked on.

[0048] Optionally, the type of dirt in the unit to be worked on includes at least one of the following:

[0049] Solid dirt, liquid dirt, stubborn dirt;

[0050] And / or, the positional characteristics of the unit to be worked on are used to characterize:

[0051] Whether the distance between the edge of the area corresponding to the unit to be worked and the obstacle is less than a preset threshold.

[0052] Optionally, the task priority module is used to determine the task priority of a task-to-work area based on the number of task-to-work units in each of the multiple task-to-work areas and the task information of each task-to-work unit in the task-to-work area. Specifically, it is used for:

[0053] For each of the multiple pending task areas:

[0054] The task information of each unit to be worked in the work area is combined according to a preset rule to obtain combined text information;

[0055] The combined text information is processed based on a preset feature representation method to obtain initial features;

[0056] The initial features are subjected to dimensionality reduction processing to obtain a task feature vector of a preset dimension;

[0057] The task feature vector of the region to be worked on is input into a pre-trained model to determine the task priority of the region to be worked on.

[0058] Optionally, the device further includes:

[0059] A cruise module is used to control the robot to cruise within a preset area and to collect visual information of the preset area based on a vision sensor mounted on the robot, wherein the preset area covers the plurality of areas to be worked on.

[0060] The task determination module is used to determine the work units to be done within the preset area and the work task information of each work unit based on the visual information.

[0061] Optionally, the device further includes:

[0062] The work area determination module is used to determine the work area corresponding to each preset number of work units within the preset area as one of the multiple work areas.

[0063] A third aspect of this disclosure provides a computer program product including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0064] A fourth aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in the first aspect.

[0065] The fifth aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0066] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0067] This disclosure provides a method for determining the work sequence of a work area by comprehensively considering the work tasks within the work area and the locational characteristics of the work area itself. Based on this implementation, the robot can plan the work sequence according to the actual situation of the area to be cleaned, so that the robot's work route planning can fully consider the actual situation of the environment, improving the rationality of the robot's work sequence and work path planning.

[0068] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0070] Figure 1 This is a flowchart illustrating a method for determining the job sequence, as shown in some exemplary embodiments.

[0071] Figure 2 These are schematic diagrams illustrating a method for determining the job sequence, as shown in some exemplary embodiments.

[0072] Figure 3 These are schematic diagrams illustrating a cruise method using some exemplary embodiments.

[0073] Figure 4 These are schematic diagrams illustrating the structure of a pre-trained model, showcasing some exemplary embodiments.

[0074] Figure 5 This is a block diagram illustrating a job sequence determination apparatus according to some exemplary embodiments.

[0075] Figure 6 These are hardware structure diagrams of an electronic device illustrated by some exemplary embodiments. Detailed Implementation

[0076] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0077] As described in the background section, in related technologies, cleaning robots typically rely on a user-preset sequence to mechanically clean multiple target cleaning areas. However, firstly, such a strategy may cause the robot to perform a lot of meaningless cleaning during the cleaning process, while truly urgent cleaning needs (such as milk spilled in a room, which may cause the floor to smell bad if not cleaned up in time) cannot be addressed in a timely manner.

[0078] In view of the above, this disclosure provides a method, apparatus, electronic device, and storage medium for determining the order of operations. The embodiments of this disclosure will now be described in detail.

[0079] The first aspect of this disclosure provides a method for determining the work sequence, which can be applied to cleaning robots (e.g., sweeping robots), industrial obstacle removal robots, and other operational robots. Please see [link to relevant documentation]. Figure 1 It may include steps S101 to S103.

[0080] Step S101: Determine the task priority of the waiting area based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area.

[0081] The following text is based on Figure 2 The scenario illustrated below illustrates the embodiments of this disclosure. In the above steps, the work area is the area that the robot needs to go to for work, as indicated by the robot's work process. For example, in a typical household scenario, users usually set the robot vacuum cleaner to clean rooms such as the bedroom, living room, bathroom, and kitchen after they leave home. These rooms to be cleaned can be regarded as different work areas, meaning the robot needs to perform cleaning tasks on four work areas: the bedroom, living room, bathroom, and kitchen. Of course, it should be understood that the work area can also include multiple rooms, or only a part of a room, or it can be distinguished not based on "rooms." For example, it can be divided according to furniture (instructing the robot to clean a carpet), according to floors, according to the preset work area shape (for example, dividing a room into several rectangular areas and using each rectangular area as a work area for easy cleaning), or according to user instructions. This disclosure does not limit this.

[0082] A workflow typically includes one or more task units, which can be categorized based on the object being cleaned by the robot. For example, cleaning a room with a robotic vacuum cleaner might involve collecting dust from the entire room and then selectively cleaning specific dirt. Specifically, for a spilled milk puddle, the robot might need to mop it up; for eraser shavings scattered on the floor, the robot might need to increase the suction power of its dust collection system to collect them completely; and for scratches caused by a chair dragging on the floor (due to dust, dirt on the chair legs, or friction between the material and the floor), the robot might need to repeatedly clean them using a rotating brush. In this disclosure, these dirt items can be considered different task units because they represent a complete unit targeted by the robot's work.

[0083] It should be understood that the specific method for determining the work units in this disclosure can be based on a preset large model or a preset recognition algorithm, and the specific judgment criteria can be determined according to the application scenario and functional design of the robot. For example, some work units that are not specific or targeted, but are predictable and generally exist in each work area, can be ignored in the method provided in this disclosure. For example, in areas of a room that are not dirty, the robot only needs to perform ordinary cleaning, and this is the case for every room (and it can be assumed that in a typical home scenario, each room will have some areas that are not dirty). In this case, only the dirty areas in the work area can be used as work units and used to determine the task priority, while the areas that are not dirty are not used to calculate the task priority (the influence of these areas without dirt on the work order of the work area can be reflected by the "location priority" mentioned later).

[0084] The task information of the unit to be worked on is information related to the task of the unit to be worked on. Optionally, the task information of the unit to be worked on can at least be used to determine the robot's work actions (work process) when performing work on the unit to be worked on. For example, the task information of the unit to be worked on includes at least one of the following: the type of dirt on the unit to be worked on, the area of ​​the unit to be worked on, the location characteristics of the unit to be worked on, the ground material covered by the unit to be worked on, and whether there are animals in the work area corresponding to the unit to be worked on. Information in these dimensions can greatly improve the flexibility of robot route planning and its adaptability to the environment. These optional dimensions are described below.

[0085] The type of dirt in the task unit can refer to the classification of the task unit in a preset dimension. For example, the type of dirt in the task unit can include at least one of the following: solid dirt (such as the eraser shavings mentioned above), liquid dirt (such as the milk mentioned above), and stubborn dirt (such as viscous substances like honey, dried liquid stains, and the wear marks on the chair legs mentioned above; stubborn dirt usually corresponds to repetitive tasks or complex work processes). It should be understood that the type of dirt can also have more refined dimensions, such as whether the task object targeted by the task unit is prone to spoilage or odor, or whether the task unit involves hair (which may need to be cut), etc. For example, the rate at which dirt spoils can be positively correlated with the task priority corresponding to that dirt, that is, the robot can be made to prioritize the processing of dirt that is more prone to spoilage and odor.

[0086] The area of ​​the unit to be worked on can be the area of ​​the work area corresponding to the unit, the coverage area of ​​the unit, or the projected area on the ground. The "work area corresponding to the unit" can be the same as or different from the coverage area or the projected area on the ground. This is because a robot is a physical entity that occupies a certain space, and the installation position and cleaning actions of the cleaning components mounted on the robot may differ. Therefore, for actual operation, the work area corresponding to the unit can be different from the area covered by the unit. For example, the dirt may be circular, but the cleaning area when the robot cleans the dirt is the circumscribed rectangle of that circle. The specific method for determining this area can be related to the robot's specific structural and functional design, which will not be elaborated upon in this disclosure.

[0087] The positional characteristics of the unit to be cleaned can be used to characterize whether the distance between the edge of the area corresponding to the unit and an obstacle is less than a preset threshold. Here, obstacles can refer to walls, table legs, chair legs, etc. In simpler terms, for a robotic vacuum cleaner, the positional characteristics of the unit to be cleaned can be understood as whether the unit is "against a wall" (areas and corners near walls are usually difficult to clean, so the robotic vacuum cleaner's handling logic for dirt near walls is usually different from that for dirt not near walls).

[0088] The floor material to be covered by the cleaning unit can be categorized based on its chemical and physical properties. Chemically, the cleaning agents used by the robot vacuum may be corrosive to some materials, such as leather and wood. Physically, some of the robot's actions may damage certain materials; for example, while stiff rotating brushes are suitable for cleaning stubborn stains, they may leave scratches on glass and tiles. Furthermore, carpets and other fabrics are highly absorbent of liquids; if milk is spilled on them and not cleaned up promptly, it may cause the carpet to smell bad (while materials like tiles are less prone to this). Therefore, the floor material covered by the cleaning unit can actually affect the robot's actions, thus impacting the overall workflow and the difficulty of the task.

[0089] The presence or absence of animals in the work area corresponding to the unit to be worked on may also affect the robot's overall work process and difficulty. For example, pets may interfere with the robot's work, and the robot's work may also disturb the pets. Therefore, for example, the presence of animals can correspond to a lower task priority (that is, the robot may tend to go to areas where there are no animals first), and the absence of animals can correspond to a higher task priority.

[0090] In step S101, the "multiple work areas" can be pre-divided, for example, rooms such as the "living room" and "bedroom" can be pre-defined as work areas. Alternatively, the "multiple work areas" can be divided based on the work units after the robot has first cruised within a preset area and identified them. For example, the method may further include: controlling the robot to cruise within the preset area and collecting visual information of the preset area based on a vision sensor mounted on the robot, wherein the preset area covers the multiple work areas; and determining the work units within the preset area and the work task information of each work unit based on the visual information.

[0091] In other words, the robot can have pre-recorded maps (e.g.) Figure 2 The robot can either navigate through the four rooms shown in the image or use pre-defined pathfinding logic (e.g., if the user instructs the robot to "clean the dirt within 3 meters ahead," then the "preset area" could be an area 3 meters long and the robot's own width). The robot can then navigate (move) within this preset area, collecting information about the environment using sensors such as vision sensors, infrared sensors, ultrasonic radar, and lidar. The specific types of information collected may vary depending on the type of sensor used. For example, for... Figure 2 In the scenario shown, the robot can be based on Figure 3 The illustrated route completes the aforementioned cruise process and obtains... Figure 2 The various types of dirt shown.

[0092] Taking a visual sensor (camera) as an example, the robot can continuously acquire images within the camera's field of view during movement and identify these images based on a preset algorithm or large model to determine the work units contained in the preset area. Furthermore, this process can also determine the task information of the work units, such as what specific dirt needs to be treated within the preset area and what type of dirt it is. In other steps of this disclosure, information related to the work units can be determined based on the above steps. These steps enable the robot to automatically adapt to the environment without human intervention, resulting in high flexibility and a superior user experience.

[0093] Next, the method may further include: taking the work area corresponding to each preset number of work units in the preset area as one of the multiple work areas.

[0094] In other words, in some cases, although the user's instruction is to clean the entire room, if there are only a few stains (the units to be cleaned) and other areas do not need cleaning (this is just an example), then the work areas corresponding to these stains can be designated as several work areas, and these work areas can be included as part or all of the "multiple work areas" in step S101. Similarly, if a room has no work units, then that room does not actually need cleaning.

[0095] In addition, the robot can have more decision-making logic in this process, such as dividing and merging these work units in pairs (that is, taking the work areas corresponding to every two work units as the same work area), or taking the work areas corresponding to work units with a distance less than a distance threshold as the same work area. This disclosure does not limit this.

[0096] The above steps further enable flexible division of the robot's work area without human intervention, which is highly flexible and can also improve the robot's work efficiency and reduce its energy consumption.

[0097] Step S102: Determine the location priority of each area to be worked based on its location information.

[0098] Location priority refers to the priority obtained from the perspective of region and position. For example, the location information may include at least one of the following: the distance between the area to be worked and the robot's starting point (this distance may be negatively correlated with the location priority of the area to be worked, that is, a more distant area to be worked tends to be ranked later in the overall work process), the area of ​​the area to be worked (in some cases, the area of ​​the area to be worked may be consistent with the area of ​​the unit to be worked mentioned above), and the geometric features of the area to be worked (for example, irregular shapes and shapes with acute interior angles are generally more difficult to clean, while rectangular areas are generally less difficult to clean).

[0099] Location priority can be provided by a deep learning model or calculated from data. In one exemplary implementation, the location information includes at least one parameter value represented numerically. Determining the location priority of each work area based on its location information may include: normalizing the parameter values ​​of the plurality of work areas for each parameter value of the location information; and for each work area, summing the normalized results of each parameter value of its location information as the location priority of that work area.

[0100] For example, there can be a location priority set Z = {ζ1,ζ2,...,ζ} m}, where m is the number of regions to be processed. The location priority ζ for region i to be processed... i ∈Z,(i∈{1,2,...,m}), ζ i The distance d from the robot's starting point can be determined by the normalized distance. i Normalized region area s i Adding them together, we get ζ. i =d i +s i The normalization methods can include min-max normalization, Z-score normalization, etc., and are not limited here; and depending on the different robot operation logic, ζ can also be adaptively adjusted. i Take the reciprocal, or...

[0101] d i s i Taking the reciprocal is not a limitation of this disclosure.

[0102] The above steps can accurately quantify the priority of the area to be processed in terms of location, thereby achieving a more reasonable work sequence planning and avoiding detours and meaningless routes as much as possible.

[0103] Step S103: Determine the work sequence of the multiple work areas based on the task priority and location priority of each work area.

[0104] For example, the task priority can be normalized using a location priority normalization method, and then the order of the sum of the normalized task priority and location priority can be used as the task order for multiple work areas. As another example, determining the task order for the multiple work areas based on the task priority and location priority of each work area can include: determining the task order for the multiple work areas based on the task priority and location priority of each work area, as well as the weights corresponding to the task priority and location priority.

[0105] For example, there is a set H = {η1, η2, ..., η} m}, then for each η i For each ∈H, (i∈{1,2,...,m}), we can let η i =λγ i +(1-λ)ζ i Wherein, η i Let γ be the overall priority of the area to be worked on, m be the number of areas to be worked on, and γ be the overall priority of the area to be worked on. iLet γ be the task priority for the area i to be worked on (where γ is the priority of the task). i (It can also be normalized), λ∈[0,1] is a preset weight constant (or it can be set by the user or given by the deep learning model). The ranking of comprehensive priorities can be used to determine the order of operations for multiple regions to be worked.

[0106] Based on this, it is possible to easily determine a reasonable job order by considering both task priority and location priority to varying degrees, thereby improving the execution performance of the method.

[0107] The task sequence can be used to characterize the sequential relationship between multiple areas to be worked on in the overall task process. After determining the task sequence, the robot can be controlled to process multiple areas to be worked on in that sequence.

[0108] In some embodiments of this disclosure, determining the task priority of a task-to-work area based on the number of task-to-work units in each of the multiple task-to-work areas and the task information of each task-to-work unit in the task-to-work area may include: for each task-to-work area in the multiple task-to-work areas: combining the task information of each task-to-work unit in the task-to-work area according to a preset rule to obtain text combination information; processing the text combination information based on a preset feature representation method to obtain initial features; performing dimensionality reduction processing on the initial features to obtain a task feature vector of a preset dimension; and inputting the task feature vector of the task-to-work area into a pre-trained model to determine the task priority of the task-to-work area.

[0109] In this process, the information such as the type and area of ​​dirt in each unit to be worked on can first be linked or merged to form a complete text description. For example, in home life, based on the task information of multiple rooms, a text combination information such as "Dirty type: solid dirt - area: 2 square meters - location characteristics: against the wall; dirty type: pet hair - area: 0.3 square meters - location characteristics: center of the room" can be generated.

[0110] Next, the combined text information can be processed based on a preset feature representation method to obtain initial features. Specific implementation methods can employ techniques such as Bag-of-Words, TF-IDF (Term Frequency-Inverse Document Frequency), and One-Hot Encoding to convert the text information into a numerical feature vector. For example, using the Bag-of-Words method, each word in the combined information is converted into an element in the feature vector, thus obtaining the initial feature representation.

[0111] Then, the initial features are subjected to dimensionality reduction to obtain task feature vectors of a preset dimension. Specific implementations can employ dimensionality reduction algorithms such as Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA) to reduce the dimensionality of the feature vectors while retaining key information. For example, PCA can be used to reduce the high-dimensional initial feature vectors to a preset low-dimensional representation, such as from 100 dimensions to 10 dimensions. Of course, "dimensionality reduction" here means "outputting feature vectors of a preset dimension," and it doesn't necessarily reduce the actual dimensionality of the feature vectors.

[0112] Finally, the dimensionality-reduced task feature vector can be input into the pre-trained model to determine the task priority of the region to be processed. The pre-trained model can be a neural network model, such as a multilayer perceptron (MLP).

[0113] Please see Figure 4 It exemplifies a pre-trained model where, for any region to be worked on (denoted as i, i∈{1,2,...,m}), its task priority γ i This can be provided by the model. The model can have an input layer with m neurons and an output layer with one neuron, where m is the vector dimension. Each neuron in the output layer represents the task priority of the region to be processed. The hidden layer structure and number of layers are dynamically adjusted during model training and optimization, for example, through incremental training and user feedback, to continuously optimize the model. Taking a hidden layer containing one neuron as an example, the model can optionally include two fully connected layers and two activation functions. The first fully connected layer is located between the input layer and the hidden layer, with an input size of m and an output size of n (where n is the number of neurons in the hidden layer). The second fully connected layer is located between the hidden layer and the output layer, with an input size of n and an output size of 1. One set of activation functions processes the output of the first fully connected layer, and the other set processes the output of the second fully connected layer.

[0114] The above implementation method can more accurately determine the task priority of the work area, and at the same time greatly improve the robot's adaptability to the environment, enabling the robot to process various complex work task information and correctly evaluate the task priority of the work area based on this information.

[0115] In summary, this disclosure provides a method for determining the work sequence of a work area by comprehensively considering the work tasks within the work area and the locational characteristics of the work area itself. Based on this, the robot can plan the work sequence according to the actual situation of the area to be cleaned, ensuring that the robot's work route planning fully considers the actual environment and improves the rationality of the robot's work sequence and path planning. Furthermore, since the work task information is taken into account, the above method also avoids the problem of urgently needed dirt not being treated, while the robot first processes areas without dirt, improving the user experience and minimizing the occurrence of dirt drying out, spoiling, smelling bad, and disrupting users' lives for a long time.

[0116] The following combination Figure 2 and Figure 3 The above-described at least one embodiment will be comprehensively described. First, for the four rooms (kitchen, bathroom, bedroom, and living room) (the areas to be worked on), the robot can first determine the patrol route based on a preset algorithm (e.g., Figure 3 (The route shown in the image) is then cruised along the route to determine the destination. Figure 2 The dirt shown is visible.

[0117] Next, the robot can determine the location priority of the four areas—kitchen, bathroom, bedroom, and living room—based on location information such as their area and distance from the base station. An exemplary principle is that the robot tends to prioritize areas that are closer and aims to ensure a shorter total travel distance throughout the operation (i.e., the robot tends to work sequentially in adjacent areas).

[0118] Next, the robot can determine the task priority of each room based on the level of dirtiness in each room, and finally determine the work sequence of multiple areas to be worked based on task priority and location priority.

[0119] For example, a bedroom might have a carpet with liquid stains (such as pet urine), while the bathroom also has liquid stains, but these are on tiles. Additionally, the bathroom contains a solid stain. Therefore, the robot might prioritize the bedroom, as the pet urine on the carpet could cause it to smell bad if not cleaned promptly. While the bathroom has more stains, none are urgent. The living room doesn't have liquid stains, but rather a solid stain and a stubborn stain. Solid stains are generally less urgent, while cleaning stubborn stains takes longer. Therefore, the living room's priority might be lower than the bathroom and bedroom (though its location priority is higher). The kitchen has only one solid stain, so its priority is likely the lowest. In summary, in one exemplary logic, if the location priorities of the four areas—kitchen, bathroom, bedroom, and living room—are all identical, the robot might be more inclined to execute tasks in the order of bedroom, bathroom, living room, and kitchen.

[0120] Corresponding to the embodiments of the foregoing methods, this disclosure also provides embodiments of the apparatus and the terminal to which it is applied.

[0121] A second aspect of this disclosure provides a work sequence determination device; please refer to [link to relevant documentation]. Figure 5 The device includes:

[0122] The task priority module 501 is used to determine the task priority of the waiting area based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area.

[0123] The location priority module 502 is used to determine the location priority of each area to be worked based on its location information.

[0124] The sequence determination module 503 is used to determine the operation sequence of the multiple areas to be worked based on the task priority and location priority of each area to be worked.

[0125] Optionally, the location information includes at least one parameter value represented numerically, and the location priority module is used to determine the location priority of each work area based on its location information, specifically for:

[0126] For each parameter value of the location information, the parameter values ​​of the multiple areas to be worked are normalized.

[0127] For each area to be worked on, the sum of the normalized results of each parameter value of the location information of the area to be worked on is used as the location priority of the area to be worked on.

[0128] Optionally, when the sequence determination module determines the work order of the multiple work areas based on the task priority and location priority of each work area, it is specifically used for:

[0129] The order of operations for the multiple areas to be worked is determined based on the task priority and location priority of each area to be worked, as well as the weights corresponding to the task priority and location priority.

[0130] Optionally, the task information of the unit to be worked on is used at least to determine the robot's actions when performing work on the unit to be worked on; and / or,

[0131] The location information includes at least one of the following:

[0132] The distance between the work area and the robot's starting point, the area of ​​the work area, and the geometric features of the work area.

[0133] Optionally, the task information of the unit to be worked on includes at least one of the following:

[0134] The type of dirt in the unit to be worked on, the area of ​​the unit to be worked on, the location characteristics of the unit to be worked on, the ground material covered by the unit to be worked on, and whether there are animals in the work area corresponding to the unit to be worked on.

[0135] Optionally, the type of dirt in the unit to be worked on includes at least one of the following:

[0136] Solid dirt, liquid dirt, stubborn dirt;

[0137] And / or, the positional characteristics of the unit to be worked on are used to characterize:

[0138] Whether the distance between the edge of the area corresponding to the unit to be worked and the obstacle is less than a preset threshold.

[0139] Optionally, the task priority module is used to determine the task priority of a task-to-work area based on the number of task-to-work units in each of the multiple task-to-work areas and the task information of each task-to-work unit in the task-to-work area. Specifically, it is used for:

[0140] For each of the multiple pending task areas:

[0141] The task information of each unit to be worked in the work area is combined according to a preset rule to obtain combined text information;

[0142] The combined text information is processed based on a preset feature representation method to obtain initial features;

[0143] The initial features are subjected to dimensionality reduction processing to obtain a task feature vector of a preset dimension;

[0144] The task feature vector of the region to be worked on is input into a pre-trained model to determine the task priority of the region to be worked on.

[0145] Optionally, the device further includes:

[0146] A cruise module is used to control the robot to cruise within a preset area and to collect visual information of the preset area based on a vision sensor mounted on the robot, wherein the preset area covers the plurality of areas to be worked on.

[0147] The task determination module is used to determine the work units to be done within the preset area and the work task information of each work unit based on the visual information.

[0148] Optionally, the device further includes:

[0149] The work area determination module is used to determine the work area corresponding to each preset number of work units within the preset area as one of the multiple work areas.

[0150] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0151] A third aspect of this disclosure provides a computer program product including a computer program / instructions that, when executed by a processor, implement the method described in the first aspect.

[0152] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. Furthermore, the device embodiments described above are merely illustrative; the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0153] Fourthly, embodiments of the work sequence determination device provided in this disclosure can be applied to electronic devices. Please refer to [link to relevant documentation]. Figure 6The illustration exemplifies a hardware schematic of an electronic device. For example, device 600 could be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0154] Device 600 may include one or more of the following components: processing component 601, memory 602, power supply component 603, multimedia component 604, audio component 605, input / output (I / O) interface 606, sensor component 607, and communication component 608.

[0155] Processing component 601 typically controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 601 may include one or more processors 609 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 601 may include one or more modules to facilitate interaction between processing component 601 and other components. For example, processing component 601 may include a multimedia module to facilitate interaction between multimedia component 604 and processing component 601.

[0156] Memory 602 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0157] Power supply component 603 provides power to various components of device 600. Power supply component 603 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 600.

[0158] Multimedia component 604 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, swipe, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 604 includes a front-facing camera and / or a rear-facing camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0159] Audio component 605 is configured to output and / or input audio signals. For example, audio component 605 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 602 or transmitted via communication component 608. In some embodiments, audio component 605 also includes a speaker for outputting audio signals.

[0160] Input / output (I / O) interface 606 provides an interface between processing component 601 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0161] Sensor assembly 607 includes one or more sensors for providing state assessments of various aspects of device 600. For example, sensor assembly 607 may detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 607 may also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 607 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 607 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0162] Communication component 608 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as Wi-Fi, 2G or 3G, 4G or 5G, or combinations thereof. In one exemplary embodiment, communication component 608 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 608 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IRDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0163] In an exemplary embodiment, device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the operation sequence determination method of the above-described electronic device.

[0164] Fifthly, in exemplary embodiments, this disclosure also provides a non-transitory computer-readable storage medium including instructions, such as a memory 602 including instructions, which can be executed by a processor 609 of device 600 to complete the operation sequence determination method of the electronic device. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0165] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0167] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0168] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining the order of operations, characterized in that, The method includes: The task priority of each waiting area is determined based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area. Based on the location information of each area to be worked on, the location priority of the area to be worked on is determined; The order of operations for the multiple areas to be worked on is determined based on the task priority and location priority of each area.

2. The method for determining the work sequence according to claim 1, characterized in that, The location information includes at least one parameter value represented numerically. Determining the location priority of each work area based on its location information includes: For each parameter value of the location information, the parameter values ​​of the multiple areas to be worked are normalized. For each area to be worked on, the sum of the normalized results of each parameter value of the location information of the area to be worked on is used as the location priority of the area to be worked on.

3. The method for determining the work sequence according to claim 2, characterized in that, The step of determining the work order of the multiple work areas based on the task priority and location priority of each work area includes: The order of operations for the multiple areas to be worked is determined based on the task priority and location priority of each area to be worked, as well as the weights corresponding to the task priority and location priority.

4. The method for determining the work sequence according to claim 1, characterized in that, The task information of the unit to be worked on is used at least to determine the robot's work actions when performing work on the unit to be worked on; and / or, The location information includes at least one of the following: The distance between the work area and the robot's starting point, the area of ​​the work area, and the geometric features of the work area.

5. The method for determining the work sequence according to claim 1, characterized in that, The task information of the unit to be worked on includes at least one of the following: The type of dirt in the unit to be worked on, the area of ​​the unit to be worked on, the location characteristics of the unit to be worked on, the ground material covered by the unit to be worked on, and whether there are animals in the work area corresponding to the unit to be worked on.

6. The method for determining the work sequence according to claim 5, characterized in that, The type of dirt in the unit to be worked on includes at least one of the following: solid dirt, liquid dirt, stubborn dirt; and / or, The positional characteristics of the unit to be worked on are used to characterize whether the distance between the edge of the area corresponding to the unit to be worked on and the obstacle is less than a preset threshold.

7. The method for determining the work sequence according to claim 1, characterized in that, The step of determining the task priority of a task-to-work area based on the number of task-to-work units in each of the multiple task-to-work areas and the task information of each task-to-work unit in the task-to-work area includes: For each of the multiple pending task areas: The task information of each unit to be worked in the work area is combined according to a preset rule to obtain combined text information; The combined text information is processed based on a preset feature representation method to obtain initial features; The initial features are subjected to dimensionality reduction processing to obtain a task feature vector of a preset dimension; The task feature vector of the region to be worked on is input into a pre-trained model to determine the task priority of the region to be worked on.

8. The method for determining the work sequence according to claim 1, characterized in that, The method further includes: The robot is controlled to cruise within a preset area, and visual information of the preset area is collected based on the vision sensors mounted on the robot, wherein the preset area covers the multiple areas to be worked on; Based on the visual information, the units to be worked in the preset area and the task information of each unit to be worked are determined.

9. The method for determining the work sequence according to claim 8, characterized in that, The method further includes: Each preset number of work areas corresponding to the work units to be done within the preset area is taken as one of the multiple work areas to be done.

10. A device for determining the sequence of operations, characterized in that, The device includes: The task priority module is used to determine the task priority of the waiting area based on the number of waiting units in each waiting area and the task information of each waiting unit in the waiting area. The location priority module is used to determine the location priority of each area to be worked on based on its location information. The sequence determination module is used to determine the operation sequence of the multiple areas to be worked based on the task priority and location priority of each area to be worked.

11. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method as described in any one of claims 1 to 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 9.

Images