Multi-robot scheduling method, robot and storage medium

Abstract

The invention discloses a multi-robot scheduling method, a robot and a storage medium, and relates to the technical field of robot scheduling, and the method comprises the steps: responding to a triggered user instruction, and determining a target region corresponding to the user instruction in a preset environment semantic map; identifying first environment information of the target area, and determining at least one to-be-called robot as an execution robot according to the first environment information; and calling at least one execution robot to perform task execution on the target area. According to the method, the robots are distributed according to the environment information of the target area, disordered competition or work repetition among the robots is avoided, high efficiency of task execution of the robots is achieved, and the task execution efficiency among the multiple robots is improved.

Description

Technical Field

[0001] This application relates to the field of robot scheduling technology, and in particular to multi-robot scheduling methods, robots, and storage media. Background Technology

[0002] With the rapid development of artificial intelligence technology, service robots have moved from the laboratory to real-world applications. In fields such as hotels, restaurants, and healthcare, the collaborative operation of multiple service robots can significantly improve service efficiency. In current multi-robot service systems, the commonly used operating mode is as follows: users issue natural language commands to a central server via voice, mobile terminals, or fixed panels; the central server then broadcasts or distributes these commands to a group of service robots that are idle or ready.

[0003] However, in practical applications, when multiple service robots receive the same instruction (such as cleaning a specific private room), they will independently determine themselves as the optimal subject to perform the task and respond and head to the same target location. This can easily lead to task overlap or resource competition, which in turn affects the task execution efficiency of each robot and reduces the efficiency of robot services.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a multi-robot scheduling method, a robot, and a storage medium, aiming to solve the technical problem of how to improve the task execution efficiency among multiple robots.

[0006] To achieve the above objectives, this application proposes a multi-robot scheduling method, which is applied to robots and includes the following steps: In response to a triggered user command, the target area corresponding to the user command in a preset environmental semantic map is determined; Identify the first environmental information of the target area, and determine at least one robot to be called as the execution robot based on the first environmental information; At least one of the aforementioned execution robots is invoked to perform tasks in the target area.

[0007] In one embodiment, the step of determining the target area corresponding to the user instruction in a preset environmental semantic map includes: When the user command is a voice command, the direction of the target sound source is obtained by locating the sound source of the voice command using a microphone array; The area encompassed by the direction of the target sound source in the environmental semantic map is defined as the target area.

[0008] In one embodiment, the first environmental information includes at least one of second environmental information and third environmental information, and the step of identifying the first environmental information of the target area includes: In the case where the target area includes a sub-region, the second environmental information of the sub-region is determined; In the case where the target area includes at least two sub-regions, a target sub-region is determined among the at least two sub-regions, and third environmental information of the target sub-region is determined.

[0009] In one embodiment, the step of determining the target sub-region within the at least two sub-regions includes: Identify the environmental characteristics of each of the sub-regions; The target sub-region is determined based on the degree of matching between the user instruction and the environmental characteristics of each sub-region.

[0010] In one embodiment, the step of determining the target sub-region based on the matching degree between the user instruction and the environmental characteristics of each sub-region includes: If there is only one sub-region with the highest matching degree, then the sub-region with the highest matching degree is determined as the target sub-region. In the case where there are at least two sub-regions with equal highest matching degree, the sub-region with the highest matching degree is determined as the area to be cleaned, and the priority of each area to be cleaned is determined according to the environmental characteristics of each area to be cleaned, and the target sub-region is determined according to the priority of each area to be cleaned.

[0011] In one embodiment, the step of determining at least one robot to be invoked as the execution robot based on the first environmental information includes: Based on the first environmental information and the robot matrix corresponding to each robot, at least one robot to be called is determined as the execution robot. The robot matrix includes at least one of the following: robot model structure, robot arm load, idle flag, remaining power, task priority, and / or current position.

[0012] In one embodiment, the step of determining at least one robot to be invoked as the executing robot based on the first environmental information and the robot matrix of each robot includes: If the first environmental information includes a preset low-profile object, the robot with a model structure of a sweeping robot is identified as the robot to be called. If the first environmental information includes a preset cleaning object, the robot with a robotic arm structure is identified as the robot to be called. Based on the arm load, idle status, remaining battery power, task priority, and current position of each robot to be called, an execution robot is selected from the robots to be called.

[0013] In one embodiment, before the step of determining the target region in a preset environmental semantic map corresponding to the user instruction in response to the triggered user instruction, the method further includes: Collect point cloud data of a preset work area and construct a point cloud map of the work area; The objects in the working area are segmented and identified to obtain area labeling information; By fusing the point cloud map and the region annotation information, an environmental semantic map of the working area is obtained.

[0014] Furthermore, to achieve the above objectives, this application also proposes a multi-robot scheduling device, which includes: The region determination module is used to determine the target region corresponding to the user command in a preset environmental semantic map in response to the triggered user command. The robot determination module is used to identify first environmental information of the target area and determine at least one robot to be called as the execution robot based on the first environmental information. The robot invocation module is used to invoke at least one of the aforementioned execution robots to perform tasks in the target area.

[0015] In addition, to achieve the above objectives, this application also proposes an electronic device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-robot scheduling method as described above.

[0016] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the multi-robot scheduling method described above.

[0017] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the multi-robot scheduling method described above.

[0018] The one or more technical solutions proposed in this application have at least the following technical effects: First, by responding to triggered user commands, the robot determines the target area corresponding to the user command in a preset environmental semantic map, quickly responding to user needs and avoiding the inefficiency caused by multiple robots acting blindly; then, it identifies the first environmental information of the target area and determines at least one robot to be called as the execution robot based on the first environmental information. By acquiring detailed environmental characteristics of the target area in real time, it provides rich and accurate data support for multi-robot scheduling, so as to achieve precise and dynamic adaptation of robot resources and task requirements; then, it calls at least one execution robot to perform tasks in the target area. Based on the precise planning and matching of the first two steps, it achieves efficient scheduling and collaborative work of one or more robots. This application avoids disorderly competition or duplication of work between robots by allocating robots according to the environmental information of the target area; at the same time, through the collaborative cooperation between multiple robots, it achieves high efficiency in robot task execution and improves the task execution efficiency among multiple robots. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating an embodiment of the multi-robot scheduling method of this application. Figure 2 A flowchart illustrating the determination of environmental information provided in Embodiment 1 of this application; Figure 3 This is an example diagram of a robot work area provided in Embodiment 1 of this application; Figure 4 This is a schematic diagram of the module structure of the multi-robot scheduling device according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the multi-robot scheduling method in this application embodiment.

[0022] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0024] It should be noted that in the description of this application and the appended claims, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0026] In current multi-robot service systems, the common operating mode is as follows: users issue natural language commands to a central server via voice, mobile terminals, or fixed panels; the central server then broadcasts or distributes these commands to a group of service robots that are idle or ready. However, in practical applications, when multiple service robots receive the same command (such as cleaning a specific private room), each robot independently determines itself as the optimal agent to perform the task and responds and heads to the same destination. This can easily lead to task overlap or resource competition, thus affecting the task execution efficiency of each robot and reducing the overall service efficiency.

[0027] This application provides a solution. First, by responding to triggered user commands, the robot determines the target area corresponding to the user command in a pre-defined environmental semantic map, quickly responding to user needs and avoiding the inefficiency caused by multiple robots acting blindly. Next, it identifies the first environmental information of the target area and determines at least one robot to be called as the execution robot based on this information. By acquiring detailed environmental characteristics of the target area in real time, it provides rich and accurate data support for multi-robot scheduling, achieving precise and dynamic adaptation of robot resources to task requirements. Then, it calls at least one execution robot to perform tasks in the target area. Based on the precise planning and matching of the first two steps, it achieves efficient scheduling and collaborative work of one or more robots. This application avoids disorderly competition or duplication of work among robots by allocating robots according to the environmental information of the target area; at the same time, through collaborative cooperation among multiple robots, it achieves high efficiency in robot task execution and improves the overall task execution efficiency among multiple robots.

[0028] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a robot capable of performing the above functions. The following description uses a robot as the executing entity to illustrate this embodiment and the subsequent embodiments.

[0029] Based on this, embodiments of this application provide a multi-robot scheduling method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the multi-robot scheduling method of this application.

[0030] In this embodiment, the multi-robot scheduling method includes steps S10 to S30: Step S10: In response to the triggered user command, determine the target area corresponding to the user command in the preset environmental semantic map; User instructions refer to the operational intentions conveyed by users to robots through specific input methods (such as voice, buttons, gestures, etc.), and usually include key fields such as instruction type (such as cleaning, delivery, etc.) and target area.

[0031] An environmental semantic map is a pre-constructed digital map that includes physical location information and semantic annotations for each location. The target area, on the other hand, refers to a specific spatial range defined on the environmental semantic map according to user instructions; that is, the specific physical location where the robot needs to go and perform its task.

[0032] For example, a user can issue a command through the terminal panel configured on the robot to clean room 101; after receiving the user's command, the robot accesses the environmental semantic map, searches for semantic information that matches the user's command, and determines the physical location information of the spatial range (i.e., the target area) corresponding to the matching semantic information; then, the robot can go to the target area based on the physical location information to further determine which execution robot needs to be called.

[0033] Step S20: Identify the first environmental information of the target area, and determine at least one robot to be called as the execution robot based on the first environmental information; Environmental information refers to various environmental parameters and characteristic data related to the target area that are collected in real time. This information can be obtained in real time through cameras configured within the target area, or it can be obtained through cameras configured on the robot after the robot, which receives user commands, moves to the target area. This implementation does not impose specific limitations on this. For distinction, the environmental information of the target area is referred to as the first environmental information.

[0034] An execution robot refers to one or more robot entities used to execute tasks corresponding to user instructions within a target area. Each robot entity has a unique SN (Serial Number) code to prevent duplicate allocation; the robot that receives user instructions can also be an execution robot.

[0035] For example, after identifying a target area, the robot can initiate query requests to the physical network sensors and cameras deployed in the target area to obtain real-time first environmental information of the target area; alternatively, it can autonomously travel to the target area and collect the first environmental information of the target area in real time through its configured cameras. Furthermore, based on the task objects included in the first environmental information of the target area, the type of robot required can be determined, such as a robot vacuum cleaner needed to clean under a sofa or a robot with a robotic arm needed to clean a wine bottle. The estimated load of the task object can be further determined, and combined with the idle status, remaining load, path distance, and remaining battery power of the corresponding type of robot, the executing robot can be selected, avoiding congestion and resource competition caused by blind deployment.

[0036] Understandably, by first determining the target area and then querying the real-time environmental information of that area, robot scheduling decisions suitable for the current environment can be made. This directly avoids resource misallocation and competition caused by unclear cleaning targets among multiple robots after the instructions are issued, so as to achieve collaborative cooperation among multiple robots and improve the task execution efficiency among multiple robots.

[0037] In one feasible implementation, the first environmental information includes at least one of second environmental information and third environmental information, and the step of identifying the first environmental information of the target area in step S20 includes: Step S21: If the target area includes a sub-region, determine the second environmental information of the sub-region; A subregion refers to a relatively independent small area within the target area, divided according to specific rules (such as spatial location, functional characteristics, etc.).

[0038] The second environmental information represents real-time environmental information specific to a certain sub-region, which can also be called the first environmental information of the target region.

[0039] For example, a user might give the robot the command "clean this room" at the door of a private room. The robot can determine the location of the sound source and define the area contained in that location as the target area. This target area may contain only one room (sub-area) or it may contain multiple rooms. If the target area contains only one room, that room can be directly defined as the target area, and the second environmental information of that room can be defined as the first environmental information of the target area.

[0040] Step S22: In the case where the target area includes at least two sub-regions, determine the target sub-region within the at least two sub-regions, and determine the third environmental information of the target sub-region.

[0041] The target sub-region refers to the highest priority sub-region object selected from the multiple sub-regions included in the target region. It is the task execution area of ​​the robot in the current decision cycle.

[0042] The third environmental information representation is specifically designed for real-time environmental information collected from the target sub-region.

[0043] For example, in a target area containing multiple compartments (sub-areas) and where the user instruction includes a cleaning task, the cleanliness level of each sub-area can be determined based on the environmental information within each sub-area, and sub-areas with a cleanliness level below a preset threshold can be identified as candidate areas. Then, a target sub-area is determined based on the cleanliness level of each candidate sub-area, and this target sub-area is designated as the target area. The collected third environmental information of the target sub-area is identified as the first environmental information, and the robot is invoked to proceed to the target sub-area to perform the task. After the task in the target sub-area is completed, candidate sub-areas can be re-identified, and the steps of determining the target sub-area based on the cleanliness level of each sub-area and subsequent steps can be repeated until all candidate sub-areas within the target area have completed their cleaning tasks.

[0044] Understandably, more accurate regional positioning can avoid the waste of resources caused by sending robots to non-task areas. At the same time, it allows robots to focus their resources on the target sub-area instead of planning tasks for all sub-areas immediately, thereby improving the robot's response speed and avoiding path conflicts and waiting caused by multiple robots being sent to different sub-areas, thus improving the task execution efficiency among multiple robots.

[0045] In one feasible implementation, the step of determining the target sub-region in at least two sub-regions in step S22 includes: Step S221: Identify the environmental characteristics of each sub-region; Environmental characteristics refer to the quantitative or symbolic attributes that reflect the objects contained in a sub-region. They typically include information such as the size and quantity of furniture objects (e.g., sofas) and cleaning objects (e.g., wine bottles), and can be perceived and determined through sensors, cameras, etc.

[0046] Step S222: Determine the target sub-region based on the matching degree between the user instruction and the environmental characteristics of each sub-region.

[0047] Matching degree refers to a numerical score calculated by a predefined algorithm (such as cosine similarity, weighted scoring model or neural network, etc.), which is used to quantify the degree of fit between user instructions and the environmental features of a sub-region.

[0048] Optionally, the task object to be processed can be extracted from the user instruction, and the entity object can be determined from the environmental features of each sub-region; then, the matching degree between the task object to be processed and the entity object of each sub-region can be determined through preset matching rules; then, the sub-region with the highest matching degree can be determined as the target sub-region.

[0049] For example, when the user instruction is "The sofa here is dirty, there are too many empty wine bottles, come and clean it up," the task objects to be processed can be extracted: empty wine bottles and sofa. At the same time, the environmental features of each sub-region are identified by the camera to determine the entity objects contained therein, such as sofa, empty wine bottles, cups, coffee table, etc. Then, based on the number of task objects to be processed in each sub-region contained in the entity objects of each sub-region, the matching degree of each sub-region is determined. For example, the matching degree of a sub-region containing only empty wine bottles or sofas in the entity objects is 50%, while the matching degree of a sub-region containing both empty wine bottles and sofas in the entity objects is 100%. Then, at least one sub-region with the highest matching degree is determined as the target sub-region.

[0050] In one feasible implementation, step S222 includes: Step S2221: If there is only one sub-region with the highest matching degree, determine the sub-region with the highest matching degree as the target sub-region. The sub-region with the highest matching degree refers to one or more sub-regions with the largest matching degree value after calculating the matching degree of multiple sub-regions according to specific rules (such as the degree of fit with the task object to be processed in the user instruction).

[0051] For example, if there is only one sub-region with the highest matching degree, the sub-region can be directly identified as the target sub-region and the target sub-region can be identified as the target region to be processed. The third environmental information of the target sub-region can be identified as the first environmental information of the target region. Then, based on the first environmental information and the status information of each robot, at least one robot can be invoked to execute the task indicated by the user instruction.

[0052] Step S2222: In the case that there are at least two sub-regions with equal highest matching degree, the sub-region with the highest matching degree is determined as the area to be cleaned, and the priority of each area to be cleaned is determined according to the environmental characteristics of each area to be cleaned, and the target sub-region is determined according to the priority of each area to be cleaned.

[0053] For example, after matching degree calculation and sorting, if two or more sub-regions have the highest matching degree, these sub-regions with the highest matching degree are identified as areas to be cleaned. Then, the priority of each area to be cleaned is determined by comparing the environmental characteristics of each area to be cleaned with preset original images of each area to be cleaned. The original images are used to reflect the state of the area to be cleaned when it is clean and free of debris or obstacles. Then, the area to be cleaned with the highest priority is identified as the target sub-region, and this target sub-region is cleaned first. After cleaning is completed, the area to be cleaned with the second highest priority is identified as the target sub-region, and so on, until all areas to be cleaned are cleaned.

[0054] For example, please refer to Figure 2 , Figure 2 A flowchart for determining environmental information is provided; after receiving a voice command triggered by the user, the robot determines the target area (B101) based on the command, for example... Figure 3 The circles in the diagram represent the positions of a robot, and A and B are both boxes (sub-regions). Figure 3 A customer at the entrance of booth B issued a voice command, "There are too many empty bottles here, come and clean them up." Upon receiving this voice, the robot performs sound source localization to determine the direction of the target sound source and identifies booths A and B, encompassed by that direction, as the target area in the environmental semantic map. Then, after determining the target area (B101), it executes B102 to determine if there are multiple sub-areas within the target area. If there is only one sub-area, it executes B103 to directly determine the second environmental information of that sub-area; if the target area contains multiple sub-areas, it executes B103 to directly determine the second environmental information of that sub-area. Figure 3 If two or more sub-regions are shown, the environmental characteristics of each sub-region are further identified, and the matching degree between the user command and the environmental characteristics of each sub-region is determined (B104); then it is determined whether there are multiple sub-regions with the same highest matching degree (B105). If only one sub-region with the same highest matching degree exists, then that sub-region is directly identified as the target sub-region (B106), for example... Figure 3 If only private room B has empty bottles within the target area, and thus has the highest matching degree, then private room B is designated as the target sub-area. If multiple sub-areas have the same highest matching degree, then all of these sub-areas are designated as areas to be cleaned, and their respective priorities (B108) are further determined based on the environmental characteristics of each area to be cleaned. Figure 3Empty wine bottles may exist in both private rooms A and B in the target area. Since the matching degree between the two is equal and the highest, both private rooms A and B are identified as areas to be cleaned. Furthermore, the cleanliness of the area is judged based on the environmental characteristics of private rooms A and B to determine their cleaning priority. After determining the priority, the target sub-area can be determined based on the priority (B108), and the third environmental information of the target sub-area can be determined (B109).

[0055] In this embodiment, the target area to be cleaned is determined based on the matching degree between the user command and the environmental characteristics of each sub-area, ensuring that the task execution matches the user command and avoiding unnecessary waste of resources; furthermore, the order of task execution is determined based on the environmental characteristics of the area to be cleaned, thereby optimizing the execution sequence in a multi-task scenario and improving the overall task execution efficiency.

[0056] Step S30: Invoke at least one execution robot to perform the task in the target area.

[0057] For example, based on the determination of the target area and the execution robot, at least one of the determined execution robots is invoked to go to the target area and perform the task instructed by the user.

[0058] This embodiment provides a multi-robot scheduling method. In the process of determining the target area, the method determines the target area that conforms to the user's instructions by matching with the user's instructions and by prioritizing based on environmental characteristics. Furthermore, by prioritizing the determination of the target area and sensing the real-time status of the target area (first environmental information), the method determines the robot to execute the task. This fundamentally avoids the task overlap, path conflict, and resource competition caused by multiple robots blindly responding to the same instructions, thereby improving the task execution efficiency and throughput of single and multiple robots and enhancing the user's service experience.

[0059] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter. Based on this, step S20, which involves determining at least one robot to be invoked as the execution robot based on the first environmental information, includes: Step A21: Based on the first environmental information and the robot matrix corresponding to each robot, determine at least one robot to be called as the execution robot. The robot matrix includes at least one of the following: robot model structure, robot arm load, idle flag, remaining power, task priority, and / or current position.

[0060] A robot matrix is ​​a multi-dimensional data structure used to describe the state and performance characteristics of each robot. The robot matrix for each robot contains its corresponding serial number (SN) code, as well as information on multiple dimensions such as robot structure, robot arm load, idle status, remaining battery power, task priority, and current position.

[0061] Robot structure refers to the physical configuration and functional modules of a robot, such as a robotic vacuum cleaner or a robotic arm. It is used to determine whether a corresponding task can be performed; for example, only a robotic arm can pick up empty bottles. The robotic arm load refers to the remaining capacity of the robotic arm; the robotic arm load of a robotic vacuum cleaner is null in the robot matrix.

[0062] The idle flag is a Boolean value used to indicate whether the robot is available; its value is either yes or no.

[0063] Task priority refers to the processing priority of new tasks when the robot is in a non-idle state. For example, when the robot is in a non-idle state and is performing a cleaning task, the priority of new tasks is lower than that of the current cleaning task; while when the robot is in a non-idle state and is on its return journey, the priority of new tasks is higher than that of the current return journey task.

[0064] Optionally, firstly, based on the entity objects contained in the first environmental information, the required robot model structure is determined; then, each robot matrix is ​​traversed, and multiple candidate robots are obtained based on the idle flag and the determined model structure; then, based on the first environmental information, the garbage load of the target area is determined, and multiple calling schemes are generated based on the garbage load and the robotic arm load of each candidate robot, wherein each calling scheme includes at least one robot; then, based on the remaining battery power, current position, and other attributes of the robots included in each calling scheme, the comprehensive score of each calling scheme is determined, wherein the more remaining battery power, the higher the comprehensive score, and the closer the current position is to the target area, the higher the comprehensive score; finally, the robot in the calling scheme with the highest comprehensive score is determined as the executing robot.

[0065] Optionally, the robot matrix may also include robot size; after the above steps of generating multiple calling schemes, the footprint of each calling scheme may be determined based on the robot size included in each calling scheme, and the accommodating area of ​​the target area may be compared with the footprint of each calling scheme, and calling schemes with a footprint greater than or equal to the accommodating area may be eliminated; then subsequent steps such as comprehensive score calculation may be performed.

[0066] In one feasible implementation, step A21 includes: Step A211: If the first environmental information includes a preset low-profile object, the robot with a model structure of a sweeping robot is identified as the robot to be called. Preset low-profile objects refer to objects that are pre-defined in the robot's database and are of relatively low height, suitable for the robot vacuum to clean or handle, such as sofas and carpets.

[0067] Step A222: If the first environmental information includes a preset cleaning object, the robot with a robotic arm structure is identified as the robot to be called. The preset cleaning objects refer to the target objects that the robotic arm robot needs to clean or operate, as determined in the robot's data in advance, such as empty wine bottles, cups, windows, etc.

[0068] For example, the robot can collect first environmental information of the target area through a camera, analyze and disassemble it to determine the objects to be processed, such as sofas, wine bottles, etc.; then, if there are objects to be processed that belong to preset low objects, the sweeping robot is identified as the robot to be called; and if there are objects to be processed that belong to preset cleaning objects, the robotic arm robot is also identified as the robot to be called.

[0069] Step A223: Select the execution robot from the robots to be called based on the robot arm load, idle flag, remaining power, task priority and current position of each robot to be called.

[0070] For example, when the first environmental information includes both a preset low-profile object and a preset cleaning object, both the sweeping robot and the robotic arm robot can be identified as robots to be invoked.

[0071] For example, after determining the robot to be called, candidate robots can be determined according to the idle flag corresponding to each called robot, and at least one execution robot can be selected according to the information such as the robot arm load, remaining power, task priority and current position corresponding to each candidate robot. The specific selection method is similar to the specific implementation of step A21 in this embodiment, so it will not be described again.

[0072] Understandably, accurately determining the type of robot to be called upon through the initial environmental information allows the appropriate robot to quickly respond to task requirements, avoiding the waste of resources caused by mismatched robots performing tasks.

[0073] In this embodiment, by combining the first environmental information of the target area and the robot matrix of each robot, the appropriate robot is called to execute the corresponding task, so as to achieve efficient collaborative operation among multiple robots and improve the overall utilization rate and task execution efficiency of the robot cluster.

[0074] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and second embodiments described above can be referred to the above description and will not be repeated hereafter. Furthermore, before step S10, the following is also included: Step S01: Collect point cloud data of the preset work area and construct a point cloud map of the work area; The preset work area refers to the physical space boundary that needs to be scanned and modeled before the task is executed.

[0075] Point cloud data refers to a collection of point data in three-dimensional space. Each point contains at least its coordinate information in three-dimensional space (such as x, y, and z coordinates), and usually also contains other possible attribute information (such as color, reflectance intensity, etc.). Point cloud data can be obtained by scanning and sampling the surface of an object using sensors such as LiDAR and depth cameras, and can accurately describe the shape and spatial position of the object.

[0076] Point cloud maps are digital maps built on point cloud data to represent the spatial structure and object distribution of a work area. They can provide important environmental information for tasks such as robot navigation and target recognition.

[0077] For example, a robot equipped with a lidar can move and scan within a preset working area to collect point cloud data within that area. Then, the collected point cloud data can be processed, stitched together, and optimized to form a continuous and accurate three-dimensional spatial model, thus obtaining a point cloud map.

[0078] Step S02: Segment and identify each object in the working area to obtain area labeling information; An object is an independent entity within the work area that has specific physical characteristics (such as shape, color, texture, etc.) and functional attributes.

[0079] Region labeling information refers to structured data that includes information such as object location, category, and attributes, generated after segmenting and identifying objects within a working area.

[0080] For example, during the acquisition of point cloud data, graphic information in the working area can be acquired simultaneously through an RGBD camera, and then the graphic information can be processed by a preset segmentation algorithm to segment the various objects in the working area; then the features of the segmented objects can be extracted, and the feature information of each object can be classified and identified using a trained separation model to determine the category to which each object belongs, and the coordinate information of each object can be labeled, thereby obtaining the region labeling information of the working area.

[0081] Step S03: Merge the point cloud map and the region annotation information to obtain the environmental semantic map of the working area.

[0082] For example, the geographic coordinates of each object in the regional coordinate information can be aligned and matched with the coordinates in the point cloud map so that the two have a consistent reference system in space; then, the regional labeling information can be associated with the corresponding points or regions in the point cloud map to obtain an environmental semantic map containing semantic information.

[0083] Understandably, by combining map information and semantic information to construct an environmental semantic map, a shared and unified environmental cognition framework is provided for multiple robots, ensuring the consistency of different robots' understanding of the work area and physical objects, avoiding task overlap or conflict, and facilitating the improvement of the overall task execution efficiency among multiple robots.

[0084] In one feasible implementation, step S10, which involves determining the target region corresponding to the user command in a preset environmental semantic map, includes: Step S11: When the user command is a voice command, the direction of the target sound source is obtained by locating the sound source of the voice command through a microphone array. A microphone array is an acoustic sensor system consisting of a group of multiple microphones arranged in a specific geometric structure (such as linear, circular, or spherical) in space. It can obtain spatial acoustic information by processing the phase difference and intensity difference of the sound signals received by different microphones, thereby locating the sound source.

[0085] Sound source localization (SSL) refers to the process of accurately estimating the direction, distance, and even three-dimensional coordinates of a sound source by utilizing the physical characteristics of sound signals (such as propagation time, intensity, and frequency variations) and the geometric layout of a microphone array. Common sound source localization methods include generalized cross-correlation, beamforming, and time difference of arrival (TDOA). This embodiment does not impose specific restrictions on the specific sound source localization method used.

[0086] The direction of a target sound source refers to the specific direction of the sound source determined by sound source localization technology, which can be expressed in terms of angle or orientation.

[0087] Step S12: Determine the area contained in the direction of the target sound source in the environmental semantic map as the target area.

[0088] For example, the audio signals collected by each microphone in the microphone array can be analyzed, the time difference between each signal can be calculated, and the direction of the sound source relative to the microphone array can be determined by the TDOA positioning method to obtain the direction of the target sound source. Then, a coordinate system can be established based on the robot's current orientation, and the direction of the target sound source based on the microphone array can be transformed and calculated in the robot coordinate system to obtain the angular offset of the target sound source relative to the robot's current orientation. Then, based on the calculated angular offset, the area included by the angular offset centered on the robot's current position can be determined as the target area in the environmental semantic map.

[0089] In this embodiment, by constructing an environmental semantic map and determining the directional information represented by inaccurate indicator pronouns (such as "here" and "there") in the user's voice commands through sound source localization, the accuracy and reliability of target area localization are ensured, and the accuracy of understanding and responding to user commands is improved.

[0090] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the multi-robot scheduling method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0091] This application also provides a multi-robot scheduling device; please refer to... Figure 4 The multi-robot scheduling device includes: The region determination module 10 is used to determine the target region corresponding to the user command in the preset environmental semantic map in response to the triggered user command; The robot determination module 20 is used to identify the first environmental information of the target area and determine at least one robot to be called as the execution robot based on the first environmental information. The robot invocation module 30 is used to invoke at least one execution robot to perform tasks in the target area.

[0092] The multi-robot scheduling device provided in this application, employing the multi-robot scheduling method described in the above embodiments, can solve the technical problem of how to improve the task execution efficiency among multiple robots. Compared with the prior art, the beneficial effects of the multi-robot scheduling device provided in this application are the same as those of the multi-robot scheduling method described in the above embodiments, and other technical features in the multi-robot scheduling device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0093] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the multi-robot scheduling method in the first embodiment described above.

[0094] The following is for reference. Figure 5 The diagram illustrates a structural schematic of an electronic device suitable for implementing embodiments of this application. The electronic devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0095] like Figure 5 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0096] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0097] The electronic device provided in this application, employing the multi-robot scheduling method described in the above embodiments, can solve the technical problem of how to improve the task execution efficiency among multiple robots. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the multi-robot scheduling method described in the above embodiments, and other technical features of the electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0098] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0099] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0100] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the multi-robot scheduling method described above.

[0101] The computer-readable storage medium provided in this application embodiment may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0102] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0103] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by an electronic device, the electronic device: responds to a triggered user instruction, determines the target area corresponding to the user instruction in a preset environmental semantic map; identifies first environmental information of the target area, and determines at least one robot to be invoked as the execution robot based on the first environmental information; and invokes at least one execution robot to perform a task in the target area.

[0104] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0106] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0107] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-described multi-robot scheduling method, and can solve the technical problem of how to improve the task execution efficiency among multiple robots. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the multi-robot scheduling method provided in the above embodiments, and will not be repeated here.

[0108] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the multi-robot scheduling method described above.

[0109] The computer program product provided in this application can solve the technical problem of how to improve the task execution efficiency among multiple robots. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the multi-robot scheduling method provided in the above embodiments, and will not be repeated here.

[0110] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A multi-robot scheduling method, characterized in that, The multi-robot scheduling method is applied to robots, and the method includes: In response to a triggered user command, the target area corresponding to the user command in a preset environmental semantic map is determined; Identify the first environmental information of the target area, and determine at least one robot to be invoked as the execution robot based on the first environmental information; At least one of the aforementioned execution robots is invoked to perform tasks in the target area.

2. The multi-robot scheduling method as described in claim 1, characterized in that, The step of determining the target area corresponding to the user command in the preset environmental semantic map includes: When the user command is a voice command, the direction of the target sound source is obtained by locating the sound source of the voice command using a microphone array; The area encompassed by the direction of the target sound source in the environmental semantic map is defined as the target area.

3. The multi-robot scheduling method as described in claim 1, characterized in that, The first environmental information includes at least one of second environmental information and third environmental information, and the step of identifying the first environmental information of the target area includes: In the case where the target area includes a sub-region, the second environmental information of the sub-region is determined; In the case where the target area includes at least two sub-regions, a target sub-region is determined among the at least two sub-regions, and third environmental information of the target sub-region is determined.

4. The multi-robot scheduling method as described in claim 3, characterized in that, The step of determining the target sub-region within the at least two sub-regions includes: Identify the environmental characteristics of each of the sub-regions; The target sub-region is determined based on the degree of matching between the user instruction and the environmental characteristics of each sub-region.

5. The multi-robot scheduling method as described in claim 4, characterized in that, The step of determining the target sub-region based on the matching degree between the user instruction and the environmental characteristics of each sub-region includes: If there is only one sub-region with the highest matching degree, then the sub-region with the highest matching degree is determined as the target sub-region. In the case where there are at least two sub-regions with equal highest matching degree, the sub-region with the highest matching degree is determined as the area to be cleaned, and the priority of each area to be cleaned is determined according to the environmental characteristics of each area to be cleaned, and the target sub-region is determined according to the priority of each area to be cleaned.

6. The multi-robot scheduling method as described in claim 1, characterized in that, The step of determining at least one robot to be invoked as the execution robot based on the first environmental information includes: Based on the first environmental information and the robot matrix corresponding to each robot, at least one robot to be called is determined as the execution robot. The robot matrix includes at least one of the following: robot model structure, robot arm load, idle flag, remaining power, task priority, and / or current position.

7. The multi-robot scheduling method as described in claim 6, characterized in that, The step of determining at least one robot to be called as the execution robot based on the first environmental information and the robot matrix of each robot includes: If the first environmental information includes a preset low-profile object, the robot with a model structure of a sweeping robot is identified as the robot to be called. If the first environmental information includes a preset cleaning object, the robot with a robotic arm structure is identified as the robot to be called. Based on the arm load, idle status, remaining battery power, task priority, and current position of each robot to be called, an execution robot is selected from the robots to be called.

8. The multi-robot scheduling method as described in claim 1, characterized in that, Before the step of determining the target area in the preset environmental semantic map corresponding to the user command in response to the triggered user command, the method further includes: Collect point cloud data of a preset work area and construct a point cloud map of the work area; The objects in the working area are segmented and identified to obtain area labeling information; By fusing the point cloud map and the region annotation information, an environmental semantic map of the working area is obtained.

9. A robot, characterized in that, The robot includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the multi-robot scheduling method as described in any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the multi-robot scheduling method as described in any one of claims 1 to 8.

Images