Task robot control method and device, electronic equipment and system

Abstract

The embodiment of the invention provides a task robot control method and device, electronic equipment and a system. Relates to the technical field of artificial intelligence. At least one task robot exists in the working area of the task robots, and each task robot is provided with a three-dimensional visual sensor used for scanning a local area on the running path of the task robot; the method comprises the steps that in the task execution process of any task robot, under the condition that a three-dimensional vision sensor of the task robot detects that an abnormal object exists on a driving path of the task robot, first global position information of the abnormal object is determined; and based on the first global position information and the second global position information and the driving path of each task robot, each task robot is controlled to run without collision in the working area. When the operation process of the robot is controlled, the operation safety of the robot is guaranteed, and the overall working efficiency of the task robot is effectively improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a control method, device, electronic equipment, and system for a task robot. Background Art

[0002] When robots are performing sorting tasks on the sorting platform, cargo often falls due to its large size, center of gravity offset, or scratches. These dropped cargo not only affect the robot's task completion but can also hinder the normal operation of other robots. Therefore, it is necessary to notify staff promptly to clean up the cargo to ensure stable operation of the system.

[0003] Currently, the identification of objects on the sorting platform mostly uses a single-line radar scanning solution, which uses radar scanning to detect whether there are fallen goods or other abnormal objects on the platform.

[0004] However, this solution has limitations in terms of recognition coverage and system operation efficiency, resulting in poor task execution of the robot. Summary of the Invention

[0005] The embodiments of the present application provide a control method, device, electronic equipment and system for a task robot, which can ensure the safety of the robot's operation while effectively improving the overall work efficiency of the task robot.

[0006] In a first aspect, an embodiment of the present application provides a control method for a task robot, wherein at least one task robot is present in a working area of ​​the task robot, and each task robot is equipped with a three-dimensional visual sensor for scanning a local area on its driving path, where the local area covers a driving plane in the forward direction of the task robot;

[0007] The method comprises:

[0008] During the execution of a task by any task robot, if the three-dimensional vision sensor of the task robot detects the presence of an abnormal object on its driving path, determining the first global position information of the abnormal object; wherein the global position information is a parameter used to characterize the spatial position in the global coordinate system of the working area; the first global position information is a spatial coordinate parameter obtained after coordinate conversion based on the three-dimensional visual data acquired by the three-dimensional vision sensor;

[0009] Based on the first global position information, and the second global position information and driving path of each task robot, each task robot is controlled to travel in the working area without collision.

[0010] By equipping each task robot with a 3D vision sensor that can scan a local area of ​​the forward travel plane, the above solution can accurately detect abnormal objects on the travel path and determine their first global position information. This is then combined with the second global position information of each task robot to achieve collision-free travel control. In this solution, the 3D vision sensor can effectively identify abnormal objects of different sizes and heights, improving detection comprehensiveness. Furthermore, the global position information can be used to control the task robots in a targeted manner, avoiding unnecessary downtime of unrelated robots. This ensures the operational safety of the task robots while improving the efficiency of task execution within the overall work area.

[0011] In one possible implementation, the three-dimensional vision sensor of the task robot detects an abnormal object on its driving path, including:

[0012] When the three-dimensional vision sensor detects that there is a task object on its driving path, determining the first local position information of the task object; wherein the first local position information is the spatial coordinate parameter of the task object in the first local coordinate system to which the task robot belongs;

[0013] When the first local position information meets a preset height threshold condition, determining that the task object is an abnormal object;

[0014] Accordingly, determining the first global location information of the abnormal object includes:

[0015] The first global position information of the abnormal object is determined according to the first local position information and the second global position information of the task robot.

[0016] During the driving process of the task robot, the above-mentioned scheme can accurately distinguish between task objects in normal operating conditions and task objects in abnormal conditions such as falling by comparing the first local position information of the task object with a preset height threshold set based on the distance between it and the ground or the horizontal plane of the workbench, providing a reliable basis for the accurate identification of abnormal objects, thereby improving the smoothness and safety of the task robot's path planning and safe driving.

[0017] In a possible implementation, the three-dimensional vision sensor detects the presence of a task object on its driving path, including:

[0018] In the case where the three-dimensional vision sensor recognizes a physical object within its scanning field of view, calling a preset object recognition model;

[0019] Two-dimensional visual data of the physical object is acquired, and the two-dimensional visual data is input into the object recognition model to determine whether the physical object belongs to a preset task object category.

[0020] The above implementation method, by real-time collection and processing of three-dimensional visual sensor data, and combining it with the classification and processing of physical objects by deep learning models, can identify whether there are task objects on the driving path, providing a reliable basis for subsequent anomaly detection and path planning, and improving the task execution effect of the task robot.

[0021] In a possible implementation, determining the first local position information of the task object includes:

[0022] Determine the second local position information of the task object; wherein the second local position information is the spatial coordinate parameter of the task object in the second local coordinate system of the three-dimensional vision sensor;

[0023] Acquire a coordinate transformation relationship between the first local coordinate system and the second local coordinate system;

[0024] The second local position information is transformed according to the coordinate transformation relationship to obtain the first local position information of the task object.

[0025] During the above-mentioned position conversion process, the task robot accurately obtains the spatial position information of the task object relative to itself based on its own first local coordinate system, which can provide basic data support for the subsequent abnormal object determination of the task object and the execution of related control logic.

[0026] In one possible implementation, the three-dimensional vision sensor is a depth camera;

[0027] The three-dimensional visual data includes pixel position coordinates and depth position coordinates;

[0028] Determining the second local position information of the task object includes:

[0029] Obtaining the pixel position coordinates of the task object and the depth position coordinates corresponding to the pixel position coordinates;

[0030] Determine second local position information of the task object according to the pixel position coordinates, the depth position coordinates, and the camera intrinsic parameters of the depth camera.

[0031] By using a depth camera as a three-dimensional vision sensor and utilizing its collected pixel position coordinates, corresponding depth position coordinates, and camera internal parameters to determine the second local position information of the task object, the spatial position of the task object in the depth camera coordinate system can be accurately obtained, laying a high-precision data foundation for the subsequent combination of coordinate transformation relationships to obtain the first local position information, realize abnormal object recognition, and global position confirmation, thereby improving the accuracy and reliability of the task robot's perception and positioning of abnormal objects during driving.

[0032] In a possible implementation, the first local position information satisfies a preset height threshold condition, including:

[0033] Obtaining a preset height threshold; wherein the height threshold is determined based on the driving plane of the task object;

[0034] determining an execution height of the execution object based on the first local position information;

[0035] When the execution height is less than the preset height threshold, it is determined that the first local position information meets the preset height threshold condition.

[0036] During the above-mentioned implementation process, by determining the execution height of the task object based on the Z-axis coordinate in the first local position information and comparing it with the preset height threshold, it is possible to accurately determine whether the first local position information meets the height threshold condition, thereby effectively distinguishing between normally executed task objects and abnormal obstacles formed by falling, and providing a reliable judgment basis for the safe driving and path planning of the task robot.

[0037] In a possible implementation, determining the first global position information of the abnormal object according to the first local position information and the second global position information of the task robot includes:

[0038] Acquire a first position coordinate along the robot's travel direction in the first local position information; wherein the first position coordinate represents the relative distance between the abnormal object and the task robot in the forward direction of the task robot;

[0039] The first global position information of the abnormal object is generated according to the relative distance and the second global position information of the task robot.

[0040] During the implementation of the above method, by extracting the relative distance of the abnormal object in the direction of travel in the local coordinate system of the task robot and converting it into the global coordinates of the robot itself, the first global position information of the abnormal object in the global coordinate system of the working area can be accurately generated, which provides a standardized position reference for the subsequent unified driving control of all task robots in the working area (such as path avoidance scheduling), ensuring the spatial positioning consistency and control accuracy when multiple robots collaborate to handle abnormal objects.

[0041] In one possible implementation, controlling each task robot to travel within the work area without collision based on the first global position information and the second global position information and driving path of each task robot includes:

[0042] generating an electronic fence for the abnormal object according to the first global position information and the space occupancy information of the task robot;

[0043] Based on the position information of the electronic fence, and the second global position information and driving path of each task robot, each task robot is controlled to drive without collision in the working area.

[0044] The above implementation method uses electronic fences to clearly define the boundaries of the dangerous area, combines the dynamic comparison of the real-time position and path of the task robot, and generates corresponding control logic, making collision-free driving control more operational and accurate.

[0045] In one possible implementation, controlling each task robot to travel within the work area without collision based on the location information of the electronic fence and the second global location information and driving path of each task robot includes:

[0046] For any task robot, when it is determined based on the second global position information and driving path of the task robot that the task robot spatially overlaps with the position information of the electronic fence at the same time, obtaining the driving state of the task robot;

[0047] When the driving state indicates that the task robot is in motion, controlling the task robot to stop driving; or

[0048] When the driving state indicates that the task robot is not driving, the driving path of the task robot is updated based on the position information of the electronic fence, and the task robot is controlled to drive based on the updated driving path.

[0049] This differentiated control strategy, which distinguishes the real-time status of the robot (driving / not driving), can not only avoid immediate collisions by pausing in emergency situations, but also ensure the driving safety of robots that are not started through path updates, thereby achieving precise collision-free control of robots for each task.

[0050] In a second aspect, an embodiment of the present application provides a control device for a task robot, wherein at least one task robot is present in a working area of ​​the task robot, and each task robot is equipped with a three-dimensional visual sensor for scanning a local area on its driving path, where the local area covers the driving plane of the task robot; the device comprises:

[0051] A position information acquisition module is configured to determine the first global position information of any task robot when the task robot's three-dimensional vision sensor detects an abnormal object in its driving path during the task robot's execution of a task; wherein the global position information is a parameter used to characterize the spatial position in the global coordinate system of the working area; and the first global position information is a spatial coordinate parameter obtained by coordinate conversion based on the three-dimensional visual data acquired by the three-dimensional vision sensor;

[0052] The robot travel control module is used to control each task robot to travel without collision in the working area based on the first global position information and the second global position information of each task robot.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0054] The memory stores computer-executable instructions;

[0055] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0057] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0058] In a sixth aspect, an embodiment of the present application provides a control system for a task robot, comprising:

[0059] At least one task robot; wherein each task robot is equipped with a three-dimensional vision sensor for scanning a local area on its driving path, wherein the local area covers the driving plane of the task robot;

[0060] An electronic device, used to execute the task robot control method in the first aspect and / or various possible implementations of the first aspect.

[0061] The control method, device, electronic device, and system for task robots provided in the embodiments of the present application, by configuring each task robot with a three-dimensional vision sensor capable of scanning a local area of ​​the travel plane in the forward direction, can accurately detect abnormal objects on the travel path and determine their first global position information, and then combine the second global position information of each task robot to achieve collision-free travel control. In the above scheme, the three-dimensional vision sensor can effectively identify abnormal objects of different sizes and heights to improve the comprehensiveness of detection; in addition, the task robots can be controlled in a targeted manner based on the global position information, avoiding unnecessary downtime of unrelated robots, thereby ensuring the operational safety of the task robots while improving the efficiency of task execution within the overall work area. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0063] Figure 1 Schematic diagram of the task robot control scenario provided for this application Figure 1 ;

[0064] Figure 2 Schematic diagram of the task robot control scenario provided for this application Figure 2 ;

[0065] Figure 3 This is a diagram of an application scenario when the control method provided in this application is collaboratively executed by a task robot and an upstream system;

[0066] Figure 4 This is an application scenario diagram when the control method provided in this application is executed solely by an upstream system;

[0067] Figure 5 A schematic diagram of a control method for a task robot provided in this application Figure 1 ;

[0068] Figure 6 A schematic diagram of a coordinate system conversion scenario provided by this application;

[0069] Figure 7 A schematic diagram of the structure of a controller provided in this application;

[0070] Figure 8 This is a structural block diagram of an electronic device provided by this application.

[0071] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0072] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0073] When robots are performing sorting tasks on the sorting platform, cargo often falls due to its large size, center of gravity offset, or scratches. These dropped cargo not only affect the robot's task completion but can also hinder the normal operation of other robots. Therefore, it is necessary to notify staff promptly to clean up the cargo to ensure stable operation of the system.

[0074] As noted in the background, current sorting platforms often use a single-line radar scanning solution for object recognition. This method uses radar scanning to detect dropped goods or other unusual objects on the platform. However, this solution has significant limitations in terms of recognition coverage and system efficiency. Firstly, due to hardware installation constraints, its ability to recognize certain object shapes (such as small objects) is insufficient, making comprehensive early warning difficult. Secondly, its fault handling mechanism lacks specificity, which can easily cause unrelated equipment to be forced to shut down, thus impacting overall business process efficiency. In summary, when robots perform tasks, anomaly detection and robot control based on single-line radar scanning present technical challenges, resulting in poor control effectiveness.

[0075] The control method of the task robot provided in this application is intended to solve the above technical problems. Specifically, by configuring each task robot with a three-dimensional visual sensor that can scan the local area of ​​the driving plane in the forward direction, it is possible to accurately detect abnormal objects on the driving path and determine their first global position information, and then combine the second global position information of each task robot to achieve collision-free driving control. In the above scheme, abnormal objects of different sizes and heights can be effectively identified by the three-dimensional visual sensor to improve the comprehensiveness of detection; in addition, the task robots can be controlled in a targeted manner based on the global position information to avoid unnecessary shutdown of unrelated robots, thereby ensuring the operational safety of the task robots while improving the efficiency of task execution in the overall working area.

[0076] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0077] In order to more clearly understand the control method of the task robot provided by this application, the following will be combined with Figure 1 and Figure 2 A brief description is given of the application scenarios to which the control method of the task robot in this application is applicable.

[0078] Combine Figure 1 and Figure 2 The application scenarios provided by the embodiments of the present application are suitable for scenarios where robots need to complete tasks autonomously and ensure driving safety and efficiency, such as automated warehousing and sorting, workshop material transfer, etc.

[0079] Exemplarily, any application scenario may specifically include: at least one task robot is deployed in the working area of ​​the task robot, and each task robot is equipped with a three-dimensional vision sensor, which is used to scan the local area on its driving path, and the local area can cover the driving plane in the forward direction of the task robot.

[0080] It should be noted that in order to achieve real-time environmental perception of the task robot's driving path, each task robot can be equipped with a three-dimensional visual sensor, wherein the three-dimensional visual sensor includes but is not limited to a depth camera and a lidar, etc., and its scanning range covers the local area in front of the task robot when it is moving. Specifically, the scanning field of view of the sensor not only includes the plane on which the task robot is currently driving, that is, the ground or the surface of the work platform, but also includes the space at a certain height above the plane, so as to be able to capture various objects or obstacles that may exist in the direction of the task robot's forward movement. These obstacles include fallen goods, other robots, protrusions, etc. The obstacle recognition results can provide accurate environmental data support for subsequent path planning and driving control, thereby achieving both ensuring its operational safety and effectively improving the overall work efficiency of the task robot when controlling the robot's operation process.

[0081] In this application, the control method of the task robot can be performed by the task robot in conjunction with the preset upstream system, or it can be completed by the upstream system alone. Figure 2 and Figure 3 The specific process of the technical solution of this application is described in detail. Figure 3 This is an application scenario diagram of the control method provided in this application when it is collaboratively executed by a task robot and an upstream system. Figure 4 This is an application scenario diagram when the control method provided in this application is executed solely by an upstream system.

[0082] exist Figure 3 In the coordinated execution scenario shown, the task robot first obtains the sensor data collected by the three-dimensional vision sensor. After analyzing the data, if it determines that there is an abnormal object on its driving path, it generates the local position information of the abnormal object in the robot's local coordinate system and uploads the local position information to the upstream system.

[0083] After receiving the local position information, the upstream system determines the first global position information of the abnormal object in the global coordinate system of the working area through coordinate conversion; at the same time, the upstream system synchronously obtains the second global position information of all task robots in the working area.

[0084] Based on the above-mentioned first global position information, and the second global position information and driving paths of all task robots, the upstream system uniformly dispatches each task robot to ensure that it can achieve collision-free driving within the working area.

[0085] exist Figure 4In the described scenario of independent execution, the three-dimensional vision sensor transmits the detected sensor data directly to the upstream system; the upstream system analyzes the sensor data. If it is determined that there is an abnormal object on the driving path of the corresponding task robot, the second global position information of the task robot in the global coordinate system of the working area and the sensor data can be combined to calculate the first global position information of the abnormal object in the same global coordinate system.

[0086] During this process, the upstream system synchronously obtains the second global position information and driving paths of all task robots in the working area, and then uniformly controls each task robot based on the above-mentioned first global position information and the second global position information and driving paths of all task robots to ensure that they can achieve collision-free driving in the working area.

[0087] The following detailed description of the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems is provided with specific embodiments. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The technical solution of the present application will be described below in conjunction with the accompanying drawings.

[0088] Figure 5 A schematic diagram of a control method for a task robot provided in this application Figure 1 The method in this application can be implemented by software, hardware or a combination of software and hardware. Figure 5 As shown, the method includes the following steps:

[0089] S501. During the execution of a task by any task robot, when the three-dimensional visual sensor of the task robot detects an abnormal object on its driving path, determine the first global position information of the abnormal object.

[0090] In this application, a task robot can be understood as a robot that performs a preset task in a specific work area, and its type varies depending on the type of work. For example, when the work area is a sorting platform, the corresponding task robot can be a sorting robot that performs sorting operations.

[0091] To ensure the robot can successfully complete its mission, each task robot is equipped with a 3D vision sensor to detect obstacles in its path, known as abnormal objects. If an abnormal object is detected, the 3D visual data collected by the sensor is transformed to determine the abnormal object's spatial position parameters within the global coordinate system of the work area, known as the first global position information.

[0092] During the implementation of the above method, when the task robot deploys a three-dimensional vision sensor, it can be installed on its own central axis to ensure that the sensor always faces the front of the path during the robot's walking process; on this basis, it is also necessary to set a pitch angle at a certain downward angle to ensure that its scanning field of view can cover a sufficiently rich spatial range.

[0093] This deployment method, which takes both mounting position and angle into consideration, allows the 3D vision sensor's scanning field of view to cover a wide range of spaces, effectively identifying unusual objects of varying sizes and heights. This enhances the robot's comprehensive detection capabilities and provides greater safety during operation.

[0094] S502: Based on the first global position information, and the second global position information and driving path of each task robot, control each task robot to drive in the working area without collision.

[0095] In this application, since an abnormal object will not only hinder the driving of the task robot that recognizes it, but may also affect the driving of other task robots in the work area, after obtaining the first global position information of the abnormal object in the global coordinate system of the work area, it is necessary to synchronously obtain the second global position information and driving path corresponding to each of the task robots in the work area. Furthermore, based on the first global position information of the abnormal object and the second global position information and driving paths of all task robots, the spatial distance and path intersection between each task robot and the abnormal object are determined. By dynamically planning obstacle avoidance routes, adjusting driving speeds, or temporarily changing the order of tasks, it is ensured that all task robots can bypass the abnormal object in the work area, thereby achieving collision-free and efficient collaborative operations.

[0096] In summary, by controlling the task robots in a targeted manner through global position information and driving paths, unnecessary shutdown of unrelated task robots can be avoided, thereby ensuring the operational safety of the task robots while improving the efficiency of task execution in the overall work area.

[0097] The technical solution provided by this application, by configuring each task robot with a three-dimensional visual sensor that can scan a local area of ​​the driving plane in the forward direction, can accurately detect abnormal objects on the driving path and determine their first global position information, and then combine the second global position information and driving path of each task robot to achieve collision-free driving control. In the above solution, the deployment method that takes into account the installation position and angle setting allows the scanning field of view of the three-dimensional visual sensor to cover a rich spatial range, thereby being able to effectively identify abnormal objects of different sizes and heights. In this way, by enhancing the comprehensiveness of the detection of the task robot, a higher safety guarantee is provided during its operation; in addition, the task robot is controlled in a targeted manner through the global position information and driving path, ensuring that all task robots can bypass abnormal objects in the working area, avoiding unnecessary shutdown of irrelevant task robots, thereby ensuring the safety of the task robot operation while also improving the efficiency of task execution in the overall working area.

[0098] Next, the process of determining whether an abnormal object exists based on a three-dimensional visual sensor and obtaining the first global information of the abnormal object is described in detail. However, the following content is only an exemplary description of this solution and cannot be used as a limitation of this solution.

[0099] In order to clearly introduce the technical solution of this application, the detection process of a three-dimensional vision sensor deployed on an arbitrary task robot is described below.

[0100] For any task robot, an optional implementation method for determining that its three-dimensional vision sensor detects the presence of an abnormal object on its driving path may include: determining the first local position information of the task object when the three-dimensional vision sensor detects the presence of the task object on its driving path; and determining that the task object is an abnormal object when the first local position information meets a preset height threshold condition.

[0101] In this application, the task object can be understood as the working object of the task robot. For example, for a sorting robot, the task object is the goods to be sorted.

[0102] It can be explained that the first local position information is the spatial coordinate parameters of the task object in the first local coordinate system of the task robot. In other words, the first local position information is a parameter used to describe the spatial position of the task object, and its coordinate reference is the first local coordinate system of the task robot itself.

[0103] For example, a coordinate system can be established with the task robot as the origin, for example, a three-dimensional coordinate system with the center of the task robot as the origin, with the forward X-axis, the rightward Y-axis, and the upward Z-axis. The specific coordinate values ​​of the position of the task object are the X, Y, and Z axis coordinates.

[0104] The first local position information can be used to intuitively obtain the orientation and distance of the task object relative to the task robot, which is the basic data for the robot to judge the spatial relationship between itself and the task object.

[0105] Based on this, when the task robot travels within the work area, its deployed 3D vision sensor scans parts of the travel path in real time. When an object is detected on the path, it will first identify the object. If it is determined to be a task object, it will further identify the object and determine its first local position information in the task robot's coordinate system. By comparing this first local position information with a preset height threshold, it can be determined whether the task object is currently being performed by another task robot or has fallen into the work area.

[0106] For example, taking the sorting robot scenario as an example, when other task robots are performing operations, their task objects, that is, the goods to be sorted, are usually carried by the robot body and maintained above the work surface. At this time, the first local position information of the task object contains its height relative to its driving plane, that is, the ground or the work surface. This height can also be referred to as the execution height of the task object when it is executed. Due to the structural differences of different types of task robots and the physical characteristics of different task objects, there may be differences in the execution height. Therefore, the preset height threshold condition in this scheme can be dynamically generated based on statistical analysis of the execution height distribution of multiple task objects under normal execution status. When it is detected that the execution height of a task object is lower than the threshold, it is determined to be an abnormal object that may hinder passage.

[0107] During the driving process of the task robot, the above-mentioned scheme can accurately distinguish between task objects in normal operating conditions and task objects in abnormal conditions such as falling by comparing the first local position information of the task object with a preset height threshold set based on the distance between it and the ground or the horizontal plane of the workbench, providing a reliable basis for the accurate identification of abnormal objects, thereby improving the smoothness and safety of the task robot's path planning and safe driving.

[0108] During the implementation of the above-mentioned scheme, an optional implementation method in which the three-dimensional vision sensor detects the presence of a task object on the driving path of the task robot may include: when the three-dimensional vision sensor recognizes a physical object within its scanning field of view, calling a preset object recognition model; inputting the two-dimensional visual data containing the physical object obtained by the three-dimensional vision sensor into the object recognition model, and determining whether the physical object belongs to a preset task object category by analyzing the three-dimensional visual data.

[0109] As the task robot moves, it reads sensor data from the 3D vision sensor in real time. Optionally, the sensor data can be divided into 2D vision data and 3D vision data. 2D vision data corresponds to visual information along the X and Y axes, while 3D vision data corresponds to visual information along the Z axis.

[0110] Specifically, sensor data may include video data. To facilitate subsequent recognition, the video data can be captured as multiple frames of image data, from which images containing physical objects are filtered to obtain target images. Physical objects here refer to all objects within the work area, including but not limited to the task object and other task robots.

[0111] During this process, the preset object recognition model can also be called synchronously, and the targets can be input into the model one by one to perform object type recognition, so as to determine whether the entity object in the image is the task object.

[0112] Alternatively, the object recognition model can be built using a deep learning framework and trained by integrating multi-dimensional data collected by visual sensors, enabling it to accurately classify objects for different types of tasks. For example, the model can extract key features such as the physical object's outline, size, and height to distinguish between the task object, other task robots, fixed facilities, and foreign objects, and output the corresponding category label and confidence score.

[0113] In addition, to adapt to the dynamic changes in the working area, the model can also support online incremental learning. By continuously receiving newly labeled data and iteratively optimizing parameters, it can improve the robustness of recognition of new task objects or objects with special shapes, and provide a reliable classification basis for subsequent anomaly judgment and path planning.

[0114] The above implementation method, by real-time collection and processing of three-dimensional visual sensor data, and combining it with the classification and processing of physical objects by deep learning models, can identify whether there are task objects on the driving path, providing a reliable basis for subsequent anomaly detection and path planning, and improving the task execution effect of the task robot.

[0115] When a task object is detected in the driving path of a task robot based on a three-dimensional vision sensor, it is necessary to determine the spatial coordinate parameters of the task object in the first local coordinate system of the task robot, that is, the first local position information, so as to judge whether the task object is an abnormal object in the working area according to preset conditions.

[0116] Optionally, an optional implementation method for determining the first local position information of the task object includes: determining the second local position information of the task object; wherein the second local position information is the spatial coordinate parameters of the task object in the second coordinate system to which the three-dimensional vision sensor belongs; obtaining the coordinate transformation relationship between the first coordinate system and the second coordinate system; and converting the second local position information according to the coordinate transformation relationship to obtain the first local position information of the task object.

[0117] In this application, after the 3D vision sensor recognizes the task object, it can determine the spatial coordinate parameters of the task object in its pre-established second local coordinate system, i.e., the second local position information. Here, the second local coordinate system can also be interpreted as a spatial coordinate system established based on the 3D vision sensor itself.

[0118] Because the 3D vision sensor is fixedly mounted on the task robot, and there exists a fixed spatial relationship between the two, the coordinate transformation relationship between the second local coordinate system and the first local coordinate system of the task robot can be pre-determined. Since the first local coordinate system is based on the spatial coordinate system established by the task robot itself, this coordinate transformation relationship can also be referred to as the position calibration result between the 3D vision sensor and the task robot.

[0119] Subsequently, based on the above-mentioned coordinate transformation relationship, coordinate transformation processing is performed on the second local position information to obtain the spatial coordinate parameters of the task object in the first local coordinate system.

[0120] During the above-mentioned position conversion process, the task robot accurately obtains the spatial position information of the task object relative to itself based on its own first local coordinate system, which can provide basic data support for the subsequent abnormal object determination of the task object and the execution of related control logic.

[0121] When the first local position information of the task object is determined based on the above method, it is judged based on a preset height condition threshold, and when it meets the preset height condition threshold, it is determined whether it is an abnormal object. Optionally, an optional implementation method of determining whether the first local position information meets the preset height threshold condition may include: obtaining a preset height threshold; wherein the height threshold is determined based on the driving plane of the task object;

[0122] Based on the first local position information, an execution height of the execution object is determined; when the execution height is less than the preset height threshold, it is determined that the first local position information meets the preset height threshold condition.

[0123] Optionally, since the first local position information includes the specific coordinate values ​​of the location of the task object, i.e., the X, Y, and Z axis coordinates, the execution height of the task object when being executed is determined based on the Z axis coordinate in the above coordinate values. At the same time, a height threshold value dynamically generated by statistically analyzing the execution height distribution of various task objects under normal execution status can be obtained in advance. On this basis, the execution height is compared with the preset height threshold; optionally, if the execution height of the task object is higher than the preset height threshold, it means that the first local position information does not meet the preset height threshold condition. At this time, the task object is in a state of being normally executed by the task robot and is not an abnormal object. On the contrary, if the execution height of the task object is lower than the preset height threshold, it means that the first local position information meets the preset height threshold condition, i.e., the task object has fallen to the driving plane of the task robot and is an obstacle that may hinder the robot's travel, i.e., an abnormal object.

[0124] During the above-mentioned implementation process, by determining the execution height of the task object based on the Z-axis coordinate in the first local position information and comparing it with the preset height threshold, it is possible to accurately determine whether the first local position information meets the height threshold condition, thereby effectively distinguishing between normally executed task objects and abnormal obstacles formed by falling, and providing a reliable judgment basis for the safe driving and path planning of the task robot.

[0125] When it is determined that there is an abnormal object on the driving path of the task robot, that is, there is an obstacle that hinders the driving of the task robot, its spatial coordinate parameters in the global coordinate system of the working area can be determined, that is, the first global position information, and based on this position information, driving control processing can be performed on all task robots in the working area.

[0126] In the above implementation process, an optional implementation method for determining the first global position information of the abnormal object may include: obtaining the first position coordinate along the robot's travel direction in the first local position information; wherein, the first position coordinate represents the relative distance between the abnormal object and the task robot in the forward direction of the task robot; based on the relative distance and the second global position information of the task robot, generating the first global position information of the abnormal object.

[0127] Specifically, the first position coordinate along the task robot's travel direction (the X-axis direction in the first local position information) can be filtered from the abnormal object's spatial coordinate parameters in the task robot's coordinate system, i.e., the first local position information. Here, the first position coordinate can reflect the abnormal object's relative distance from the task robot along its travel path. For example, an X-axis coordinate value of 50 cm indicates that the abnormal object is 50 cm in front of the robot.

[0128] During this period, the spatial coordinate parameters of the task robot in the global coordinate system of the working area, that is, the second global position information, are obtained. For example, the coordinate parameters can be expressed as (X0, Y0, Z0) in the global coordinate system.

[0129] Since the first position coordinates have clearly defined the distance of the abnormal object relative to the task robot in the forward direction, the relative distance can be combined with the second global position information of the robot through coordinate conversion to calculate the first global position information of the abnormal object in the global coordinate system.

[0130] Specifically, if the forward coordinate of the task robot's second global position information is X0 and the relative distance of the abnormal object is ΔX, the forward coordinate of the abnormal object in the global coordinate system is X1 = X0 + ΔX. Furthermore, since the abnormal object is located on the task robot's travel path, its Y-axis and Z-axis coordinates in the global coordinate system can be determined based on Y0 and Z0 in the robot's second global position information: Y1 = Y0 and Z1 = Z0. Ultimately, combining the three coordinate parameters X1, Y1, and Z1 yields the abnormal object's first global position information (X1, Y1, Z1) in the global coordinate system.

[0131] During the implementation of the above method, by extracting the relative distance of the abnormal object in the direction of travel in the local coordinate system of the task robot and converting it into the global coordinates of the robot itself, the first global position information of the abnormal object in the global coordinate system of the working area can be accurately generated, which provides a standardized position reference for the subsequent unified driving control of all task robots in the working area (such as path avoidance scheduling), ensuring the spatial positioning consistency and control accuracy when multiple robots collaborate to handle abnormal objects.

[0132] In some application scenarios, the task robot may be a sorting robot, and the 3D vision sensor deployed on its body may be a depth camera. Based on this, the following uses a depth camera as an example to explain in detail the process of identifying abnormal objects and confirming global position information during the task robot's operation. It should be noted that this example is provided solely to facilitate understanding of the technical solution of this application and does not limit the scope of protection of this application.

[0133] Optionally, the three-dimensional vision sensor is a depth camera; the three-dimensional vision data includes pixel position coordinates and depth position coordinates; accordingly, an optional implementation method for determining the second local position information of the task object includes: obtaining the pixel position coordinates of the task object, and the depth position coordinates corresponding to the pixel position coordinates; determining the second local position information of the task object based on the pixel position coordinates, depth position coordinates and the camera intrinsic parameters of the depth camera.

[0134] Specifically, while the task robot is driving, it reads the data stream from the vehicle's onboard depth camera in real time. The data stream is divided into RBG data stream and depth data stream, and the two are aligned in the camera driver content.

[0135] Based on the data stream, the video frame is intercepted to obtain an RGB image of any frame. In addition, the RGB image is cropped to retain the range of the forward path in the camera's field of view.

[0136] On this basis, if there is a physical object in the camera's field of view, the preset AI model is called to identify the category of the physical object. Optionally, the AI ​​model can be a model trained based on YOLOv8 to achieve high-precision real-time detection and recognition of multiple categories and multiple targets. Subsequently, the RGB image is input into the model for recognition; when the physical object in the image is detected as a cargo, the pixel coordinates U (u x ,u y ).

[0137] During this period, the depth image corresponding to the same timestamp is intercepted from the depth data stream, that is, the pixel coordinate U(u x ,u y ) corresponds to the depth pixel value z depth , that is, get the distance in the depth Z direction in the camera coordinate system.

[0138] According to the camera firmware, the known camera internal parameters F are read, and the spatial coordinates P ( , , ).

[0139] Specifically, the expression for calculating the spatial coordinates of the cargo in the camera coordinate system may include: .

[0140] Furthermore, if Figure 6 As shown, the transformation relationship [R, T] between the camera coordinate system obtained by the calibration tool and the global coordinate system is obtained, where Represents the rotation relationship between two coordinate systems, Indicates the translation relationship between the two coordinate systems. In this way, based on the transformation relationship and the spatial coordinates obtained in the camera coordinate system, the coordinates Q (X, Y, Z) of the cargo in the global coordinate system are obtained.

[0141] Specifically, the expression for calculating the coordinates of the cargo in the global coordinate system may include: .

[0142] Extract the Z value of the obtained Q coordinate, that is, the height Z direction of the goods in the global coordinate system; if the height value is lower than the preset height plane, it means that the goods are suspected to have fallen on the sorting platform and are on the current sorting robot's route, which is an abnormal acquisition; otherwise, it means that the goods are being sorted.

[0143] When the cargo is determined to be abnormal, the upstream system containing the sorting platform map and sorting task information will control all sorting robots that may be affected by it to adjust the sorting path based on the coordinates Q of the abnormal cargo, so as to ensure the safety of the sorting robot operation while improving the efficiency of sorting task execution in the entire work area.

[0144] On the basis of the above, when an abnormal object is found in the working area, in order to control each task robot to travel without collision in the working area, each task robot can be controlled in a targeted manner based on the first global position information of the abnormal object and the second global position information of each task robot in the working area, thereby avoiding unnecessary shutdown of unrelated robots.

[0145] Optionally, an optional implementation method for collision-free driving control of each task robot may include: generating an electronic fence for the abnormal object based on the first global position information and the space occupancy information of the task robot; and controlling each task robot to drive collision-free within the working area based on the position information of the electronic fence and the second global position information and driving path of each task robot.

[0146] In this application, space occupancy information can be understood as information about the area or volume occupied by the task robot in three-dimensional space. Here, the space occupancy information of the task robot includes its own external dimensions and safety buffer distance, such as a 10cm buffer space reserved to avoid collisions.

[0147] Specifically, after determining the first global position information X1, Y1, and Z1 of the abnormal object, a three-dimensional electronic fence can be generated with (X1, Y1) as the center, combined with the estimated volume of the abnormal object and the robot's safety buffer distance. This fence covers the abnormal object and the danger zone where a collision may occur. For example, the electronic fence is centered on X1 with an X-axis range of ±20 cm, centered on Y1 with a Y-axis range of ±20 cm, and the Z-axis extends from the ground to the height of the top of the abnormal object.

[0148] Then, the second global position information of each task robot and its preset path are obtained in real time. For example, the coordinates of robot A are (X a 、Y a 、Z a ), the coordinates of robot B are (X b 、Y b、Z b );Robot A plans to start from (X a1 、Y a1 ) Drive to (X a2 、Y a2 ), Robot B plans to start from (X b1 、Y b1 ) Drive to (X b2 、Y b2 ).

[0149] By comparing each task robot's real-time position and driving path with the position of the electronic fence, each task robot is controlled to avoid collisions. Optionally, if a task robot's current position is less than a preset distance from the fence, it will be immediately stopped. If its preset path intersects the fence, the robot will be rerouted, for example, to a point 10 cm outside the fence's X-axis. If the paths do not intersect, the robot is allowed to proceed as planned. Ultimately, this ensures that all task robots avoid dangerous areas covered by electronic fences within their work area, achieving collision-free driving.

[0150] The above implementation method uses electronic fences to clearly define the boundaries of the dangerous area, combines the dynamic comparison of the real-time position and path of the task robot, and generates corresponding control logic, making collision-free driving control more operational and accurate.

[0151] Optionally, an optional implementation method for collision-free driving control of each task robot may include: for any task robot, when it is determined based on the second global position information and driving path of the task robot that there is spatial overlap with the position information of the electronic fence at the same time, obtaining the driving status of the task robot; when the driving status indicates that the task robot is in driving, controlling the task robot to pause driving; or, when the driving status indicates that the task robot is not driving, updating the driving path of the task robot based on the position information of the electronic fence, and controlling the task robot to drive based on the updated driving path.

[0152] Specifically, take robots A and B in the working area as an example: assume that an electronic fence has been generated for abnormal objects, and its range in the global coordinate system is X∈[5,7], Y∈[3,5], and the Z axis is from the ground to a height of 50 cm.

[0153] For robot A, its second global position is (6, 4, 0), and its pre-set path is a straight line from (6, 4) to (8, 4). The system detects through real-time comparison that robot A is within the geo-fence (there is spatial overlap with the fence) and its driving status is "driving." At this point, the system immediately sends a pause command to robot A, halting its movement and preventing it from colliding with an undesirable object.

[0154] For robot B, its second global position is (3,4,0), and its preset path is a straight line from (3,4) to (7,4). Upon detection, robot B's preset path will cross the X∈[5,7], Y=4 segment of the geo-fence, meaning it will overlap with the fence at some point in the future. However, robot B is currently not in motion, such as in the waiting state or awaiting instructions. At this point, the path can be replanned based on the geo-fence location information. For example, the original path can be adjusted from (3,4) → (5,2) → (7,2) → (7,4), allowing the new path to bypass the geo-fence area. Robot B can then be controlled to follow the updated path.

[0155] This differentiated control strategy, which distinguishes the real-time status of the robot (driving / not driving), can not only avoid immediate collisions by pausing in emergency situations, but also ensure the driving safety of robots that are not started through path updates, thus achieving precise collision-free control of robots for each task.

[0156] Figure 7 This is a schematic diagram of the structure of a controller provided by this application. Figure 7 The control device 70 of the task robot includes: a position information acquisition module 701 and a robot travel control module 702; wherein,

[0157] The position information acquisition module 701 is used to determine the first global position information of any task robot when the task robot's three-dimensional vision sensor detects the presence of an abnormal object in its driving path during the task robot's execution of the task. The global position information is a parameter used to represent the spatial position in the global coordinate system of the working area. The first global position information is a spatial coordinate parameter obtained by coordinate conversion based on the three-dimensional visual data obtained by the three-dimensional vision sensor.

[0158] The robot travel control module 702 is used to control each task robot to travel in the working area without collision based on the first global position information and the second global position information of each task robot.

[0159] In a possible implementation, the location information acquisition module 701 includes:

[0160] A first local position information determination submodule is configured to determine the first local position information of the task object when the three-dimensional vision sensor detects that the task object exists on its driving path; wherein the first local position information is the spatial coordinate parameters of the task object in the first local coordinate system to which the task robot belongs;

[0161] an abnormal object determination submodule, configured to determine that the task object is an abnormal object when the first local position information meets a preset height threshold condition;

[0162] The first global position information determining submodule is used to determine the first global position information of the abnormal object based on the first local position information and the second global position information of the task robot.

[0163] In a possible implementation, the first local position information determining submodule includes:

[0164] a model calling unit, configured to call a preset object recognition model when the three-dimensional vision sensor recognizes a physical object within its scanning field of view;

[0165] The object category determination unit is used to obtain two-dimensional visual data of the physical object and input the two-dimensional visual data into the object recognition model to determine whether the physical object belongs to a preset task object category.

[0166] In a possible implementation, the first local position information determining submodule includes:

[0167] A second local position information determining unit, configured to determine the second local position information of the task object; wherein the second local position information is a spatial coordinate parameter of the task object in a second local coordinate system of the three-dimensional vision sensor;

[0168] a coordinate transformation relationship determining unit, configured to obtain a coordinate transformation relationship between the first local coordinate system and the second local coordinate system;

[0169] The first local position information determining unit is configured to perform conversion processing on the second local position information according to the coordinate conversion relationship to obtain the first local position information of the task object.

[0170] In a possible implementation, the three-dimensional vision sensor is a depth camera;

[0171] The three-dimensional visual data includes pixel position coordinates and depth position coordinates;

[0172] The second local position information determining unit includes:

[0173] A position coordinate acquisition subunit, configured to acquire the pixel position coordinates of the task object and the depth position coordinates corresponding to the pixel position coordinates;

[0174] The second local position information determining subunit is configured to determine the second local position information of the task object according to the pixel position coordinates, the depth position coordinates, and the camera intrinsic parameters of the depth camera.

[0175] In a possible implementation, the abnormal object determination submodule includes:

[0176] A threshold value obtaining unit, configured to obtain a preset height threshold value; wherein the height threshold value is determined based on the driving plane of the task object;

[0177] an execution height determining unit, configured to determine an execution height of the execution object based on the first local position information;

[0178] The abnormal object determining unit is configured to determine, when the execution height is less than the preset height threshold, whether the first local position information satisfies a preset height threshold condition.

[0179] In a possible implementation, the first global position information determination submodule includes:

[0180] A first position coordinate acquisition unit is configured to acquire a first position coordinate in the first local position information along the robot's travel direction; wherein the first position coordinate represents a relative distance between the abnormal object and the task robot in the forward direction of the task robot;

[0181] The first global position information determining unit is used to generate the first global position information of the abnormal object according to the relative distance and the second global position information of the task robot.

[0182] In one possible implementation, the robot travel control module 702 includes:

[0183] an electronic fence generation submodule, configured to generate an electronic fence for the abnormal object according to the first global position information and the space occupancy information of the task robot;

[0184] The driving control submodule is used to control each task robot to drive without collision in the working area based on the position information of the electronic fence and the second global position information and driving path of each task robot.

[0185] In one possible implementation, the driving control submodule includes:

[0186] a driving state acquiring unit, configured to acquire the driving state of any task robot when it is determined based on the second global position information and driving path of the task robot that the task robot spatially overlaps with the position information of the electronic fence at the same time;

[0187] A first driving control unit is configured to control the task robot to stop driving when the driving state indicates that the task robot is driving; or

[0188] The second driving control unit is used to update the driving path of the task robot based on the position information of the electronic fence when the driving state indicates that the task robot is not driving, and control the task robot to drive based on the updated driving path.

[0189] This application also provides a control system for a task robot. The system includes: at least one task robot; each task robot is equipped with a three-dimensional vision sensor for scanning a local area on its travel path, where the local area covers the travel plane of the task robot; and electronic equipment for executing the task robot control method described in the above embodiments.

[0190] Figure 8 This is a structural block diagram of an electronic device provided by this application. Figure 8 , device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output interface 812 , a sensor component 814 , and a communication component 816 .

[0191] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 802 may include one or more modules to facilitate interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate interaction between the multimedia component 808 and the processing component 802.

[0192] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0193] The power supply component 806 provides power to the various components of the device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 800.

[0194] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, it may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensors can detect not only the boundaries of a touch or slide action, but also the duration and pressure associated with the touch or slide action. In some embodiments, the multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a capture mode or a video mode, the front-facing camera and / or the rear-facing camera can receive external multimedia data. Each front-facing camera and the rear-facing camera can have a fixed optical lens system or have variable focal length and optical zoom capabilities.

[0195] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals may be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0196] The input / output interface 812 provides an interface between the processing component 802 and peripheral interface modules, such as a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0197] Sensor assembly 814 includes one or more sensors for providing various status assessments of device 800. For example, sensor assembly 814 can detect the open / closed state of device 800, the relative positioning of components, such as the display and keypad of device 800. Sensor assembly 814 can also detect changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and changes in the temperature of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include an optical sensor, such as a Complementary Metal Oxide Semiconductor (CMOS) sensor or a Charge-Coupled Device (CCD) sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0198] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 4G or 5G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra wide band (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0199] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0200] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions. The instructions are executable by the processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, or the like.

[0201] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a server, enables the server to execute the control method of the task robot mentioned above.

[0202] An embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the control method of the task robot in the above embodiment.

[0203] An embodiment of the present application also provides a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are run on a computer, the computer executes the technical solution of the control method of the task robot in the above embodiment.

[0204] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the control method of the task robot in the above embodiment.

[0205] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0206] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0207] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0208] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A control method for a task robot, characterized in that: There is at least one task robot in the working area of ​​the task robot, and each task robot is equipped with a three-dimensional visual sensor for scanning a local area on its driving path, and the local area covers the driving plane in the forward direction of the task robot; The method comprises: During the execution of a task by any task robot, if the three-dimensional vision sensor of the task robot detects the presence of an abnormal object on its driving path, determining the first global position information of the abnormal object; wherein the global position information is a parameter used to characterize the spatial position in the global coordinate system of the working area; the first global position information is a spatial coordinate parameter obtained after coordinate conversion based on the three-dimensional visual data acquired by the three-dimensional vision sensor; Based on the first global position information, and the second global position information and driving path of each task robot, each task robot is controlled to travel in the working area without collision.

2. The method according to claim 1, characterized in that The three-dimensional visual sensor of the task robot detects an abnormal object on its driving path, including: When the three-dimensional vision sensor detects that there is a task object on its driving path, determining the first local position information of the task object; wherein the first local position information is the spatial coordinate parameter of the task object in the first local coordinate system to which the task robot belongs; When the first local position information meets a preset height threshold condition, determining that the task object is an abnormal object; Accordingly, determining the first global location information of the abnormal object includes: The first global position information of the abnormal object is determined according to the first local position information and the second global position information of the task robot.

3. The method according to claim 2, characterized in that The three-dimensional vision sensor detects that there is a task object on its driving path, including: In the case where the three-dimensional vision sensor recognizes a physical object within its scanning field of view, calling a preset object recognition model; Two-dimensional visual data of the physical object is acquired, and the two-dimensional visual data is input into the object recognition model to determine whether the physical object belongs to a preset task object category.

4. The method according to claim 2, characterized in that Determining first local position information of the task object includes: Determine the second local position information of the task object; wherein the second local position information is the spatial coordinate parameter of the task object in the second local coordinate system of the three-dimensional vision sensor; Acquire a coordinate transformation relationship between the first local coordinate system and the second local coordinate system; The second local position information is transformed according to the coordinate transformation relationship to obtain the first local position information of the task object.

5. The method according to claim 4, characterized in that The three-dimensional visual sensor is a depth camera; The three-dimensional visual data includes pixel position coordinates and depth position coordinates; Determining the second local position information of the task object includes: Obtaining the pixel position coordinates of the task object and the depth position coordinates corresponding to the pixel position coordinates; Determine second local position information of the task object according to the pixel position coordinates, the depth position coordinates, and the camera intrinsic parameters of the depth camera.

6. The method according to claim 2, characterized in that The first local position information satisfies a preset height threshold condition, including: Obtaining a preset height threshold; wherein the height threshold is determined based on the driving plane of the task object; determining an execution height of the execution object based on the first local position information; When the execution height is less than the preset height threshold, it is determined that the first local position information meets the preset height threshold condition.

7. The method according to claim 2, characterized in that Determining the first global position information of the abnormal object according to the first local position information and the second global position information of the task robot includes: Acquire a first position coordinate along the robot's travel direction in the first local position information; wherein the first position coordinate represents the relative distance between the abnormal object and the task robot in the forward direction of the task robot; The first global position information of the abnormal object is generated according to the relative distance and the second global position information of the task robot.

8. The method according to any one of claims 1 to 7, characterized in that Based on the first global position information, and the second global position information and driving path of each task robot, controlling each task robot to travel within the working area without collision, including: generating an electronic fence for the abnormal object according to the first global position information and the space occupancy information of the task robot; Based on the position information of the electronic fence, and the second global position information and driving path of each task robot, each task robot is controlled to drive without collision in the working area.

9. The method according to claim 8, characterized in that Based on the position information of the electronic fence, and the second global position information and driving path of each task robot, controlling each task robot to travel within the working area without collision, including: For any task robot, when it is determined based on the second global position information and driving path of the task robot that the task robot spatially overlaps with the position information of the electronic fence at the same time, obtaining the driving state of the task robot; When the driving state indicates that the task robot is in motion, controlling the task robot to stop driving; or When the driving state indicates that the task robot is not driving, the driving path of the task robot is updated based on the position information of the electronic fence, and the task robot is controlled to drive based on the updated driving path.

10. A control device for a task robot, characterized in that: There is at least one task robot in the working area of ​​the task robot, and each task robot is equipped with a three-dimensional visual sensor for scanning a local area on its driving path, and the local area covers the driving plane of the task robot; the device includes: A position information acquisition module is configured to determine the first global position information of any task robot when the task robot's three-dimensional vision sensor detects an abnormal object in its driving path during the task robot's execution of a task; wherein the global position information is a parameter used to characterize the spatial position in the global coordinate system of the working area; and the first global position information is a spatial coordinate parameter obtained by coordinate conversion based on the three-dimensional visual data acquired by the three-dimensional vision sensor; The robot travel control module is used to control each task robot to travel without collision in the working area based on the first global position information and the second global position information of each task robot.

11. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 9.

12. A control system for a task robot, characterized in that: include: At least one task robot; wherein each task robot is equipped with a three-dimensional vision sensor for scanning a local area on its driving path, wherein the local area covers the driving plane of the task robot; An electronic device for executing the task robot control method according to any one of claims 1 to 9.

Images