Mobile robot operation decision-making method and device based on large-scale simulation platform

Abstract

The present application provides a mobile robot operation decision-making method and device based on a large-scale simulation platform, belonging to the field of robotics. The method includes: determining the robot's mobility evaluation information based on point cloud information of a target scene and the robot's current position information; determining the robot's movement direction and movement step length based on the robot's mobility evaluation information, so as to control the robot's movement based on the movement direction and movement step length; in response to the robot moving to a specified position, determining the robot's operability evaluation information based on point cloud information of the target scene and the robot's current position information; and determining the robot's operating position for the target object based on the operability evaluation information. The mobile robot operation decision-making method and device based on a large-scale simulation platform provided in the present application can improve the robot's mobile operation capabilities.

Description

Technical Field

[0001] The present application belongs to the field of robotics technology, and more specifically, relates to a mobile robot operation decision-making method and device based on a large-scale simulation platform. Background Art

[0002] For robots to become ubiquitous in every household and indispensable assistants in human life, they must master the ability to manipulate objects in a variety of indoor environments. Given the complexities of indoor environments, robots must possess advanced mobile manipulation capabilities, including but not limited to navigating, identifying, and manipulating objects within confined spaces. This is undoubtedly a challenging research topic. Summary of the Invention

[0003] The purpose of this application is to provide a mobile robot operation decision-making method and device based on a large-scale simulation platform to improve the robot's mobile operation capability.

[0004] A first aspect of the embodiments of the present application provides a mobile robot operation decision-making method based on a large-scale simulation platform, comprising:

[0005] Determining mobility assessment information of the robot based on point cloud information of the target scene and current position information of the robot; the point cloud information of the target scene includes point cloud information of the target object and point cloud information of the obstruction in the target scene;

[0006] determining a moving direction and a moving step length of the robot based on the mobility evaluation information of the robot, so as to control the movement of the robot based on the moving direction and the moving step length;

[0007] In response to the robot moving to a designated position, determining the maneuverability evaluation information of the robot based on the point cloud information of the target scene and the current position information of the robot; the designated position is the position where the maneuverability evaluation information has the maximum value;

[0008] An operation position of the robot on the target object is determined based on the operability evaluation information.

[0009] A second aspect of the embodiments of the present application provides a mobile robot operation decision-making device based on a large-scale simulation platform, comprising:

[0010] a first evaluation module, configured to determine mobility evaluation information of the robot based on point cloud information of a target scene and current position information of the robot; the point cloud information of the target scene includes point cloud information of a target object and point cloud information of an obstruction in the target scene, and the current position information is used to indicate the current position of the robot;

[0011] a movement control module, configured to determine a movement direction and a movement step length of the robot based on the mobility evaluation information of the robot, so as to control the movement of the robot based on the movement direction and the movement step length;

[0012] a second evaluation module, configured to determine, in response to the robot moving to a designated position, operability evaluation information of the robot based on point cloud information of the target scene and current position information of the robot; the designated position being a position where the value of the operability evaluation information is the largest;

[0013] The operation control module determines an operation position of the robot on the target object based on the operability evaluation information.

[0014] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of the above-mentioned mobile robot operation decision method based on a large-scale simulation platform are implemented.

[0015] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned mobile robot operation decision-making method based on a large-scale simulation platform are implemented.

[0016] The beneficial effects of the mobile robot operation decision-making method and device based on a large-scale simulation platform provided by the embodiments of the present application are:

[0017] The embodiment of the present application adopts an innovative method, namely, based on a given target scene, quantifying the impact of target objects and obstructions on the movement of the robot at its current position, and obtaining mobility evaluation information. Based on the mobility evaluation information, the movement direction and movement step length of the robot can be determined, thereby providing the robot with accurate movement guidance and realizing efficient navigation of the robot.

[0018] On this basis, when the robot reaches the specified position, the robot's operability evaluation information is determined based on the point cloud information of the target scene and the robot's current position information. The operability evaluation information not only reflects the geometric and functional characteristics of the target object itself, but also comprehensively considers the impact of the robot's position and obstructions on the feasibility of the operation. Therefore, according to the operability evaluation information, an optimal object surface operation position can be determined, allowing the robot to operate the object more stably and improve the robot's mobile operation capability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 A flowchart of a mobile robot operation decision-making method based on a large-scale simulation platform provided in one embodiment of the present application;

[0021] Figure 2 A structural block diagram of a first neural network provided in one embodiment of the present application;

[0022] Figure 3 A visual diagram of a "mobile field" provided in one embodiment of the present application;

[0023] Figure 4 This is a structural block diagram of a mobile robot operation decision-making device based on a large-scale simulation platform provided in one embodiment of the present application;

[0024] Figure 5 A schematic block diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0026] In order to make the purpose, technical solutions and advantages of this application clearer, specific embodiments will be described below with reference to the accompanying drawings.

[0027] Please refer to Figure 1 , Figure 1 A flowchart of a mobile robot operation decision-making method based on a large-scale simulation platform provided in one embodiment of the present application is provided. The method includes:

[0028] S101: Determine mobility assessment information of the robot based on point cloud information of a target scene and current position information of the robot; the point cloud information of the target scene includes point cloud information of a target object and point cloud information of an obstruction in the target scene.

[0029] In this embodiment, the target object is a specific object that the robot needs to manipulate (e.g., grasp, carry, place, etc.) within the target scene. The target scene is the specific three-dimensional spatial environment in which the robot performs the manipulation task. The target scene typically includes the target object and obstructions that hinder the robot's movement. The target scene can be described using point cloud information, which includes point cloud information of the target object and obstructions.

[0030] By analyzing the point cloud information of the target object and the obstruction, combined with the robot's current position information, we can understand the environmental conditions around the robot, determine which areas are passable and which areas have obstacles, and thus obtain mobility assessment information (also known as "mobility field" information), providing a basic judgment basis for the robot's mobile control.

[0031] Specifically, the mobility evaluation information can be represented by a mobility field score, where a higher mobility field score indicates better mobility.

[0032] S102: Determine the movement direction and movement step length of the robot based on the mobility evaluation information of the robot, so as to perform movement control on the robot based on the movement direction and movement step length.

[0033] In this embodiment, based on mobility assessment information, the robot can determine which direction to move in to avoid obstacles and get closer to the target object. Furthermore, an appropriate step length ensures the robot can efficiently reach the target object while remaining safe. For example, if there's an open space ahead and mobility is good, a larger step length can be chosen for faster movement. However, if there are many obstacles around, the step length needs to be reduced to avoid collisions.

[0034] S103: In response to the robot moving to the designated position, determining the maneuverability evaluation information of the robot based on the point cloud information of the target scene and the current position information of the robot; the designated position is the position where the value of the maneuverability evaluation information is the largest.

[0035] In this embodiment, during the movement of the robot, mobility evaluation information (mobility field score) can be calculated in real time. When the mobility field score reaches the maximum value, it indicates that the robot has reached the optimal position (i.e., the designated position) for operating the target object.

[0036] After the robot reaches the designated location, the point cloud information of the target scene and the current position information of the robot can be used again to evaluate the feasibility of the robot's operation on the target object, such as whether it can accurately grasp the target object and whether there is enough space for operation.

[0037] Specifically, the operability evaluation information can be represented by an operability score. The higher the operability score, the higher the probability of successful operation.

[0038] S104: Determine the robot's operating position on the target object based on the operability evaluation information.

[0039] In this embodiment, once the robot reaches a designated location, it can use the operability evaluation information to determine the optimal operating position for the target object, helping the robot accurately perform its operation task. Specifically, the operating position on the target object surface with the highest operability score can be selected as the optimal operating position, thereby improving the success rate and accuracy of the robot's operation.

[0040] As can be seen from the above, the core challenge of robot mobility, as considered in the embodiments of this application, lies in how to skillfully circumvent various obstructions while navigating to the most optimal position for performing the operation. Traditional research often relies on a detailed understanding of the scene and then deriving the corresponding movement strategy through complex motion planning and control algorithms. This approach may work in theory, but it often struggles to achieve effective generalization in dynamic and unpredictable environments.

[0041] Therefore, the embodiment of the present application adopts an innovative method, namely, based on a given target scene, quantifying the impact of target objects and obstructions on the movement of the robot at its current position, and obtaining mobility evaluation information. According to the mobility evaluation information, the movement direction and movement step of the robot can be determined, thereby providing the robot with accurate movement guidance and realizing efficient navigation of the robot.

[0042] On this basis, when the robot reaches the specified position, the robot's operability evaluation information is determined based on the point cloud information of the target scene and the robot's current position information. The operability evaluation information not only reflects the geometric and functional characteristics of the target object itself, but also comprehensively considers the impact of the robot's position and obstructions on the feasibility of the operation. Therefore, according to the operability evaluation information, an optimal object surface operation position can be determined, allowing the robot to operate the object more stably and improve the robot's mobile operation capability.

[0043] In one embodiment of the present application, determining the mobility assessment information of the robot based on the point cloud information of the target scene and the current position information of the robot includes:

[0044] The point cloud information of the target object, the point cloud information of the obstruction and the current position information of the robot are input into the first neural network to obtain the mobility evaluation information of the robot.

[0045] In this embodiment, the point cloud information of the target object, the point cloud information of the obstruction, and the current position information of the robot are input into the first neural network to obtain the mobility evaluation information of the robot.

[0046] For details, please refer to Figure 2 , Figure 2 A schematic diagram of the structure of the first neural network provided in one embodiment of the present application. For the target object point cloud and the occluder point cloud, this embodiment uses two independent PointNet++ encoders to extract features, respectively, to obtain the features of the target object and the features of the occluder. PointNet++ is a deep learning model based on hierarchical point cloud processing that can effectively capture the geometric structure and contextual information of the point cloud through local feature aggregation and multi-scale learning. The two encoders do not share weights to ensure that the features of the target object and the occluder can be learned separately.

[0047] For the current position information of the robot, this embodiment uses a multi-layer perceptron (MLP) encoder to perform feature extraction to obtain the robot position features to capture the impact of the position on the operation.

[0048] The features of the target object, the features of the occluder, and the robot position are combined and input into the mobility predictor (which can be implemented using an MLP) to predict the mobility field score. The loss function uses the L2 loss (L2Loss), which is calculated as follows:

[0049]

[0050] Among them, M gt is the true moving field score of the current position, M pred is the moving field score of the current position predicted by the first neural network, and N is the number of samples.

[0051] In this embodiment, the “mobility field” visualization information can be obtained based on the mobility evaluation information output by the first neural network, such as Figure 3 As shown. Among them, Figure 3 In the figure, the area marked with “×” is the area with low moving field score, and the area marked with “ ” indicates areas with higher moving field scores.

[0052] From the above, it can be concluded that this embodiment integrates the effects of the target object and all obstructing objects, and the obtained mobility assessment information can provide accurate guidance for the robot's movement, ensuring its efficient navigation in complex spaces.

[0053] In one embodiment of the present application, the mobile robot operation decision-making method based on a large-scale simulation platform further includes: a step of training the first neural network, including:

[0054] Determine the target moving position information of the robot based on the point cloud information of the target object in the historical data;

[0055] determining a first movement field score based on current position information of the robot and target movement position information of the robot in historical data;

[0056] determining a second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data;

[0057] determining a third moving field score based on a difference between the first moving field score and the second moving field score;

[0058] determining the third moving field score as a true value corresponding to the output value of the first neural network;

[0059] Constructing sample data based on the current position information of the robot in the historical data, the point cloud information of the obstruction in the historical data, the point cloud information of the target object in the historical data, and the true value corresponding to the output value of the first neural network;

[0060] The first neural network is trained based on the sample data.

[0061] In this embodiment, historical data can be obtained by randomly placing target objects and obstructions in a simulation environment, executing movement operation tasks, and recording the operation results. Specifically, the data collection process collects the following information: the observed point cloud S of the current scene (i.e., the point cloud of the target scene, including the target object point cloud and the obstruction point cloud), the robot's current position R, and the true value M(x, y) of the "movement field" at this position. This data provides comprehensive input and supervision signals for subsequent model training. Each time data is collected, a target operation point (such as a point on a drawer of a table) is selected from the target object point cloud. Based on this point, a search is performed in the target scene to find a position that does not collide with other objects (including the target object and obstructions) and is as close as possible to the target operation point. This position is used as the robot's target movement position.

[0062] In this embodiment, the true value M(x, y) of the "moving field" at the current position comprehensively considers the influence of the target object and all obstructing objects. Specifically, the first moving field score can be determined based on the current position information of the robot and the target moving position information of the robot in the historical data, and the second moving field score can be determined based on the current position information of the robot in the historical data and the point cloud information of the obstruction in the historical data.

[0063] Based on this, we consider that the closer the robot is to the target movement position (the final position the robot needs to move to), the closer it is to the target object, and the higher the corresponding movement field score. The closer the robot is to an occlusion, the greater the occlusion's impact on the robot's movement, and the lower the corresponding movement field score. Therefore, the target object and the occlusion have opposite effects on the robot's movement. The difference between the first and second movement field scores is used as the third movement field score, which is the true value M(x, y) corresponding to the output of the first neural network.

[0064] Specifically, the third moving field score can be calculated using the second formula, which is:

[0065]

[0066] in, represents the third moving field score, represents the first moving field score, represents the second moving field score, Indicates the impact score of each occluder on the robot's movement.

[0067] On the basis of obtaining the true value corresponding to the output value of the first neural network, sample data can be constructed based on the current position information of the robot in the historical data, the point cloud information of the obstruction in the historical data, the point cloud information of the target object in the historical data and the true value corresponding to the output value of the first neural network, so as to train the first neural network based on the above sample data.

[0068] From the above, it can be concluded that when constructing the sample data of the first neural network, this embodiment comprehensively considers the impact of the target object and each blocking object on the movement of the robot, which is conducive to obtaining accurate true values, thereby improving the training accuracy of the first neural network.

[0069] In one embodiment of the present application, determining a first movement field score based on the current position information of the robot and the target movement position information of the robot in the historical data includes:

[0070] determining a first distance between the current position of the robot and the target movement position of the robot based on the current position information of the robot and the target movement position information of the robot in the historical data;

[0071] The first moving field score is determined based on a negative correlation between the first distance and the first moving field score.

[0072] This embodiment provides a specific implementation for calculating the first moving field score. First, based on the robot's current position information and target moving position information in historical data, a first distance between the robot's current position and the robot's target moving position is determined. Then, considering that a greater first distance between the robot's current position and the robot's target moving position indicates a greater distance from the target object, the first moving field score decreases. Therefore, based on the obtained first distance, the first moving field score can be determined based on the negative correlation between the first distance and the first moving field score.

[0073] Specifically, the first moving field score can be calculated using the third formula, which is:

[0074]

[0075] in, Indicates the first distance between the robot's current position and the robot's target movement position.

[0076] In one embodiment of the present application, determining the second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data includes:

[0077] Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data;

[0078] Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain a second motion field score;

[0079] The calculation process of the fourth moving field score corresponding to any occluder includes:

[0080] determining a second distance between the robot and any obstruction based on the point cloud information of any obstruction and the current position information of the robot in the historical data;

[0081] If the second distance between the robot and any obstruction is less than or equal to the radius of any obstruction, the fourth movement field score is calculated based on a negative correlation between the second distance and the fourth movement field score.

[0082] This embodiment provides a specific implementation method for calculating the second movement field score. First, the impact score of each obstruction on the robot's movement in the historical data is calculated, i.e., the fourth movement field score. Then, the fourth movement field scores corresponding to multiple obstructions are accumulated to obtain the second movement field score.

[0083] For any obstruction, the second distance between the robot and any obstruction is first calculated based on the point cloud information of any obstruction and the current position information of the robot in the historical data; then, considering that when the second distance between the robot and any obstruction is less than or equal to the radius of any obstruction, the obstruction has a greater impact on the robot's movement, and the smaller the second distance, the larger the fourth movement field score, therefore, the fourth movement field score can be calculated based on the negative correlation between the second distance and the fourth movement field score.

[0084] Specifically, the fourth moving field score can be calculated using the fourth formula, which is:

[0085]

[0086] in, represents the fourth movement field score (that is, the impact score of each occluder on the robot's movement), Represents the second distance between the robot and any obstruction, Indicates the radius of any occluder.

[0087] In one embodiment of the present application, determining the second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data includes:

[0088] Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data;

[0089] Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain a second motion field score;

[0090] The calculation process of the fourth moving field score corresponding to any occluder includes:

[0091] determining a second distance between the robot and any obstruction based on the point cloud information of any obstruction and the current position information of the robot in the historical data;

[0092] If the second distance between the robot and any obstruction is greater than the radius of any obstruction, the fourth moving field score is calculated based on a negative correlation between the radius of any obstruction and the fourth moving field score.

[0093] In this embodiment, if the second distance between the robot and any obstruction is greater than the radius of any obstruction, the robot is now outside the obstruction radius of the obstruction, and the obstruction has little impact on the robot's movement. At this time, the impact of the obstruction on the robot's movement can be regarded as a fixed value determined by the radius of the obstruction. Therefore, the fourth moving field score can be calculated based on the negative correlation between the radius of any obstruction and the fourth moving field score.

[0094] Specifically, the fourth moving field score can be calculated using the fifth formula, which is:

[0095] .

[0096] From the above, it can be concluded that this embodiment selects different calculation methods to calculate the fourth movement field score based on the relative size relationship between the second distance between the robot and any obstruction and the radius of any obstruction, which can more accurately characterize the impact of the obstruction on the robot movement.

[0097] In one embodiment of the present application, determining the movement direction and movement step length of the robot based on the mobility evaluation information of the robot includes:

[0098] Calculating a gradient value of the mobility evaluation information relative to the current position information of the robot;

[0099] The robot's moving direction and moving step length are determined based on the gradient value.

[0100] In this embodiment, by calculating the gradient value of the mobility assessment information relative to the current position information of the robot, the rate and direction of change of the mobility assessment information near the current position can be obtained.

[0101] The direction of the gradient indicates the direction of the fastest increase in mobility assessment information. Controlling the robot to move in this direction helps it more efficiently approach its target location or avoid obstructions. The magnitude of the gradient indicates the severity of the change in mobility assessment information. A larger gradient indicates a more pronounced change in mobility assessment information in the current direction, allowing the robot to adopt a larger step size to more quickly reach a location with greater mobility assessment information. Conversely, a smaller gradient value allows for a smaller step size to avoid over-movement and potentially falling into an unfavorable position.

[0102] Therefore, this embodiment dynamically adjusts the movement direction and movement step length of the robot based on the gradient value, which can ensure the movement efficiency of the robot while improving the safety and accuracy of the robot's movement.

[0103] In one embodiment of the present application, determining the movement direction and movement step length of the robot based on the gradient value includes:

[0104] The robot's moving direction and moving step length are determined based on the first formula, which is:

[0105]

[0106] in, Indicates the next moving position information of the robot, Indicates the current position information of the robot. Indicates the reference value of the moving step, Indicates the change in mobility assessment information, Indicates the change in the robot's current position information. Represents the gradient value of the mobility assessment information relative to the current position information of the robot. The direction of the gradient value indicates the movement direction of the robot. The size of dictates the robot's moving step size.

[0107] In this embodiment, the first formula above ensures that the robot moves in the optimal direction of its "movement field" with each step, enabling efficient navigation in complex environments. Furthermore, by dynamically adjusting its step length, the robot can flexibly respond to dynamic changes in the environment, improving its adaptability and operational efficiency.

[0108] In one embodiment of the present application, determining the operability evaluation information of the robot based on the point cloud information of the target scene and the current position information of the robot includes:

[0109] Determining a first occlusion vector based on point cloud information of the occluder in the target scene and current position information of the robot;

[0110] determining a second occlusion vector based on point cloud information of the occluder in the target scene and point cloud information of the target object;

[0111] calculating an occlusion field score based on the first occlusion vector and the second occlusion vector;

[0112] Selecting the occlusion area point cloud information from the point cloud information of the occluder based on the occlusion field score;

[0113] The point cloud information of the target object, the point cloud information of the occluded area and the current position information of the robot are input into the second neural network to obtain the operability evaluation information of the robot.

[0114] In this embodiment, occlusion is a common and complex challenge for robotic operation. Traditional methods often address this problem by inferring and recovering the shape of obstructing objects, but this approach can be computationally expensive and inflexible in dynamic environments. To more effectively address the object occlusion problem, this embodiment adopts a novel approach, focusing on obstructions that directly affect the robot's ability to manipulate the target object at its current location, starting from the perspective of robotic operation. This approach allows this embodiment to accurately extract the portion of the entire environment that has a substantial impact on the operation, which can be defined as an "occlusion field."

[0115] Specifically, the calculation of the “occlusion field” can be based on the concept of three-dimensional space, where Represents the current position of the robot To the target object position The "blocking field" (first occlusion vector) and (Second occlusion vector) represents the distance from the point on the occlusion object to the current position of the robot and the target object position The 3D vector can be used to calculate the "occlusion field" (occlusion field score) by vector cross product:

[0116]

[0117] Through the above calculation process, the impact of obstructions on the robot's operation can be quantified, providing the robot with accurate information about its operating environment.

[0118] Based on the above formula, we can know that for a point in three-dimensional space close to the robot position R and the target object position T, F R,T The closer (x, y, z) is to 0, the greater the occlusion effect on the robot operation, that is:

[0119]

[0120] Therefore, we can set a threshold to remove points that are unimportant to robot operation (points with large FR,T(x, y, z) values) and extract the points that are most important to robot operation as the occlusion area point cloud information. For example, we can sort the occlusion field scores corresponding to all points and select the 50 points with the lowest occlusion field scores as the occlusion area point cloud with the greatest impact on robot operation in the "occlusion field."

[0121] On this basis, the point cloud information of the target object, the point cloud information of the occluded area and the current position information of the robot are input into the second neural network to obtain the operability evaluation information of the robot.

[0122] A second neural network may be used to calculate the operability evaluation information of the robot, wherein the network structure of the second neural network is the same as the network structure of the first neural network in the above embodiment, and will not be repeated here.

[0123] At the same time, considering that the supervisory signal of the second neural network is the success or failure of the final operation result (the value is only 0 and 1), the loss function adopts the cross-entropy function (Cross-Entropy Loss), and its calculation formula is as follows:

[0124]

[0125] in, is the true label (1 for success and 0 for failure), is the operation success rate predicted by the model, and N is the number of samples.

[0126] Corresponding to the mobile robot operation decision-making method based on a large-scale simulation platform in the above embodiment, Figure 4 This is a structural block diagram of a mobile robot operation decision-making device based on a large-scale simulation platform provided in one embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 4 The mobile robot operation decision-making device 20 based on a large-scale simulation platform includes: a first evaluation module 21, a movement control module 22 and an operation control module 23.

[0127] The first evaluation module 21 is configured to determine mobility evaluation information of the robot based on point cloud information of the target scene and current position information of the robot; the point cloud information of the target scene includes point cloud information of the target object and point cloud information of the obstruction in the target scene, and the current position information is used to indicate the current position of the robot;

[0128] a movement control module 22, configured to determine a movement direction and a movement step length of the robot based on the mobility evaluation information of the robot, so as to control the movement of the robot based on the movement direction and the movement step length;

[0129] a second evaluation module 23 for determining, in response to the robot moving to a designated position, operability evaluation information of the robot based on the point cloud information of the target scene and the current position information of the robot; the designated position being the position where the value of the maneuverability evaluation information is the largest;

[0130] The operation control module 24 determines the operation position of the robot on the target object based on the operability evaluation information.

[0131] In one embodiment of the present application, the first evaluation module 21 is specifically configured to:

[0132] The point cloud information of the target object, the point cloud information of the obstruction and the current position information of the robot are input into the first neural network to obtain the mobility evaluation information of the robot.

[0133] In one embodiment of the present application, the first evaluation module 21 is further configured to:

[0134] Training the first neural network includes:

[0135] Determine the target moving position information of the robot based on the point cloud information of the target object in the historical data;

[0136] determining a first movement field score based on current position information of the robot and target movement position information of the robot in historical data;

[0137] determining a second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data;

[0138] determining a third moving field score based on a difference between the first moving field score and the second moving field score;

[0139] determining the third moving field score as a true value corresponding to the output value of the first neural network;

[0140] Constructing sample data based on the current position information of the robot in the historical data, the point cloud information of the obstruction in the historical data, the point cloud information of the target object in the historical data, and the true value corresponding to the output value of the first neural network;

[0141] The first neural network is trained based on the sample data.

[0142] In one embodiment of the present application, the first evaluation module 21 is further configured to:

[0143] determining a first distance between the current position of the robot and the target movement position of the robot based on the current position information of the robot and the target movement position information of the robot in the historical data;

[0144] The first moving field score is determined based on a negative correlation between the first distance and the first moving field score.

[0145] In one embodiment of the present application, the first evaluation module 21 is further configured to:

[0146] Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data;

[0147] Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain a second motion field score;

[0148] The calculation process of the fourth moving field score corresponding to any occluder includes:

[0149] determining a second distance between the robot and any obstruction based on the point cloud information of any obstruction and the current position information of the robot in the historical data;

[0150] If the second distance between the robot and any obstruction is less than or equal to the radius of any obstruction, the fourth movement field score is calculated based on a negative correlation between the second distance and the fourth movement field score.

[0151] In one embodiment of the present application, the first evaluation module 21 is further configured to:

[0152] Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data;

[0153] Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain a second motion field score;

[0154] The calculation process of the fourth moving field score corresponding to any occluder includes:

[0155] determining a second distance between the robot and any obstruction based on the point cloud information of any obstruction and the current position information of the robot in the historical data;

[0156] If the second distance between the robot and any obstruction is greater than the radius of any obstruction, the fourth moving field score is calculated based on a negative correlation between the radius of any obstruction and the fourth moving field score.

[0157] In one embodiment of the present application, the movement control module 22 is specifically configured to:

[0158] Calculating a gradient value of the mobility evaluation information relative to the current position information of the robot;

[0159] The robot's moving direction and moving step length are determined based on the gradient value.

[0160] In one embodiment of the present application, the movement control module 22 is further configured to:

[0161] The robot's moving direction and moving step length are determined based on the gradient value, including:

[0162] The robot's moving direction and moving step length are determined based on the first formula, which is:

[0163]

[0164] in, Indicates the next moving position information of the robot, Indicates the current position information of the robot. Indicates the reference value of the moving step, Indicates the change in mobility assessment information, Indicates the change in the robot's current position information. Represents the gradient value of the mobility assessment information relative to the current position information of the robot.

[0165] In one embodiment of the present application, the second evaluation module 23 is specifically configured to:

[0166] Determining a first occlusion vector based on point cloud information of the occlusion object in the target scene and current position information of the robot;

[0167] determining a second occlusion vector based on point cloud information of the occluder in the target scene and point cloud information of the target object;

[0168] calculating an occlusion field score based on the first occlusion vector and the second occlusion vector;

[0169] Selecting the occlusion area point cloud information from the point cloud information of the occluder based on the occlusion field score;

[0170] The point cloud information of the target object, the point cloud information of the occluded area and the current position information of the robot are input into the second neural network to obtain the operability evaluation information of the robot.

[0171] See also Figure 5 , Figure 5 This is a schematic block diagram of an electronic device provided in one embodiment of the present application. Figure 5 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. The processor 301 is configured to call the program instructions to execute the functions of the modules / units in the above-mentioned device embodiments, such as Figure 4 The functions of the first evaluation module 21 , the movement control module 22 , the second evaluation module 23 and the operation control module 24 are shown.

[0172] It should be understood that in the embodiment of the present application, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0173] The input device 302 may include a touchpad, a fingerprint collection sensor (for collecting user fingerprint information and fingerprint direction information), a microphone, etc. The output device 303 may include a display (LCD, etc.), a speaker, etc.

[0174] The memory 304 may include a read-only memory and a random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store device type information.

[0175] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present application can execute the implementation methods described in the first and second embodiments of the mobile robot operation decision method based on a large-scale simulation platform provided in the embodiments of the present application, and can also execute the implementation methods of the electronic device described in the embodiments of the present application, which will not be repeated here.

[0176] In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. The computer program includes program instructions. When the program instructions are executed by a processor, all or part of the process of the method in the above embodiment is implemented. The computer program can also be used to instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments are implemented. The computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium.

[0177] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc. Furthermore, the computer-readable storage medium can include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or is about to be output.

[0178] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0179] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or can be an electrical, mechanical or other form of connection.

[0181] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0182] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0183] The above are only specific embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A mobile robot operation decision-making method based on a large-scale simulation platform, characterized in that: include: Determining mobility assessment information of the robot based on point cloud information of the target scene and current position information of the robot; The point cloud information of the target scene includes point cloud information of the target object and point cloud information of the obstruction in the target scene; determining a moving direction and a moving step length of the robot based on the mobility evaluation information of the robot, so as to control the movement of the robot based on the moving direction and the moving step length; In response to the robot moving to a designated position, determining the maneuverability evaluation information of the robot based on the point cloud information of the target scene and the current position information of the robot; the designated position is the position where the maneuverability evaluation information has the maximum value; determining an operating position of the robot on the target object based on the operability evaluation information; The determining of the robot's mobility assessment information based on the point cloud information of the target scene and the current position information of the robot includes: Inputting the point cloud information of the target object, the point cloud information of the obstruction, and the current position information of the robot into a first neural network to obtain mobility evaluation information of the robot; The mobile robot operation decision-making method based on a large-scale simulation platform further includes: a step of training the first neural network, including: Determine the target moving position information of the robot based on the point cloud information of the target object in the historical data; determining a first movement field score based on current position information of the robot and target movement position information of the robot in historical data; determining a second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data; determining a third moving field score based on a difference between the first moving field score and the second moving field score; Determining the third moving field score as a true value corresponding to the output value of the first neural network; Constructing sample data based on the current position information of the robot in the historical data, the point cloud information of the obstruction in the historical data, the point cloud information of the target object in the historical data, and the true value corresponding to the output value of the first neural network; Training the first neural network based on the sample data; Determining a first movement field score based on current position information of the robot and target movement position information of the robot in historical data includes: determining a first distance between the current position of the robot and the target movement position of the robot based on the current position information of the robot and the target movement position information of the robot in the historical data; The first moving field score is determined based on a negative correlation between the first distance and the first moving field score.

2. The mobile robot operation decision-making method based on a large-scale simulation platform according to claim 1, characterized in that: The determining of the second movement field score based on the current position information of the robot in the historical data and the point cloud information of the obstruction in the historical data includes: Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data; Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain the second motion field score; The calculation process of the fourth moving field score corresponding to any occluder includes: Determining a second distance between the robot and any obstruction based on the point cloud information of the any obstruction and the current position information of the robot in the historical data; If the second distance between the robot and any one of the obstructions is less than or equal to the radius of any one of the obstructions, the fourth moving field score is calculated based on a negative correlation between the second distance and the fourth moving field score.

3. The mobile robot operation decision-making method based on a large-scale simulation platform according to claim 1, characterized in that: The determining of the second movement field score based on the current position information of the robot in the historical data and the point cloud information of the obstruction in the historical data includes: Calculating a fourth movement field score corresponding to each occluder based on the point cloud information of each occluder in the historical data and the current position information of the robot in the historical data; Accumulating the fourth motion field scores corresponding to the multiple occluders to obtain the second motion field score; The calculation process of the fourth moving field score corresponding to any occluder includes: Determining a second distance between the robot and any obstruction based on the point cloud information of the any obstruction and the current position information of the robot in the historical data; If the second distance between the robot and any one of the obstructions is greater than the radius of any one of the obstructions, the fourth moving field score is calculated based on a negative correlation between the radius of any one of the obstructions and the fourth moving field score.

4. The mobile robot operation decision-making method based on a large-scale simulation platform according to claim 1, characterized in that: The determining of the moving direction and moving step length of the robot based on the mobility evaluation information of the robot includes: Calculating a gradient value of the mobility assessment information relative to the current position information of the robot; The moving direction and moving step length of the robot are determined based on the gradient value.

5. The mobile robot operation decision-making method based on a large-scale simulation platform according to claim 4, characterized in that: Determining the movement direction and movement step length of the robot based on the gradient value includes: The moving direction and moving step length of the robot are determined based on a first formula, wherein the first formula is: in, Indicates the next moving position information of the robot, Indicates the current position information of the robot. Indicates the reference value of the moving step, Indicates the change in mobility assessment information, Indicates the change in the robot's current position information. Indicates the gradient value of the mobility assessment information relative to the current position information of the robot.

6. The mobile robot operation decision-making method based on a large-scale simulation platform according to claim 1, characterized in that: The determining of the robot's operability evaluation information based on the point cloud information of the target scene and the current position information of the robot includes: Determining a first occlusion vector based on point cloud information of the occlusion object in the target scene and current position information of the robot; determining a second occlusion vector based on point cloud information of the occluder in the target scene and point cloud information of the target object; calculating an occlusion field score based on the first occlusion vector and the second occlusion vector; Selecting occlusion area point cloud information from the point cloud information of the occluder based on the occlusion field score; The point cloud information of the target object, the point cloud information of the blocked area and the current position information of the robot are input into a second neural network to obtain the operability evaluation information of the robot.

7. A mobile robot operation decision-making device based on a large-scale simulation platform, characterized in that: include: A first evaluation module is configured to determine mobility evaluation information of the robot based on point cloud information of the target scene and current position information of the robot; The point cloud information of the target scene includes point cloud information of the target object and point cloud information of the obstruction in the target scene, and the current position information is used to indicate the current position of the robot; a movement control module, configured to determine a movement direction and a movement step length of the robot based on the mobility evaluation information of the robot, so as to control the movement of the robot based on the movement direction and the movement step length; a second evaluation module, configured to determine, in response to the robot moving to a designated position, operability evaluation information of the robot based on point cloud information of the target scene and current position information of the robot; the designated position being a position where the value of the operability evaluation information is the largest; an operation control module, which determines an operation position of the robot on the target object based on the operability evaluation information; The first evaluation module is specifically used to: Inputting the point cloud information of the target object, the point cloud information of the obstruction, and the current position information of the robot into the first neural network to obtain the mobility evaluation information of the robot; The first evaluation module is further configured to: Training the first neural network includes: Determine the target moving position information of the robot based on the point cloud information of the target object in the historical data; determining a first movement field score based on current position information of the robot and target movement position information of the robot in historical data; determining a second movement field score based on the current position information of the robot in the historical data and the point cloud information of the occluder in the historical data; determining a third moving field score based on a difference between the first moving field score and the second moving field score; determining the third moving field score as a true value corresponding to the output value of the first neural network; Constructing sample data based on the current position information of the robot in the historical data, the point cloud information of the obstruction in the historical data, the point cloud information of the target object in the historical data, and the true value corresponding to the output value of the first neural network; Training the first neural network based on the sample data; The first evaluation module is further configured to: determining a first distance between the current position of the robot and the target movement position of the robot based on the current position information of the robot and the target movement position information of the robot in the historical data; The first moving field score is determined based on a negative correlation between the first distance and the first moving field score.

Images