Method, device, storage medium and electronic device for controlling device movement

Abstract

The present disclosure relates to a method, device, storage medium and electronic device for controlling the movement of a device, and belongs to the field of device navigation technology. The method includes: in the process of the device moving along the initial driving path, detecting whether there is a first obstacle around the device; in the case of the first obstacle, obtaining a local target point through a global planner, the local target point is the target point on the initial driving path after the device bypasses the first obstacle; and controlling the device to move from the current position to the local target point. Using the method for controlling the movement of a device provided by the present disclosure, the global planner can be called to obtain the local target point only when the first obstacle exists, which reduces the number of times the global planner is called, shortens the navigation time, and improves the navigation speed.

Description

Technical Field

[0001] The present disclosure relates to the field of device navigation technology, and in particular, to a method, an apparatus, a storage medium and an electronic device for controlling the movement of a device. Background Art

[0002] With the continuous advancement of science and technology, automatic navigation technology for mobile devices such as unmanned vehicles and robots has gradually become a research hotspot. Navigation is to specify a path from the starting point to the destination point for mobile devices such as unmanned vehicles and robots to drive the mobile device to move from the starting point to the destination point.

[0003] At present, in the process of navigating mobile devices, reinforcement learning obstacle avoidance algorithm is used as a local planner to calculate the appropriate walking path of the mobile device in the local area. Because the rules of reinforcement learning obstacle avoidance algorithm have limitations, the mobile device needs to frequently call the global planner. The global planner needs to combine the current position of the mobile device, the position of the target point and the environment map to recalculate the new walking path to ensure that the robot walks on the correct walking path.

[0004] In this process, since the path planning process of the global planner is relatively complicated, frequent calls to the global planner will greatly increase the navigation time and slow down the navigation speed. Summary of the invention

[0005] The purpose of the present disclosure is to provide a method, apparatus, storage medium and electronic device for controlling the movement of a device to solve the above-mentioned related technical problems.

[0006] In order to achieve the above-mentioned object, a first aspect of an embodiment of the present disclosure provides a method for controlling movement of a device, the method comprising:

[0007] During the process of the device moving along the initial driving path, detecting whether there is a first obstacle around the device;

[0008] In the case where the first obstacle exists, a local target point is obtained through a global planner, where the local target point is a target point on the initial driving path after the device bypasses the first obstacle;

[0009] The device is controlled to move from a current position to the local target point.

[0010] Optionally, obtaining a local target point includes:

[0011] Get map information;

[0012] Acquire, according to the map information, a plurality of alternative paths that bypass the first obstacle;

[0013] From the multiple candidate paths, the target point of the candidate path with the shortest travel distance is used as the local target point.

[0014] Optionally, controlling the device to move from a current position to the local target point includes:

[0015] Determining multiple driving strategies for bypassing the second obstacle according to distribution information of the second obstacle around the device, coordinate information of the local target point, and current linear velocity and angular velocity of the device;

[0016] determining a target driving strategy from the plurality of driving strategies;

[0017] The device is controlled to move from the current position to the local target point using a target driving strategy.

[0018] Optionally, determining a target driving strategy from the multiple driving strategies includes:

[0019] Determining scores of the plurality of driving strategies based on the sparse reward and the dense reward;

[0020] The driving strategy with the highest score is used as the target driving strategy.

[0021] Optionally, determining the scores of the multiple driving strategies based on the sparse reward includes:

[0022] The scores of the plurality of driving strategies are determined according to whether the device successfully reaches the local target point.

[0023] Optionally, determining the scores of the plurality of driving strategies according to whether the device successfully reaches the local target point includes:

[0024] In the case where the device successfully reaches the local target point, increasing the score of the driving strategy;

[0025] When the device encounters the second obstacle, reducing the score of the driving strategy;

[0026] When the device fails to reach the local target point within a preset distance, the score of the driving strategy is reduced.

[0027] Optionally, determining the scores of the multiple driving strategies based on the dense reward includes:

[0028] Obtaining the distances between the last position and the current position of the device and the local target point, the angle between the line between the last position of the device and the local target point, the angle between the line between the current position and the local target point, and the angular velocity of the device at the current position;

[0029] A score of the driving strategy is determined according to the distance, the angle, and the angular velocity.

[0030] Optionally, determining the score of the driving strategy according to the distance, the angle, and the angular velocity includes:

[0031] When the distance between the previous position of the device and the local target point is greater than the distance between the current position of the device and the local target point, increasing the score of the driving strategy;

[0032] When the angle is an acute angle, increasing the score of the driving strategy;

[0033] When the angular velocity exceeds a threshold, the score of the driving strategy is reduced.

[0034] Optionally, the method further comprises:

[0035] In the absence of the obstacle, obtaining a global target point, wherein the global target point is an end point of the initial driving path;

[0036] The device is controlled to move from a current position to the global target point.

[0037] Optionally, the first obstacle is a static obstacle.

[0038] In order to achieve the above-mentioned object, a second aspect of an embodiment of the present disclosure provides an apparatus for controlling movement of a device, the apparatus comprising:

[0039] A detection module, used to detect whether there is a first obstacle around the device during the process of controlling the device to move along the initial driving path;

[0040] an acquisition module, configured to acquire a local target point through a global planner when the first obstacle exists, wherein the local target point is a target point on the initial driving path after the device bypasses the first obstacle;

[0041] The first control module is used to control the device to move from the current position to the local target point.

[0042] In order to achieve the above-mentioned purpose, the third aspect of the embodiments of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the first aspects of the embodiments of the present disclosure are implemented.

[0043] In order to achieve the above-mentioned purpose, a fourth aspect of an embodiment of the present disclosure provides an electronic device, including:

[0044] a memory having a computer program stored thereon;

[0045] A processor is used to execute the computer program in the memory to implement the steps of the method described in any one of the first aspects of the embodiments of the present disclosure.

[0046] Through the above technical solution, when there is a first obstacle around the device, the global planner is requested to obtain the local target point, and the device is controlled to return to the correct initial driving path according to the local target point. Compared with using the reinforcement learning obstacle avoidance algorithm as a local planner to call the global planner so that the device walks on the correct driving path, the present disclosure calls the global planner only when there is a first obstacle, reducing the number of times the global planner is called, thereby avoiding the problem of increased navigation time and slow navigation speed caused by frequent calls to the global planner, and improving the navigation time and navigation speed of the device navigation.

[0047] Furthermore, the present disclosure proposes to use a global planner to plan an initial driving path in advance. When the device encounters the first obstacle while moving along the initial driving path, the local planner will call the global planner to obtain the local target point that bypasses the first obstacle, and return to the initial driving path based on the local target point. In this way, every time the device encounters the first obstacle, the global planner will inform the local planner of the local target point that bypasses the first obstacle, and the local target point is located on the initial driving path, which will cause the device to eventually return to the initial driving path and travel along the initial driving path to the destination, and will not cause the device to fall into a navigation dead loop.

[0048] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the following specific embodiments, they are used to explain the present disclosure but do not constitute a limitation of the present disclosure. In the accompanying drawings:

[0050] Figure 1 is a flowchart of the steps of a method for controlling the movement of a device proposed by an exemplary embodiment of the present disclosure;

[0051] Figure 2 is a schematic diagram of bypassing a first obstacle proposed by an exemplary embodiment of the present disclosure;

[0052] Figure 3 is a flowchart of the steps of a method for controlling the movement of a device proposed by an exemplary embodiment of the present disclosure;

[0053] Figure 4 is a schematic diagram of determining a local target point proposed by an exemplary embodiment of the present disclosure;

[0054] Figure 5 is a schematic diagram for illustrating the cosine value of an angle proposed by an exemplary embodiment of the present disclosure;

[0055] Figure 6 is a block diagram of an apparatus for controlling movement of a device proposed by an exemplary embodiment of the present disclosure;

[0056] Figure 7 is a block diagram of an electronic device proposed by an exemplary embodiment of the present disclosure;

[0057] Figure 8 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0058] The specific implementation of the present disclosure is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described herein is only used to illustrate and explain the present disclosure, and is not used to limit the present disclosure.

[0059] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.

[0060] See also Figure 1 As shown, the present disclosure provides a method for controlling the movement of a device, the method comprising the following steps:

[0061] Step S11: When the device moves along the initial driving path, detecting whether there is a first obstacle around the device.

[0062] Among them, the device refers to mobile devices that can navigate automatically, such as robots and unmanned vehicles; the initial driving path refers to the straight line path initially planned by the device from the starting point to the destination point; the first obstacle refers to a static obstacle, such as walls, traffic lights and other static obstacles on the road; detection around the device refers to detection with the current position of the device as the center of the circle and the preset distance as the radius as the preset range.

[0063] Specifically, the global planner in the device will obtain map information between the starting point and the destination point, and plan a straight path from the starting point to the destination point based on the map information; when the device is driving along the straight path, the local planner in the device will use a lidar or depth camera to detect whether there is a static first obstacle within a preset range of the device.

[0064] Step S12: in the presence of the first obstacle, a local target point is obtained through a global planner, where the local target point is the target point on the initial driving path after the device bypasses the first obstacle.

[0065] Among them, in the presence of a first obstacle, in order to control the device to bypass the first obstacle, the local planner in the device needs to request the global planner to obtain the local target point. After the local planner obtains the local target point, it will control the device to bypass the first obstacle and return to the local target point on the initial driving path according to the location of the local target point.

[0066] The global planner refers to: making preliminary plans for the initial driving path of the entire route of the equipment, according to the map information pre-recorded by the equipment, combining the starting point location and the destination point location, and determining the straight line path from the starting point to the destination point on the map.

[0067] The local planner means that when the environment around the device changes during the actual movement of the device, and the initial driving path planned by the global planner is not conducive to the movement of the device, the local planner will make fine adjustments to enable the device to bypass the first obstacle and return to the initial driving path.

[0068] The local target point is the end point of the shorter driving path planned by the global planner for the device to bypass the first obstacle.

[0069] For example, see Figure 2 As shown in Figure 2, when the robot walks to the current position A, there is a large obstacle in front of the robot (at Figure 2 In order to avoid the obstacle, the local planner requests the global planner to obtain the local target point that avoids the obstacle. The global planner feeds back the local target point B to the local planner. The local planner plans a driving path that bypasses the obstacle based on the distribution information of the obstacle, the current position A, and the local target point B, and returns to the local target point B of the initial driving path.

[0070] Step S13: Control the device to move from the current position to the local target point.

[0071] Among them, after the local planner in the device obtains the local target point, it will plan a driving path that bypasses the first obstacle based on the distribution information of the first obstacle, the current position of the device and the local target point, and return to the local target point.

[0072] According to the method for controlling the movement of a device provided by the present disclosure, when there is a first obstacle around the device, the global planner is requested to obtain a local target point, and the device is controlled to return to the correct initial driving path according to the local target point. Compared with using the reinforcement learning obstacle avoidance algorithm as a local planner to call the global planner so that the device walks on the correct driving path, the present disclosure calls the global planner only when there is a first obstacle, reducing the number of times the global planner is called, thereby avoiding the problem of increased navigation time and slow navigation speed caused by frequent calls to the global planner, and improving the navigation time and navigation speed of the device navigation.

[0073] In addition, if the local planner is used to control the device from the starting point to the destination point during the movement of the device without calling the global planner, although the problems of increased navigation time and slow navigation speed caused by frequent calls to the global planner can be avoided, the local planner is better at bypassing obstacles and is not good at global path planning. If only the local planner is used to do global path planning for the device, it is easy to cause the device to fall into a navigation dead loop and fail to reach the final destination.

[0074] The present disclosure proposes that a global planner will be used to plan an initial driving path in advance. When the device encounters the first obstacle while moving along the initial driving path, the local planner will call the global planner to obtain the local target point that bypasses the first obstacle, and return to the initial driving path based on the local target point. In this way, every time the device encounters the first obstacle, the global planner will inform the local planner of the local target point that bypasses the first obstacle, and the local target point is located on the initial driving path, which will cause the device to eventually return to the initial driving path and travel along the initial driving path to the final destination, and will not cause the device to fall into a navigation dead loop.

[0075] In a possible implementation, when the device goes around the first obstacle from the current position and returns to the initial driving path, there may be multiple alternative paths, and the itinerary of each alternative path may be different. In order to shorten the time it takes for the device to go around the first obstacle, please refer to Figure 3 As shown, the present disclosure also includes the following steps:

[0076] Step S21: Obtain map information.

[0077] The map information is the current location of the device and map information around the first obstacle.

[0078] Step S22: Acquire multiple alternative paths that bypass the first obstacle based on the map information.

[0079] Step S23: From the multiple candidate paths, the target point of the candidate path with the shortest travel distance is used as the local target point.

[0080] Among them, before returning the local target point to the local planner, the global planner can obtain multiple alternative paths that bypass the first obstacle based on map information, and then select the target point of the alternative path with the shortest travel from the multiple alternative paths as the local target point based on the A* algorithm.

[0081] Specifically, the global planner can select multiple local target points on the initial driving path after bypassing the first obstacle, and then calculate the minimum cost estimate from the current position of the device to the multiple local target points based on the A* algorithm. Finally, from the multiple minimum cost estimates, the minimum cost estimate with the smallest value is selected, and the alternative path corresponding to the minimum cost estimate with the smallest value is used as the alternative path with the shortest travel.

[0082] For example, see Figure 4 As shown, the global planner bypasses the first obstacle (in Figure 4 Target points B, C, and D are selected on the initial driving path after the device reaches the destination (indicated by a four-pointed star in the figure). The global planner determines that there are a first alternative path 1 and a second alternative path 2 from the current position A of the device to the destination B, a third alternative path 3 and a fourth alternative path 4 from the current position A to the destination C, and a fifth alternative path 5 and a sixth alternative path 6 from the current position A to the destination D based on the map information. Then, based on the A* algorithm, the minimum cost estimates of the first alternative path 1, the second alternative path 2, the third alternative path 3, the fourth alternative path 4, the fifth alternative path 5, and the sixth alternative path 6 are calculated to be 0.1, 0.2, 0.3, 0.4, 0.5, and 0.6, respectively. From these six minimum cost estimates, the first alternative path 1 corresponding to the smallest value of 0.1 is selected as the alternative path with the shortest travel, and the target point B of the first alternative path 1 is used as the local target point to bypass the first obstacle this time.

[0083] Among them, the specific calculation formula of the A* algorithm is:

[0084] f*(n)=g*(n)+h*(n) (Formula 1)

[0085] In formula (1), f*(n) is the estimated minimum cost for the device to reach the destination point from the current position via the local target point; g*(n) is the estimated minimum cost for the device to reach the local target point from the current position; h*(n) is the estimated minimum cost for the device to reach the destination point from the local target point.

[0086] It can be seen from formula (1) that each candidate path has its own minimum cost estimate, and the shortest candidate path is the candidate path corresponding to the minimum cost estimate with the smallest value.

[0087] Through a method for controlling the movement of a device proposed in the present disclosure, when determining a local target point, the global controller can select a target point of an alternative path with the shortest travel distance from multiple alternative paths as the local target point, thereby shortening the time for the device to bypass the static first obstacle.

[0088] In a possible implementation, when the device moves from the current position to the local target point, it may encounter a second obstacle that has an area smaller than the preset area and / or is dynamic. In order to bypass the second obstacle, the present disclosure further includes the following steps:

[0089] Step S31: determining a plurality of driving strategies for bypassing the second obstacle according to the distribution information of the second obstacle around the device, the coordinate information of the local target point, and the current linear velocity and angular velocity of the device.

[0090] The second obstacle refers to an obstacle that is dynamic and / or occupies an area smaller than a preset area, such as a dynamically moving obstacle such as a person or a dog.

[0091] Specifically, since the second obstacle is a dynamic and / or small obstacle, the A* algorithm is usually used to plan a path for the static first obstacle to avoid the first obstacle, but it is impossible to accurately avoid dynamic and / or small obstacles. In order to accurately avoid the second obstacle, the PPO algorithm of reinforcement learning can be used to determine multiple driving strategies for bypassing the second obstacle based on the distribution information of the second obstacles distributed around the device, the coordinate information of the local target point, and the current linear speed and angular velocity of the device.

[0092] Among them, the PPO algorithm is one of the Actor-Critic reinforcement learning algorithms, which includes two parts: the Actor network and the Critic network. The Actor network uses a policy function to generate actions and interact with the environment; the Critic network uses a value function to evaluate the performance of device actions (such as learning the reward and punishment mechanism in subsequent sparse rewards and dense rewards) and guide the Actor network to take the next stage of action.

[0093] For example, the Critic network first learns the reward and punishment mechanism to know whether the action currently performed by the device increases or decreases the score; when the Actor network interacts with the environment and controls the device's actions, the Critic network tells the Actor network which actions increase the score and which actions decrease the score based on the pre-learned reward and punishment mechanism. In this way, the Critic network guides the Actor network to update the device's actions at each step in real time, and ultimately enables the device to bypass the second obstacle with a driving strategy that maximizes the reward.

[0094] The driving strategy for bypassing the second obstacle includes: the actions taken by the device to move from the current position to the local target point, such as the direction, angular velocity, and linear velocity that the device needs to travel. More specifically, when the device is a robot, it also includes the robot's posture, etc.; it also includes the score of each action of the device, and uses the score to regulate the device's actions.

[0095] Step S32: Determine a target driving strategy from the multiple driving strategies.

[0096] Among the multiple driving strategies obtained based on the PPO algorithm, each driving strategy has its own expected reward value, and the driving strategy with the largest expected reward value is used as the target driving strategy.

[0097] The target driving strategy is a better driving strategy among multiple driving strategies. In the process of moving from the current position to the local target point by adopting the target driving strategy, adopting the target driving strategy can avoid obstacles more quickly, thereby quickly moving to the local target point.

[0098] Step S33: Control the device to move from the current position to the local target point according to the target driving strategy.

[0099] Among them, in the process of controlling the device to move from the current position to the local target point, the reward and punishment rules can be added as the loss function to modify the PPO algorithm, so that the Critic network in the PPO algorithm evaluates the performance of the device action based on the reward and punishment rules. The reward and punishment rules are the evaluation criteria for regulating the good or bad behavior of the device selection, including:

[0100] Based on the sparse reward and the dense reward, the scores of the multiple driving strategies are determined, and the driving strategy with the highest score is used as the target driving strategy.

[0101] In this step, the scores of multiple driving strategies are determined based on sparse rewards: the scores of the multiple driving strategies are determined according to whether the device successfully reaches the local target point. The sparse reward is the score obtained by the device after completing the driving strategy from the current position to the local target point.

[0102] Specifically, when the device successfully reaches the local target point, the score of the driving strategy is increased; when the device encounters the second obstacle, the score of the driving strategy is reduced; when the device fails to reach the local target point within a preset distance, the score of the driving strategy is reduced.

[0103] The preset distance is set according to the type of device. For example, when the device is a robot, the preset distance can be a preset number of steps (e.g., 1,000 steps); when the device is an unmanned vehicle, the preset distance can be a preset moving distance (e.g., 500 m).

[0104] For example, after each execution of the driving strategy for bypassing the second obstacle, if the device successfully reaches the local target point, 15 points are added to inform the device that the local target point is an important task; if the device encounters the second obstacle, 15 points are subtracted to inform the device that avoiding the second obstacle is an important task, and the device is trained to choose to bypass a similar second obstacle next time; if the device does not reach the local target point within the preset distance, 5 points are subtracted to inform the device that it should reach the local target point by a shorter path.

[0105] In this process, each different task has a different score, which is determined according to the importance of the task. For example, the importance of reaching the local target point and avoiding the second obstacle is high. Therefore, when the device successfully reaches the target point or collides with the second obstacle, the score increases or decreases more. The importance of reaching the local target point within the preset distance is low. Therefore, when the local target point is not reached within the preset distance, the score decreases less.

[0106] By setting sparse rewards, you can reward the device for successfully completing the set goal. If the device successfully reaches the local target point or successfully avoids obstacles, the score will be increased to increase the probability of the device successfully reaching the local target point without encountering a second obstacle. If the device reaches the local target point within the preset distance, the score will be increased to encourage the device to reach the local target point with a shorter driving path.

[0107] In this step, the scores of multiple driving strategies can be determined based on dense rewards as follows: obtaining the distances between the previous position and current position of the device and the local target point, the line between the previous position of the device and the local target point, the angle between the line between the current position and the local target point, and the angular velocity of the device at the current position; determining the score of the driving strategy based on the distance, the angle, and the angular velocity, wherein the dense reward is the score obtained by the device according to the execution action of the device in the process of executing the driving strategy from the current position to the local target point.

[0108] Specifically, when the distance between the previous position of the device and the local target point is greater than the distance between the current position of the device and the local target point, it indicates that the device is approaching the local target point, so the score of the driving strategy can be increased to inform the device of the importance of shortening the distance to the local target point; when the cosine value of the angle is negative, it also indicates that the device is approaching the local target point, so the score of the driving strategy can be increased to further inform the device of the importance of shortening the distance to the local target point; when the angular velocity exceeds the threshold, it indicates that the device is at risk of tipping over, so the score of the driving strategy can be reduced.

[0109] The previous position varies according to the type of device. For example, when the device is a robot, the previous position may be the position of the robot in the previous step; when the device is an unmanned vehicle, the previous position may be the position of the unmanned vehicle in the previous second.

[0110] For example, if the device is a robot, when the distance from the robot's previous position to the target local point is greater than the distance from the robot's current position to the local target point, it means that the robot is moving towards the local target point. At this time, 5 points can be added to inform the robot that it is more important to shorten the distance to the local target point.

[0111] For example, see Figure 5 As shown in the figure, the line between the robot's previous position m and current position n, and the line between the robot's current position n and the local target point h 1 When the cosine value of the angle between the robot and the local target point is greater than 0, the robot is approaching the target point, and the cosine value of the angle is used as the added score to tell the robot that it is more important to shorten the distance to the local target point; and the line between the robot's previous position m and the current position n, and the line between the robot's current position n and the local target point h 2 If the cosine value of the angle between the lines is less than 0, it means that the robot is moving away from the destination, and the score does not need to be increased.

[0112] For example, if the current angular velocity of the device exceeds the threshold, it means that the device is turning too fast and there may be a risk of tipping over. In this case, 10 points can be reduced to inform the device of the importance of stable driving.

[0113] By setting dense rewards, the execution actions of the device during driving can be standardized. When the distance between the device and the local target point is shortened, the score is increased, which can prompt the device to reach the local target point faster; when the angular velocity of the device exceeds the threshold, the score is reduced, which can prompt the device to reach the local target point more stably.

[0114] In a possible implementation, when there is no first obstacle, there is no need to avoid the obstacle, so there is no need to call the global planner to obtain the local target point. The global target point can be directly determined from the initial driving path preliminarily planned by the global planner, which specifically includes the following steps:

[0115] Step S41: In the absence of the first obstacle, a global target point is obtained, where the global target point is the destination point of the initial driving path.

[0116] In the absence of the first obstacle, it means that the device can directly drive to the destination point along the straight initial driving path without obstacle avoidance. Then, after obtaining the global target point, the device can be controlled to move from the current position to the global target point to complete the initial driving path.

[0117] Step S42: Control the device to move from the current position to the global target point.

[0118] See also Figure 6 As shown, the present disclosure also provides an apparatus for controlling the movement of a device, the apparatus 1300 comprising:

[0119] A detection module 1301 is used to detect whether there is a first obstacle around the device during the process of controlling the device to move along the initial driving path;

[0120] An acquisition module 1302 is used to acquire a local target point through a global planner when the first obstacle exists, where the local target point is a target point on the initial driving path after the device bypasses the first obstacle;

[0121] The first control module 1303 is used to control the device to move from the current position to the local target point.

[0122] Optionally, the acquisition module 1302 includes:

[0123] A map information acquisition submodule is used to acquire map information;

[0124] an alternative path determination submodule, configured to obtain a plurality of alternative paths that bypass the first obstacle according to the map information;

[0125] The local target point determination submodule is used to select the target point of the candidate path with the shortest travel distance from the multiple candidate paths as the local target point.

[0126] Optionally, the first control module 1303 includes:

[0127] A plurality of driving strategy determination submodules, configured to determine a plurality of driving strategies for bypassing the second obstacle according to the distribution information of the second obstacle around the device, the coordinate information of the local target point, and the current linear velocity and angular velocity of the device;

[0128] A first target driving strategy determination submodule, configured to determine a target driving strategy from the plurality of driving strategies;

[0129] The control submodule is used to control the device to move from the current position to the local target point according to the target driving strategy.

[0130] Optionally, the target driving strategy determination submodule includes:

[0131] A first score determination submodule, configured to determine scores of the plurality of driving strategies based on the sparse reward and the dense reward;

[0132] The second target driving strategy determination submodule is used to use the driving strategy with the highest score as the target driving strategy.

[0133] Optionally, the first score determination submodule includes:

[0134] The second score determination submodule is used to determine the scores of the multiple driving strategies according to whether the device successfully reaches the local target point.

[0135] Optionally, the second score determination submodule includes:

[0136] A first score increasing submodule, configured to increase the score of the driving strategy when the device successfully reaches the local target point;

[0137] a first score reduction submodule, configured to reduce the score of the driving strategy when the device encounters the second obstacle;

[0138] The second score reduction submodule is used to reduce the score of the driving strategy when the device fails to reach the local target point within a preset distance.

[0139] Optionally, the first score determination submodule includes:

[0140] The device status information acquisition submodule is used to obtain the distances between the last position and the current position of the device and the local target point, the cosine value of the angle between the line between the last position of the device and the local target point, and the line between the current position and the local target point, and the angular velocity of the device at the current position;

[0141] The third score determination submodule is used to determine the score of the driving strategy according to the distance, the angle and the angular velocity.

[0142] Optionally, the third score determination submodule includes:

[0143] A second score increasing submodule, configured to increase the score of the driving strategy when the distance between the previous position of the device and the local target point is greater than the distance between the current position of the device and the local target point;

[0144] A third score increasing submodule, configured to increase the score of the driving strategy when the cosine value of the angle is a negative value;

[0145] The third score reduction submodule is used to reduce the score of the driving strategy when the angular velocity exceeds a threshold.

[0146] Optionally, the device 1300 further includes:

[0147] A global target point acquisition module, used for acquiring a global target point when the first obstacle does not exist, the global target point being the destination point of the initial driving path;

[0148] The second control module is used to control the device to move from the current position to the global target point.

[0149] Optionally, the first obstacle is a static obstacle.

[0150] The present disclosure also provides a non-temporary computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for controlling the movement of a device described in any of the above embodiments.

[0151] The present disclosure also provides an electronic device, comprising:

[0152] a memory having a computer program stored thereon;

[0153] A processor is used to execute the computer program in the memory to implement the steps of the method for controlling the movement of a device described in any of the above embodiments.

[0154] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0155] Figure 7 FIG. 7 is a block diagram of an electronic device 700 according to an exemplary embodiment. Figure 7As shown, the electronic device 700 may include: a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0156] The processor 701 is used to control the overall operation of the electronic device 700 to complete all or part of the steps in the above-mentioned method for controlling the movement of the device. The memory 702 is used to store various types of data to support the operation of the electronic device 700, and these data may include, for example, instructions for any application or method used to operate on the electronic device 700, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Read-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, magnetic disk or optical disk. The multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 702 or sent through the communication component 705. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 704 provides an interface between the processor 701 and other interface modules, and the other interface modules may be keyboards, mice, buttons, etc. These buttons may be virtual buttons or physical buttons. The communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, is not limited here. Therefore, the corresponding communication component 705 may include: Wi-Fi module, Bluetooth module, NFC module, etc.

[0157] In an exemplary embodiment, the electronic device 700 can be implemented by one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned method of controlling the movement of the device.

[0158] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for controlling the movement of a device are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 702 including program instructions, and the above-mentioned program instructions may be executed by the processor 701 of the electronic device 700 to complete the above-mentioned method for controlling the movement of a device.

[0159] Figure 8 1 is a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Figure 8 The electronic device 1900 includes a processor 1922, which may be one or more, and a memory 1932 for storing a computer program executable by the processor 1922. The computer program stored in the memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processor 1922 may be configured to execute the computer program to perform the above-mentioned method for controlling the movement of the device.

[0160] In addition, the electronic device 1900 may further include a power supply component 1926 and a communication component 1950, wherein the power supply component 1926 may be configured to perform power management of the electronic device 1900, and the communication component 1950 may be configured to implement communication of the electronic device 1900, for example, wired or wireless communication. In addition, the electronic device 1900 may further include an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as Windows Server™, Mac OS X™, Unix™, Linux™, etc.

[0161] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, and when the program instructions are executed by a processor, the steps of the above-mentioned method for controlling the movement of a device are implemented. For example, the computer-readable storage medium may be the above-mentioned memory 1932 including program instructions, and the above-mentioned program instructions may be executed by the processor 1922 of the electronic device 1900 to complete the above-mentioned method for controlling the movement of a device.

[0162] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program executable by a programmable device, and has a code portion for executing the above-mentioned method of controlling the movement of a device when the computer program is executed by the programmable device.

[0163] The preferred embodiments of the present disclosure are described in detail above in conjunction with the accompanying drawings; however, the present disclosure is not limited to the specific details in the above embodiments. Within the technical concept of the present disclosure, a variety of simple modifications can be made to the technical solution of the present disclosure, and these simple modifications all fall within the protection scope of the present disclosure.

[0164] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present disclosure will not further describe various possible combinations.

[0165] In addition, various embodiments of the present disclosure may be arbitrarily combined, and as long as they do not violate the concept of the present disclosure, they should also be regarded as the contents disclosed by the present disclosure.

Claims

1. A method for controlling the movement of a device, characterized in that: The method comprises: During the process of the device moving along the initial driving path, detecting whether there is a first obstacle around the device; In the case where the first obstacle exists, a local target point is obtained through a global planner, where the local target point is a target point on the initial driving path after the device bypasses the first obstacle; Controlling the device to move from a current position to the local target point; The controlling the device to move from the current position to the local target point includes: Determining multiple driving strategies for bypassing the second obstacle according to distribution information of the second obstacle around the device, coordinate information of the local target point, and current linear velocity and angular velocity of the device; Determining scores of the plurality of driving strategies based on the sparse reward and the dense reward; Using the driving strategy with the highest score as the target driving strategy; Controlling the device to move from the current position to the local target point using a target driving strategy; Wherein, determining the scores of the plurality of driving strategies based on the sparse reward and the dense reward includes: When the distance between the previous position of the device and the local target point is greater than the distance between the current position and the local target point, increasing the score of the driving strategy; When the cosine value of the angle between the current position and the local target point is negative, increasing the score of the driving strategy; When the angular velocity of the current position exceeds a threshold, the score of the driving strategy is reduced.

2. The method for controlling the movement of a device according to claim 1, characterized in that: The obtaining of the local target point comprises: Get map information; Acquire, according to the map information, a plurality of alternative paths that bypass the first obstacle; From the multiple candidate paths, the target point of the candidate path with the shortest travel distance is used as the local target point.

3. The method for controlling the movement of a device according to claim 1, characterized in that: The step of determining the scores of the plurality of driving strategies based on the sparse reward includes: The scores of the plurality of driving strategies are determined according to whether the device successfully reaches the local target point.

4. The method for controlling the movement of a device according to claim 3, characterized in that: The step of determining the scores of the plurality of driving strategies according to whether the device successfully reaches the local target point comprises: In the case where the device successfully reaches the local target point, increasing the score of the driving strategy; When the device encounters the second obstacle, reducing the score of the driving strategy; When the device fails to reach the local target point within a preset distance, the score of the driving strategy is reduced.

5. The method for controlling the movement of a device according to claim 1, characterized in that: The method further comprises: In the absence of the first obstacle, obtaining a global target point, the global target point being the destination point of the initial driving path; The device is controlled to move from a current position to the global target point.

6. The method for controlling the movement of a device according to any one of claims 1 to 5, characterized in that: The first obstacle is a static obstacle.

7. A device for controlling the movement of a device, characterized in that: The device comprises: A detection module, used to detect whether there is a first obstacle around the device during the process of controlling the device to move along the initial driving path; an acquisition module, configured to acquire a local target point through a global planner when the first obstacle exists, wherein the local target point is a target point on the initial driving path after the device bypasses the first obstacle; A first control module, used for controlling the device to move from a current position to the local target point; The first control module is further configured to determine a plurality of driving strategies for bypassing the second obstacle according to the distribution information of the second obstacle around the device, the coordinate information of the local target point, and the current linear velocity and angular velocity of the device; Determining scores of the plurality of driving strategies based on the sparse reward and the dense reward; Using the driving strategy with the highest score as the target driving strategy; Controlling the device to move from the current position to the local target point using a target driving strategy; Wherein, determining the scores of the plurality of driving strategies based on the sparse reward and the dense reward includes: When the distance between the previous position of the device and the local target point is greater than the distance between the current position and the local target point, increasing the score of the driving strategy; When the cosine value of the angle between the current position and the local target point is negative, increasing the score of the driving strategy; When the angular velocity of the current position exceeds a threshold, the score of the driving strategy is reduced.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are implemented.

9. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 6.

Images