Robot control method, device, electronic device and computer readable storage medium

Abstract

The present disclosure provides a robot control method, device, electronic device and computer-readable storage medium. The method includes: receiving first image data and state data sent by the robot; generating a first control strategy corresponding to the first image data and state data; simulating the first control strategy to obtain a first simulation result; when the first simulation result is reaching the destination, sending the first control strategy to the robot so that the robot executes the first control strategy. By simulating the first control strategy, the feasibility and safety of the first control strategy can be verified, thereby avoiding sending an infeasible or unsafe control strategy to the robot for execution. When the simulation passes, the feasible and safe first control is sent to the robot and executed by the robot, which can reduce the possibility of safety incidents such as collision when the controlled robot executes the first control strategy, and improve the safety of controlling the robot when moving.

Description

Background Art

[0002] With the development of robotics technology, using robots to replace humans to complete various complex and dangerous tasks has become an important direction of robotic applications. However, how to control robots intelligently, safely and efficiently is still an important issue hindering the application of robots.

[0003] In the related art, visual servo technology is used to control the navigation of the robot. In this way, the safety of controlling the robot is low, and it is easy to cause the controlled robot to collide with other safety incidents during the movement, thereby causing economic losses.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a robot control method, device, electronic device and computer-readable storage medium, which at least to some extent overcome the problem of low safety when controlling the movement of a robot in the related art.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a robot control method is provided, including: receiving first image data and status data sent by a robot; generating a first control strategy corresponding to the first image data and status data; simulating the first control strategy to obtain a first simulation result; and when the first simulation result is reaching a destination, sending the first control strategy to the robot so that the robot executes the first control strategy.

[0008] In one embodiment of the present disclosure, the generation of a first control strategy corresponding to the first image data and status data includes: acquiring second image data of a detection model and a reference object, the reference object being used to represent a destination of the robot; inputting the first image data and the second image data into the detection model, and performing detection in the first image data using the detection model as a target with the reference object corresponding to the second image data as a target to obtain a detection result; and generating a first control strategy corresponding to the first image data and status data when the detection result is that the reference object is included in the first image data.

[0009] In one embodiment of the present disclosure, generating a first control strategy corresponding to the first image data and status data includes: acquiring a first path generation model; generating a navigation path corresponding to the first image data according to the first path generation model; and generating a first control strategy corresponding to the navigation path according to the status data.

[0010] In one embodiment of the present disclosure, the simulating the first control strategy to obtain a first simulation result includes: obtaining one or more simulation models, wherein the one or more simulation models have constructed an environmental model of the environment in which the robot is located; inputting the first control strategy into the one or more simulation models, and the one or more simulation models simulating the first control strategy according to the environmental model to obtain one or more first simulation results.

[0011] In one embodiment of the present disclosure, the one or more first simulation results are multiple first simulation results; and when the first simulation result is reaching the destination, sending the first control strategy to the robot includes: when a first simulation result greater than or equal to a predetermined proportion of the multiple first simulation results is reaching the destination, sending the first control strategy to the robot.

[0012] In one embodiment of the present disclosure, it also includes: when the first simulation result is that the destination has not been reached, the simulation data generated during the simulation of the first control strategy is input into a first path generation model, and the first path generation model adjusts parameters according to the simulation data to obtain a second path generation model; according to the second path generation model, a second control strategy corresponding to the first image data and the state data is generated; the second control strategy is simulated to obtain a second simulation result; when the second simulation result is that the destination has been reached, the second control strategy is sent to the robot so that the robot executes the second control strategy.

[0013] In one embodiment of the present disclosure, it also includes: when the simulation results corresponding to the control strategy generated according to the first image data and state data for n consecutive times all fail to reach the destination, marking the first image data as a singular image; wherein the singular image is used to indicate that the control strategy generated according to the first image data for n consecutive times did not enable the robot to reach the destination in the simulation; n is an integer greater than or equal to 2.

[0014] In one embodiment of the present disclosure, the robot is a wheeled robot.

[0015] According to another aspect of the present disclosure, a robot control device is provided, including: a receiving module for receiving first image data and status data sent by a robot; a generating module for generating a first control strategy corresponding to the first image data and status data; a simulation module for simulating the first control strategy to obtain a first simulation result; and a sending module for sending the first control strategy to the robot when the first simulation result is reaching a destination, so that the robot executes the first control strategy.

[0016] In one embodiment of the present disclosure, the generation module is used to obtain a detection model and second image data of a reference object, wherein the reference object is used to represent the destination of the robot; the first image data and the second image data are input into the detection model, and the detection model is used to perform detection in the first image data with the reference object corresponding to the second image data as the target to obtain a detection result; and when the detection result is that the reference object is included in the first image data, a first control strategy corresponding to the first image data and the state data is generated.

[0017] In one embodiment of the present disclosure, the generation module is used to obtain a first path generation model; generate a navigation path corresponding to the first image data according to the first path generation model; and generate a first control strategy corresponding to the navigation path according to the state data.

[0018] In one embodiment of the present disclosure, the simulation module is used to obtain one or more simulation models, in which the environmental model of the environment in which the robot is located has been constructed; the first control strategy is input into the one or more simulation models, and the one or more simulation models simulate the first control strategy according to the environmental model to obtain one or more first simulation results.

[0019] In one embodiment of the present disclosure, the one or more first simulation results are multiple first simulation results; the sending module is used to send the first control strategy to the robot when a first simulation result greater than or equal to a predetermined proportion of the multiple first simulation results reaches the destination.

[0020] In one embodiment of the present disclosure, the device also includes: an adjustment module, which is used to input the simulation data generated during the simulation of the first control strategy into the first path generation model when the first simulation result is that the destination has not been reached, and the first path generation model adjusts parameters according to the simulation data to obtain a second path generation model; the generation module is also used to generate a second control strategy corresponding to the first image data and the state data according to the second path generation model; the simulation module is also used to simulate the second control strategy to obtain a second simulation result; the sending module is also used to send the second control strategy to the robot when the second simulation result is that the destination has been reached, so that the robot can execute the second control strategy.

[0021] In one embodiment of the present disclosure, the device also includes: a calibration module, which is used to mark the first image data as a singular image when the simulation results corresponding to the control strategy generated according to the first image data and state data for n consecutive times all fail to reach the destination; wherein the singular image is used to indicate that the control strategy generated according to the first image data for n consecutive times did not enable the robot to reach the destination in the simulation; n is an integer greater than or equal to 2.

[0022] In one embodiment of the present disclosure, the robot is a wheeled robot.

[0023] According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any of the above-mentioned robot control methods by executing the executable instructions.

[0024] According to another aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements any of the above-mentioned robot control methods when executed by a processor.

[0025] According to another aspect of the present disclosure, a computer program product is provided, wherein the computer program product comprises a computer program or computer instructions, wherein the computer program or the computer instructions are loaded and executed by a processor so as to enable a computer to implement any of the above-mentioned robot control methods.

[0026] The technical solution provided by the embodiments of the present disclosure includes at least the following beneficial effects:

[0027] The technical solution provided by the embodiment of the present disclosure uses the received first image data and state data to generate a first control strategy for controlling the robot, and then simulates the first control strategy to obtain a first simulation result. By simulating the first control strategy, the feasibility and safety of the first control strategy can be verified, thereby avoiding sending an infeasible or unsafe control strategy to the robot for execution. When the first simulation result shows that the robot can reach the destination, it is determined that the first control strategy is feasible and safe, and the feasible and safe first control is sent to the robot and executed by the robot, which can reduce the possibility of safety incidents such as collisions when the controlled robot executes the first control strategy, thereby reducing the possibility of economic losses and improving the safety of controlling the robot when it moves.

[0028] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0030] Figure 1 A schematic diagram showing a robot control system in an embodiment of the present disclosure;

[0031] Figure 2 A flow chart of a robot control method in one embodiment of the present disclosure is shown;

[0032] Figure 3 A flow chart of a first control strategy generating method in one embodiment of the present disclosure is shown;

[0033] Figure 4 A flow chart of a robot control method in another embodiment of the present disclosure is shown;

[0034] Figure 5 A signaling diagram showing a robot control method in another embodiment of the present disclosure;

[0035] Figure 6 A schematic diagram of a robot control device in an embodiment of the present disclosure is shown;

[0036] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0039] With the development of robotics technology, robots are increasingly integrated into people's production and life. In the field of robotics, how to efficiently navigate mobile robots is a hot topic at present, and faces many difficulties, such as the cost issues brought about by sensors such as binocular cameras, 3D (3-Dimensions) cameras and lidars, or the loss of industry competitiveness due to cost increases. In addition, the problem of insufficient safety of mobile routes can easily lead to collisions when robots are moving, resulting in economic losses. In this regard, the embodiments of the present disclosure provide a robot control method that can effectively improve the accuracy, efficiency and safety of robot navigation.

[0040] Figure 1 A schematic diagram of a robot control system in one embodiment of the present disclosure is shown, and the system can apply the robot control method or robot control device in various embodiments of the present disclosure.

[0041] like Figure 1 As shown, the robot control system may include: a robot 101 and a server 102 .

[0042] Among them, the robot 101 is equipped with a camera device, and can obtain image data of the surrounding environment based on the camera device. The robot 101 can also obtain state data representing its current posture, speed, acceleration, direction of travel, angular acceleration and other states from various sensors configured by itself. The robot 101 can send the image data and state data to the server 102. The server 102 can receive the image data and state data sent by the robot 101, and generate a control strategy for controlling the robot based on the image data and state data. The server 102 can also use the simulation model to simulate and verify the control strategy, and send the control strategy to the robot 101 when the simulation result shows that the destination can be reached. The robot 101 can receive and execute the control strategy.

[0043] The robot 101 and the server 102 are connected to each other through a network, which can be a wired network or a wireless network.

[0044] Optionally, the wireless network or wired network described above uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0045] The robot 101 may be a wheeled robot or other types of robots, such as a tracked robot, a legged robot, etc., which is not limited in the embodiments of the present disclosure.

[0046] The server 102 may be a server that provides various services, such as a background management server, which may analyze and process the received request and other data, and feed back the processing results to the robot 101. Optionally, the server 102 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0047] Those skilled in the art will know that Figure 1 The number of robots 101 and servers 102 in the figure is only for illustration, and any number of robots 101 and servers 102 may be provided according to actual needs. The embodiments of the present disclosure are not limited to this.

[0048] The present exemplary implementation is described in detail below with reference to the accompanying drawings and embodiments.

[0049] The present disclosure provides a robot control method, which can be executed by any electronic device with computing and processing capabilities. For example, the electronic device is a server.

[0050] Figure 2 A flow chart of a robot control method in an embodiment of the present disclosure is shown as follows: Figure 2 As shown, the robot control method provided in the embodiment of the present disclosure includes the following S201 to S204.

[0051] S201, receiving first image data and state data sent by the robot.

[0052] The robot is equipped with a camera, and the first image data is the image data obtained by the robot shooting the surrounding environment. The status data includes data generated by various sensors and encoders configured by the robot, such as a gyroscope, a speed sensor, an encoder, etc. The status data can be used to obtain the robot's current speed, acceleration, direction of travel, angular acceleration, posture and other status information. When the robot is a wheeled robot with left and right wheels, the status data can be used to obtain the rotation speed, angle, angular acceleration and other states of the left and right wheels, respectively. After the robot obtains the first image data and status data, it sends the first image data and status data to the server, and then the server receives the first image data and status data.

[0053] S202: Generate a first control strategy corresponding to the first image data and the state data.

[0054] Among them, the first control strategy is a strategy for controlling the robot. The first control strategy can drive the controller carried by the robot, and then drive the robot to perform the action indicated by the first control strategy.

[0055] In some embodiments, a first control strategy corresponding to the first image data and the state data may be generated according to a visual servoing method. For example, generating the first control strategy corresponding to the first image data and the state data may include: acquiring a first path generation model; generating a navigation path corresponding to the first image data according to the first path generation model; and generating a first control strategy corresponding to the navigation path according to the state data. Among them, the embodiment of the present disclosure does not limit what kind of specific model the path generation model is, and any model that can generate a navigation path based on image data can be applied here. For example, the first path generation model is a network model based on the branch and bound method. The navigation path refers to the path from the position where the robot obtains the first image data to the position where the reference object is located. The reference object is an object used to represent the destination of the robot among the various objects included in the first image data. For example, a marker is placed at the destination where the robot is traveling, and the marker can be used as a reference object. For another example, two different markers are placed at the destination where the robot is traveling, and the two different markers can both be used as reference objects. The marker can be any physical entity, such as a wooden block, a box, etc.

[0056] In some embodiments, obtaining the first path generation model may include: obtaining an initial path generation model; obtaining historical navigation data of the robot; using the historical navigation data as a sample to train the initial path generation model to obtain the first path generation model. The historical navigation data may include: historical image data and the corresponding historical navigation path obtained by the robot, and historical state data such as speed, acceleration, direction, angular acceleration, and posture generated during the robot's execution of the historical control strategy corresponding to the historical navigation path.

[0057] When generating the first control strategy for controlling the robot, it is necessary to consider the current state of the robot, and generate the first control strategy for driving the robot according to the current state of the robot. The current state of the robot can be determined by the state data. It should be noted that the first control strategy generated according to the state data and the navigation path is related to the robot, and the specific content of the first control strategy is different for different robots. In some embodiments, generating the first control strategy corresponding to the navigation path according to the state data may include: generating, by mathematical calculation, a first control strategy for driving the robot to move along the navigation path under the state corresponding to the current state data.

[0058] S203, simulating the first control strategy to obtain a first simulation result.

[0059] Wherein, to simulate the control strategy, a simulation model needs to be used and the environment model of the robot's environment needs to be pre-built in the simulation model. In some embodiments, simulating the first control strategy to obtain a first simulation result may include: obtaining one or more simulation models, wherein the environment model of the robot's environment has been built in the one or more simulation models; inputting the first control strategy into the one or more simulation models, and the one or more simulation models simulate the first control strategy according to the environment model to obtain one or more first simulation results.

[0060] In the case where the one or more simulation models obtained are one simulation model, the embodiment of the present disclosure does not limit what kind of simulation model the simulation model is. In the case where the one or more simulation models obtained are multiple simulation models, the multiple simulation models are different models. The embodiment of the present disclosure also does not limit what simulation models the multiple simulation models are. For example, the multiple simulation models are 2 simulation models, and the two simulation models can be Gazebo (a robot simulation model) and Rviz (a robot simulation model).

[0061] In some embodiments, when different simulation models simulate control strategies, they focus on different types of data generated during the simulation process. For example, a certain simulation model pays more attention to the acceleration of the robot and the center of gravity of the robot during simulation. When the acceleration is large, the center of gravity of the robot may shift greatly, causing the robot to fall. Therefore, by simulating the first control strategy with multiple different simulation models, the feasibility and safety of the first control strategy can be evaluated under multiple focuses, so that the simulation results corresponding to the first control strategy have a higher degree of confidence. It should be noted that since different simulation models may focus on different points during the simulation of control strategies, the simulation results obtained by different simulation models executing the same control strategy may be the same or different.

[0062] S204, when the first simulation result is reaching the destination, sending the first control strategy to the robot so that the robot executes the first control strategy.

[0063] In some embodiments, when one simulation model is used to simulate the first control strategy in S203, the one or more first simulation results obtained are one simulation result. At this time, when the first simulation result is reaching the destination, sending the first control strategy to the robot may include: sending the first control strategy to the robot when the one simulation result is reaching the destination.

[0064] In another embodiment, when multiple simulation models are used to simulate the first control strategy in S203, the one or more first simulation results obtained are multiple simulation results. At this time, when the first simulation result is reaching the destination, sending the first control strategy to the robot may include: sending the first control strategy to the robot when a first simulation result greater than or equal to a predetermined proportion of the multiple first simulation results is reaching the destination. The predetermined proportion is a set value, and the embodiment of the present disclosure does not limit the specific value of the predetermined proportion. For example, the predetermined proportion is 1, that is, when all the first simulation results in the multiple first simulation results are reaching the destination, the first control strategy is sent to the robot.

[0065] The technical solution provided by the embodiment of the present disclosure uses the received first image data and state data to generate a first control strategy for controlling the robot, and then simulates the first control strategy to obtain a first simulation result. By simulating the first control strategy, the feasibility and safety of the first control strategy can be verified, thereby avoiding sending an infeasible or unsafe control strategy to the robot for execution. When the first simulation result shows that the robot can reach the destination, it is determined that the first control strategy is feasible and safe, and the feasible and safe first control is sent to the robot and executed by the robot, which can reduce the possibility of safety incidents such as collisions when the controlled robot executes the first control strategy, thereby reducing the possibility of economic losses and improving the safety of controlling the robot when it moves.

[0066] In some other embodiments of the present disclosure, a first control strategy generation method is provided, such as Figure 3 As shown, the first control strategy generating method may include the following S301 to S303.

[0067] S301, acquiring second image data of a detection model and a reference object, where the reference object is used to represent a destination of the robot.

[0068] About the reference object Figure 2The corresponding embodiment has been described in S202 and will not be repeated here. The second image data of the reference object is the image data including the reference object. The detection model can detect the image data to determine whether the detected image includes the target object. For example, the second image data is input into the detection model, and the detection model detects other image data with the reference object included in the second image data as the target to determine whether the other image data includes the reference object. The disclosed embodiment does not limit what kind of model the detection model is. For example, the detection model is a neural network model that uses VGG16 (Visual Geometry Group Network 16, 16 is related to the number of network layers) as the backbone network and uses Faster-RCNN (Faster-Region Convolutional Neural Networks) for object detection.

[0069] S302, inputting the first image data and the second image data into a detection model, and using the detection model to perform detection in the first image data with a reference object corresponding to the second image data as a target, to obtain a detection result.

[0070] Among them, the first image data is Figure 2 The first image data received in S201 of the corresponding embodiment. In some embodiments, the first image data and the second image data are input into a detection model, and the detection model detects the reference object corresponding to the second image data in the first image data as a target to obtain a detection result, which may include: inputting the first image data into the detection model, and the detection model determines the object included in the first image data; inputting the second image data into the detection model, and the detection model determines the reference object included in the second image data; and detecting whether the reference object is included in the object included in the first image data according to the detection model to obtain a detection result.

[0071] S303: When the detection result is that the first image data includes a reference object, generate a first control strategy corresponding to the first image data and the state data.

[0072] Wherein, the implementation method of generating the first control strategy corresponding to the first image data and the state data is as follows: Figure 2 The corresponding embodiment has been described in S202 and will not be repeated here.

[0073] The technical solution provided by the embodiments of the present disclosure detects the first image data, and generates a first control strategy corresponding to the first image data and the status data when it is detected that the first image data includes a reference object for indicating the destination. In this way, it is avoided that when the first image data does not include the reference object, it is necessary to waste computing resources to generate an infeasible first control strategy that cannot reach the destination and corresponds to the first image data, thereby avoiding the robot from executing the infeasible first control strategy, improving the accuracy of generating the control strategy, and avoiding the waste of computing resources. Furthermore, when the robot executes the infeasible first control strategy, the body is prone to shaking and other phenomena, which is not conducive to the control of the robot. By detecting the first image data, it is possible to avoid sending the infeasible first control strategy to the robot, thereby avoiding the body shaking and other phenomena of the robot to a certain extent, which is conducive to better control of the robot.

[0074] In other embodiments of the present disclosure, Figure 2 In the case where the first simulation result obtained in S203 of the corresponding embodiment is that the destination has not been reached, the robot control method provided by the embodiment of the present disclosure is as follows: Figure 4 As shown, it also includes S401 to S404.

[0075] S401, when the first simulation result is that the destination has not been reached, the simulation data generated during the simulation of the first control strategy is input into the first path generation model, and the first path generation model adjusts parameters according to the simulation data to obtain a second path generation model.

[0076] Wherein, when the first simulation result is that the destination is not reached, the first control strategy can be considered to be an infeasible control strategy. The first simulation result is that the destination is not reached, which may include: when a simulation model is used to simulate the first control strategy, a first simulation result obtained is that the destination is not reached; or when multiple simulation models are used to simulate the first control strategy, the first simulation results less than a predetermined proportion of the multiple first simulation results obtained are that the destination is reached.

[0077] In some embodiments, the simulation data generated during the simulation of the first control strategy will be stored in the memory of the server, so when it is determined that the first simulation result is that the destination has not been reached, the simulation data can be directly used to train the first path generation model. During the training of the first path generation model, the parameters of the first path generation model are adjusted to obtain the second path generation model.

[0078] S402, generating a second control strategy corresponding to the first image data and the state data according to the second path generation model. In some embodiments, the implementation method of generating the second control strategy corresponding to the first image data and the state data according to the second path generation model is as follows: Figure 2 In S202 of the corresponding embodiment, the manner of generating the first control strategy corresponding to the first image data and the state data according to the first path generation model is the same, which will not be repeated here.

[0079] S403, simulating the second control strategy to obtain a second simulation result. In some embodiments, the second control strategy is simulated to obtain a second simulation result. Figure 2 In S203 of the corresponding embodiment, the first control strategy is simulated to obtain the first simulation result in the same manner, which will not be described in detail herein.

[0080] S404, when the second simulation result is that the destination is reached, sending the second control strategy to the robot so that the robot executes the second control strategy. In some embodiments, when the second simulation result is that the destination is reached, sending the second control strategy to the robot is implemented in the same manner as Figure 2 In S204 of the corresponding embodiment, when the first simulation result is reaching the destination, the manner of sending the first control strategy to the robot is the same, which will not be repeated here.

[0081] In another embodiment of the present disclosure, if the simulation results corresponding to the control strategy generated according to the first image data and the state data for n consecutive times all fail to reach the destination, the first image data is marked as a singular image; wherein the singular image is used to indicate that the control strategy generated according to the first image data for n consecutive times does not enable the robot to reach the destination in the simulation; n is an integer greater than or equal to 2. If it is determined that the first image data is a singular image, a navigation path corresponding to the first image data can be planned by manual intervention processing.

[0082] The technical solution provided by the embodiment of the present disclosure feeds back the simulation data representing the warning result of the first control strategy to the first path generation model when the first simulation result indicates that the first control strategy is an unfeasible control strategy. The first path generation model optimizes and adjusts the parameters according to the simulation data to obtain a second path generation model. The second path generation model can generate a navigation path that is more in line with the environment corresponding to the first image data, and then when simulating the second control strategy generated according to the navigation path, it can avoid the problems that the first control strategy encounters in the simulation to a certain extent, thereby increasing the possibility that the second control strategy is a feasible control strategy. In addition, when multiple simulations fail to obtain a feasible control strategy, the first image data can be marked as a singular image and manual processing can be introduced to avoid the continuous waste of computing resources.

[0083] To facilitate understanding of the robot control method provided by the embodiment of the present disclosure, Figure 2 , Figure 3 and Figure 4 The corresponding embodiments illustrate the robot control method provided by the embodiments of the present disclosure. Figure 5 As shown, it includes S501 to S511.

[0084] S501: The robot obtains first image data and state data.

[0085] S502: The robot sends first image data and status data to the server.

[0086] S503: The server detects the first image data, and executes S504 if the detection passes, and directly terminates the process (continues to execute the currently executed control strategy) if the detection fails.

[0087] S504: The server generates a navigation path corresponding to the first image data using a path generation model.

[0088] S505: The server generates a control strategy according to the navigation path and the status data.

[0089] S506: The server simulates the control strategy according to multiple simulation models to obtain simulation results.

[0090] S507: The server determines whether the simulation result meets the requirements, and executes S508 if the simulation result meets the requirements; if the simulation result does not meet the requirements, execute S509; if the simulation results corresponding to the control strategy generated according to the first image data and the state data for n consecutive times do not meet the requirements, execute S510, where n is an integer greater than or equal to 2.

[0091] S508: The server sends the control strategy to the robot.

[0092] S509: The server adjusts the parameters of the path generation model according to the simulation data generated during the simulation of the control strategy, obtains the adjusted path generation model, and executes S504.

[0093] S510: The server marks the first image data as a singular image.

[0094] S511: The robot executes the control strategy.

[0095] Figure 5 The specific implementation of each step of the corresponding embodiment can be found in Figure 2 , Figure 3 and Figure 4 The corresponding embodiments will not be described in detail here. The technical solution provided by the embodiments of the present disclosure sends the image data and status data acquired by the robot to the server, and the server detects the image data. If the detection passes, a control strategy is generated based on the image data and status data, and the control strategy is simulated. If the simulation result indicates that the control strategy is feasible, the control strategy is sent to the robot, and the robot executes the control strategy. This method can effectively improve the navigation accuracy and efficiency of the robot.

[0096] Based on the same inventive concept, the present disclosure also provides a robot control device in the following embodiments. Since the principle of solving the problem in the device embodiment is similar to that in the above method embodiment, the implementation of the device embodiment can refer to the implementation of the above method embodiment, and the repeated parts will not be repeated.

[0097] Figure 6 A schematic diagram of a robot control device in an embodiment of the present disclosure is shown. Figure 6 As shown, the device includes: a receiving module 601, used for receiving first image data and state data sent by the robot; a generating module 602, used for generating a first control strategy corresponding to the first image data and state data; a simulation module 603, used for simulating the first control strategy to obtain a first simulation result; and a sending module 604, used for sending the first control strategy to the robot when the first simulation result is reaching the destination, so that the robot executes the first control strategy.

[0098] In one embodiment of the present disclosure, a generation module 602 is used to obtain a detection model and second image data of a reference object, where the reference object is used to represent the destination of the robot; the first image data and the second image data are input into the detection model, and the detection model is used to perform detection in the first image data with the reference object corresponding to the second image data as the target to obtain a detection result; when the detection result is that the reference object is included in the first image data, a first control strategy corresponding to the first image data and the state data is generated.

[0099] In one embodiment of the present disclosure, the generation module 602 is used to obtain a first path generation model; generate a navigation path corresponding to the first image data according to the first path generation model; and generate a first control strategy corresponding to the navigation path according to the state data.

[0100] In one embodiment of the present disclosure, the simulation module 603 is used to obtain one or more simulation models, in which an environmental model of the robot's environment has been constructed in the one or more simulation models; the first control strategy is input into the one or more simulation models, and the one or more simulation models simulate the first control strategy according to the environmental model to obtain one or more first simulation results.

[0101] In one embodiment of the present disclosure, one or more first simulation results are multiple first simulation results; the sending module 604 is used to send a first control strategy to the robot when a first simulation result greater than or equal to a predetermined proportion of the multiple first simulation results is reaching the destination.

[0102] In one embodiment of the present disclosure, the device also includes: an adjustment module 605, which is used to input the simulation data generated during the simulation of the first control strategy into the first path generation model when the first simulation result is that the destination has not been reached, and the first path generation model adjusts the parameters according to the simulation data to obtain a second path generation model; the generation module 602 is also used to generate a second control strategy corresponding to the first image data and the state data according to the second path generation model; the simulation module 603 is also used to simulate the second control strategy to obtain a second simulation result; the sending module 604 is also used to send the second control strategy to the robot when the second simulation result is that the destination has been reached, so that the robot can execute the second control strategy.

[0103] In one embodiment of the present disclosure, the device also includes: a calibration module 606, which is used to mark the first image data as a singular image when the simulation results corresponding to the control strategy generated according to the first image data and the state data for n consecutive times all fail to reach the destination; wherein the singular image is used to indicate that the control strategy generated according to the first image data for n consecutive times does not enable the robot to reach the destination in the simulation; n is an integer greater than or equal to 2.

[0104] In one embodiment of the present disclosure, the robot is a wheeled robot.

[0105] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods or program products. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as "circuits", "modules" or "systems".

[0106] Refer to the following Figure 7 An electronic device 700 according to this embodiment of the present disclosure is described. Figure 7 The electronic device 700 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0107] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0108] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Specific Implementation Methods” section of this specification.

[0109] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0110] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0111] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0112] The electronic device 700 may also communicate with one or more external devices 740 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 700, and / or any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 760. Figure 7 As shown, the network adapter 760 communicates with other modules of the electronic device 700 via the bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0113] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0114] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. A program product capable of implementing the above-mentioned method of the present disclosure is stored thereon. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary implementations of the present disclosure described in the above “Specific Implementation Methods” section of this specification.

[0115] More specific examples of computer-readable storage media in the present disclosure may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] In the present disclosure, a computer readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0117] Alternatively, the program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0118] In a specific implementation, the program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0119] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0120] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0121] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0122] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The description and examples are to be regarded as exemplary only, and the true scope of the present disclosure is indicated by the appended claims.

Claims

1. A robot control method, characterized in that: include: Receiving first image data and state data sent by the robot, wherein the first image data is image data obtained by the robot photographing the surrounding environment; generating a first control strategy corresponding to the first image data and the state data; Simulating the first control strategy to obtain a first simulation result; When the first simulation result is reaching the destination, sending the first control strategy to the robot so that the robot executes the first control strategy; Wherein, the generating of the first control strategy corresponding to the first image data and the state data includes: Acquire second image data of a detection model and a reference object, wherein the reference object is used to represent a destination to which the robot is traveling; Inputting the first image data and the second image data into the detection model, and performing detection in the first image data using the detection model with a reference object corresponding to the second image data as a target to obtain a detection result; generating a first control strategy corresponding to the first image data and the state data when the detection result shows that the first image data includes the reference object; The simulating the first control strategy to obtain a first simulation result includes: Acquire one or more simulation models, wherein the one or more simulation models have constructed an environment model of the environment where the robot is located; Inputting the first control strategy into the one or more simulation models, and having the one or more simulation models simulate the first control strategy according to the environment model to obtain one or more first simulation results; The one or more first simulation results are multiple first simulation results; when the first simulation result is reaching the destination, sending the first control strategy to the robot includes: When a first simulation result greater than or equal to a predetermined proportion of the plurality of first simulation results reaches the destination, the first control strategy is sent to the robot.

2. The method according to claim 1, characterized in that: The generating of the first control strategy corresponding to the first image data and the state data comprises: Obtaining a first path generation model; generating a navigation path corresponding to the first image data according to the first path generation model; A first control strategy corresponding to the navigation path is generated according to the state data.

3. The method according to any one of claims 1 and 2, characterized in that: Also includes: When the first simulation result is that the destination is not reached, the simulation data generated in the simulation process of the first control strategy is input into the first path generation model, and the first path generation model adjusts parameters according to the simulation data to obtain a second path generation model; Generate a second control strategy corresponding to the first image data and the state data according to the second path generation model; Simulating the second control strategy to obtain a second simulation result; When the second simulation result is reaching the destination, the second control strategy is sent to the robot so that the robot executes the second control strategy.

4. The method according to claim 3, characterized in that Also includes: If the simulation results corresponding to the control strategy generated according to the first image data and the state data for n consecutive times all fail to reach the destination, marking the first image data as a singular image; The singular image is used to indicate that the control strategies generated n times in succession based on the first image data do not enable the robot to reach the destination in the simulation; n is an integer greater than or equal to 2.

5. The method according to any one of claims 1-2, characterized in that: The robot is a wheeled robot.

6. A robot control device, characterized in that: include: A receiving module, used to receive first image data and state data sent by the robot, wherein the first image data is image data obtained by the robot photographing the surrounding environment; A generation module is used to generate a first control strategy corresponding to the first image data and the state data; specifically comprising: obtaining a detection model and second image data of a reference object, the reference object being used to represent a destination of the robot; inputting the first image data and the second image data into the detection model, and performing detection in the first image data by the detection model with the reference object corresponding to the second image data as a target to obtain a detection result; and generating a first control strategy corresponding to the first image data and the state data when the detection result is that the reference object is included in the first image data; A simulation module is used to simulate the first control strategy to obtain a first simulation result; specifically comprising: obtaining one or more simulation models, wherein the one or more simulation models have constructed an environment model of the environment where the robot is located; inputting the first control strategy into the one or more simulation models, and the one or more simulation models simulate the first control strategy according to the environment model to obtain one or more first simulation results; the one or more first simulation results are multiple first simulation results; A sending module is used to send the first control strategy to the robot when the first simulation result is reaching the destination, so that the robot executes the first control strategy; the sending of the first control strategy to the robot when the first simulation result is reaching the destination specifically includes: sending the first control strategy to the robot when a first simulation result greater than or equal to a predetermined proportion of the multiple first simulation results is reaching the destination.

7. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to perform the robot control method according to any one of claims 1 to 5 by executing the executable instructions.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the robot control method according to any one of claims 1 to 5 is implemented.

Images