A control system based on robot controller and control method thereof

Abstract

The present invention discloses a control system based on a robot controller and a control method thereof, which relate to the technical field of robot controllers and include the following modules: an identification and positioning module, which scans the robot's surrounding environment through a laser sensor arranged on the top of the robot, establishes an environmental three-dimensional space model, obtains factory workshop map information, and marks the robot's current position in the map information by comparing the map information with the environmental three-dimensional space model information to obtain the robot positioning information; in this scheme, the environmental three-dimensional space model and the positioning information are calibrated through the information calibration module, and the environmental three-dimensional space model is calibrated and supplemented by comparing the images taken by multiple cameras with the environmental three-dimensional space model, so that the robot can have a wider field of view through the camera to understand the surrounding changing environment, which is beneficial to avoid the robot's limited scanning range and falling into the dilemma of local optimal path planning.

Description

Technical Field

[0001] The present invention relates to the technical field of robot controllers, and in particular to a control system based on a robot controller and a control method thereof. Background Art

[0002] With the continuous advancement of industrial automation, most production lines have been automated, and various industrial robots have been used in different production lines to adapt to different application scenarios, thereby saving a lot of manpower and financial resources and greatly improving work efficiency. The scope of application of robots is constantly expanding, and the intelligence level of robots is constantly improving. The robot controller is the component that controls the robot to perform operations, such as path planning and movement, clamping and grasping, steering and rotation, etc.

[0003] The handling robots in the factory workshop mostly move along the pre-set moving path during the handling process. When encountering obstacles on the moving path, they will stop immediately to scan the obstacles and finally bypass the obstacles and return to the moving path to continue moving. However, the environment in the factory workshop is changeable and the robot's scanning range is limited. Therefore, the following problems are often encountered in the path planning of the robot during the obstacle avoidance process:

[0004] 1. Falling into the dilemma of local optimal path planning, especially in a narrow and crowded environment, the robot often needs to constantly scan and avoid obstacles. However, the robot's scanning range is limited, and the robot is prone to wandering between multiple obstacles, which will seriously affect the robot's handling efficiency;

[0005] Second, since robots usually rely on sensors for positioning, but sensors may have errors, these errors will affect the accuracy of path planning, causing the robot to deviate from the predetermined path. Positioning errors will gradually accumulate, making path planning increasingly inaccurate, which will also seriously affect the robot's handling efficiency. For this reason, we propose a control system based on a robot controller and its control method. Summary of the invention

[0006] 1. Technical issues to be resolved

[0007] In view of the deficiencies in the prior art, the present invention provides a control system based on a robot controller and a control method thereof to solve the above-mentioned problems in the prior art.

[0008] (II) Technical solution

[0009] To achieve the above objectives, the present invention is implemented through the following technical solutions: a control system based on a robot controller, comprising the following modules:

[0010] The identification and positioning module uses a laser sensor installed on the top of the robot to scan the robot's surrounding environment, establish a three-dimensional spatial model of the environment, obtain factory workshop map information, and compare the map information with the three-dimensional spatial model information of the environment to mark the robot's current position in the map information to obtain the robot's positioning information;

[0011] The information calibration module obtains images taken by multiple cameras in the factory workshop, calibrates and corrects the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrates and corrects the robot positioning information through the images taken by multiple cameras;

[0012] The path decision module obtains the length, height and width of the robot, sets the obstacle safety distance, obtains the calibrated three-dimensional space model of the environment, obtains the calibrated robot positioning information, obtains the robot's predetermined path, determines the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulates the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path;

[0013] The control drive module transmits data instructions to the servo motor through the controller, driving the robot to move forward, backward and rotate on the spot according to the predetermined path and obstacle avoidance path.

[0014] Preferably, a three-dimensional spatial model of the environment is established in the identification and positioning module, specifically:

[0015] Step 1: Set the scanning resolution, scanning angle range and point cloud density, execute the scanning task and scan the surrounding environment three times to obtain three sets of environmental 3D point clouds respectively;

[0016] Step 2: Compare the three groups of environmental 3D point clouds, mark all common point clouds in the three groups of environmental 3D point clouds to obtain the environmental 3D point cloud set, establish a 3D space coordinate system with the shooting coordinates as the origin, mark the 3D coordinates of the environmental 3D point cloud set, mark the position of the 3D point cloud set in the 3D space coordinate system, perform triangulation and surface fitting on the 3D point cloud set in the coordinate system to achieve surface reconstruction of the environmental 3D space model, and finally obtain the environmental 3D space model.

[0017] Preferably, in the identification and positioning module, the current position of the robot is marked in the map information to obtain the robot positioning information by comparing the map information with the three-dimensional space model information of the environment, specifically:

[0018] Step 1: Identify multiple map information and establish a map 3D space model through common points in the multiple map information, convert the map 3D space model into a map 3D coordinate system point cloud set, obtain an environment 3D point cloud set, compare the map 3D point cloud set with the environment 3D point cloud set, and mark all common points in the map 3D point cloud set;

[0019] Step 2: Obtain the common point cloud cluster with the largest density, restore the three-dimensional space model of the map at the common point cloud cluster to obtain the reference landmark, obtain the coordinates of the reference landmark, and obtain the robot positioning coordinate information through the Euclidean distance formula in the environment three-dimensional space coordinate system, the map three-dimensional space coordinate system, the origin coordinates of the environment three-dimensional space coordinate system and the reference landmark coordinates.

[0020] Preferably, in the information calibration module, the three-dimensional space model of the environment is corrected by shooting pictures with multiple cameras, specifically:

[0021] Step 1: obtain multiple pictures taken by cameras at different positions, establish a shooting 3D space model using the MVS algorithm through multiple groups of shooting pictures, obtain the environment 3D space model, compare the environment 3D space model with the shooting 3D space model, and mark the common models in the environment 3D space model and the shooting 3D space model, the unique models in the shooting 3D space model, and the unique models in the environment 3D space model;

[0022] Step 2: copy the unique model in the photographed three-dimensional space model to the environmental three-dimensional space model according to the coordinate position relationship of the shared model to obtain a proofread environmental three-dimensional space model, and mark the unique model in the environmental three-dimensional space model;

[0023] Step 3: Scan the surrounding environment again through the identification and positioning module to confirm whether the unique model in the marked environmental three-dimensional space model actually exists. If the unique model in the marked environmental three-dimensional space model actually exists, the proofread environmental three-dimensional space model is defined as the final environmental three-dimensional space model. If the unique model in the marked environmental three-dimensional space model does not actually exist, delete the unique model in the environmental three-dimensional space model marked in the proofread environmental three-dimensional space model, and define the deleted environmental three-dimensional space model as the final environmental three-dimensional space model.

[0024] Preferably, in the information calibration module, the robot positioning information is corrected by taking pictures with multiple cameras, specifically:

[0025] Step 1: Obtain the three-dimensional space model of the shooting, take the workshop entrance as the origin coordinate to obtain the three-dimensional space coordinates of the robot in the shooting three-dimensional space model, obtain the three-dimensional space model of the environment, take the workshop entrance as the origin to obtain the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment;

[0026] Step 2: The positioning error is obtained by subtracting the three-dimensional spatial coordinates of the robot in the captured three-dimensional spatial model from the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment. If the positioning error is 0, the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment are used as the positioning information of the robot as the final positioning information. If the positioning error is not 0, the three-dimensional spatial coordinates of the robot in the captured three-dimensional spatial model are replaced by the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment to obtain the final positioning information.

[0027] Preferably, the positioning error is calculated as follows:

[0028]

[0029] Among them, Wh represents the positioning error, Represented as the X-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the X-axis coordinate of the robot in the three-dimensional space model. Represented as the Y-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the Y-axis coordinate of the robot in the three-dimensional space model. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model of the environment. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model.

[0030] Preferably, the starting point and the end point of the obstacle avoidance path of the robot in the predetermined path are determined in the path decision module, specifically:

[0031] Step 1: Set the emergency stop safety distance when scanning obstacles, define the position of the robot after emergency stop as the starting point of the obstacle avoidance path, obtain the calibrated three-dimensional space model of the environment, mark all obstacles in front of the robot in the predetermined path, and measure the distance and limit height of every two adjacent obstacles;

[0032] Step 2: Get the length, height and width of the robot, set the obstacle safety distance, sum the obstacle safety distance with the length, height and width of the robot to get the safety length, safety height and safety width respectively, compare the size relationship between the limit height and the safety height, if the safety height is less than the limit height, mark it as passable in height, if the safety height is greater than the limit height, mark it as inaccessible in height, compare the size relationship between the safety length and the safety width, select the smaller one as the passing direction, compare the size relationship between the length of the passing direction and the obstacle spacing, if the length of the passing direction is less than the obstacle spacing, mark it as passable in spacing, if the length of the passing direction is greater than the obstacle spacing, mark it as inaccessible in spacing;

[0033] Step 3: Determine whether the robot can pass between two adjacent obstacles. If the height is acceptable and the distance between the obstacles is acceptable, it is marked as eventually passable. Otherwise, it is marked as eventually inaccessible. The intersection of the passing path and the predetermined path is taken as the end point of the obstacle avoidance path.

[0034] Preferably, in the path decision module, the robot obstacle avoidance path is formulated according to the starting point and the end point of the obstacle avoidance path, specifically:

[0035] Step 1: If the end point of the obstacle avoidance path is not between two obstacles, measure the distance from the starting point of the obstacle avoidance path to the left and right sides of the obstacle respectively, and mark them as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the obstacle from the left. , the robot will bypass the obstacle from the right side;

[0036] Step 2: If the end point of the obstacle avoidance path is between two obstacles, the end point of the obstacle avoidance path passing through the second obstacle is defined as the virtual end point. The distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle is measured and marked as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the first obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the first obstacle. , the robot will bypass the right side of the first obstacle and measure the distance from the end point of the obstacle avoidance path to the left and right sides of the second obstacle, respectively, marked as and , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the second obstacle. , the robot will go around the right side of the second obstacle and finally get the first obstacle avoidance path;

[0037] Step 3: If the end point of the obstacle avoidance path is between two obstacles, the end point of the obstacle avoidance path passing through the second obstacle is defined as the virtual end point. The distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle is measured and marked as and , measure the distance from the left side of the first obstacle to the left side of the second obstacle, marked as , measure the distance from the right side of the first obstacle to the right side of the second obstacle, marked as , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right , and Ask for ,right , and Ask for ,Compare and If , the robot will go around the left side of the two obstacles and directly reach the virtual end point. , the robot will go around the right side of the two obstacles and directly reach the virtual end point, and finally obtain the second obstacle avoidance path;

[0038] Step 4: Get the total length of the first obstacle avoidance path , get the total length of the second obstacle avoidance path ,Compare and If , then the first obstacle avoidance path is selected as the final obstacle avoidance path. , then the second obstacle avoidance path is selected as the final obstacle avoidance path.

[0039] A control method based on a robot controller comprises the following steps:

[0040] Step 1: Scan the robot's surrounding environment through the laser sensor installed on the top of the robot, establish a three-dimensional space model of the environment, obtain the factory workshop map information, and mark the robot's current position in the map information by comparing the map information with the three-dimensional space model information of the environment to obtain the robot positioning information;

[0041] Step 2: Obtain images taken by multiple cameras in the factory workshop, calibrate and correct the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrate and correct the robot positioning information through the images taken by multiple cameras;

[0042] Step 3: Obtain the length, height and width of the robot, set the obstacle safety distance, obtain the calibrated three-dimensional space model of the environment, obtain the calibrated robot positioning information, obtain the robot's predetermined path, determine the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulate the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path;

[0043] Step 4: The controller transmits data instructions to the servo motor to drive the robot to move forward, backward and rotate in place according to the predetermined path and obstacle avoidance path. A2

[0044] (III) Beneficial effects

[0045] The present invention provides a control system based on a robot controller and a control method thereof, which have the following beneficial effects:

[0046] In this scheme, the three-dimensional space model of the environment and the positioning information are calibrated through the information calibration module. By comparing the images captured by multiple cameras and the three-dimensional space model of the environment, the three-dimensional space model of the environment is calibrated and supplemented, so as to make the three-dimensional space model of the environment more specific and rich, so that the robot can have a wider field of view through the camera to understand the changing environment around it, which is beneficial to avoid the robot's limited scanning range and fall into the dilemma of local optimal path planning, which is beneficial to avoid affecting the robot's handling efficiency. At the same time, the positioning information of the robot is calibrated through the images captured by multiple cameras, so as to comprehensively determine the positioning information of the robot by combining the shooting information and the laser sensor scanning information, which is beneficial to avoid the robot deviating from the predetermined path due to sensor errors, which is further beneficial to avoid affecting the robot's handling efficiency.

[0047] In this solution, the robot's obstacle avoidance path is selected through the path decision module, so that when the robot encounters an obstacle or a crowded and narrow space, it is convenient for the robot to choose the shortest and most suitable path to avoid the obstacle, which is beneficial to avoid the situation where the robot is prone to wandering between multiple obstacles, and further helps to avoid affecting the robot's handling efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the structure of a control system based on a robot controller of the present invention;

[0049] Figure 2 The present invention is a flowchart of a control method based on a robot controller. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1-Figure 2 The present invention provides a control system based on a robot controller, comprising the following modules:

[0052] The identification and positioning module uses a laser sensor installed on the top of the robot to scan the robot's surrounding environment, establish a three-dimensional spatial model of the environment, obtain factory workshop map information, and compare the map information with the three-dimensional spatial model information of the environment to mark the robot's current position in the map information to obtain the robot's positioning information;

[0053] The information calibration module obtains images taken by multiple cameras in the factory workshop, calibrates and corrects the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrates and corrects the robot positioning information through the images taken by multiple cameras;

[0054] The path decision module obtains the length, height and width of the robot, sets the obstacle safety distance, obtains the calibrated three-dimensional space model of the environment, obtains the calibrated robot positioning information, obtains the robot's predetermined path, determines the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulates the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path;

[0055] The control drive module transmits data instructions to the servo motor through the controller, driving the robot to move forward, backward and rotate on the spot according to the predetermined path and obstacle avoidance path.

[0056] In this embodiment, the robot's surrounding environment is scanned and identified by the identification and positioning module, and a three-dimensional spatial model of the environment is established after scanning, so as to facilitate the identification of obstacles blocking the predetermined path during the movement of the robot, and the three-dimensional spatial model of the environment is compared with the recorded factory workshop map information, so as to facilitate the determination of the robot's positioning information in the workshop, and then facilitate the subsequent robot movement work;

[0057] In this scheme, the three-dimensional space model of the environment and the positioning information are calibrated through the information calibration module, and the three-dimensional space model of the environment is calibrated and supplemented by comparing the images taken by multiple cameras with the three-dimensional space model of the environment, so as to make the three-dimensional space model of the environment more specific and rich, so that the robot can understand the changing environment around it through a wider field of view of the camera, which is beneficial to avoid the robot's limited scanning range and falling into the dilemma of local optimal path planning, which is beneficial to avoid affecting the robot's handling efficiency. At the same time, the positioning information of the robot is calibrated through the images taken by multiple cameras, so as to comprehensively determine the positioning information of the robot by combining the shooting information and the laser sensor scanning information, which is beneficial to avoid the robot deviating from the predetermined path due to sensor errors, which is further beneficial to avoid affecting the robot's handling efficiency.

[0058] In this solution, the robot's obstacle avoidance path is selected through the path decision module, so that when the robot encounters an obstacle or a crowded and narrow space, it is convenient for the robot to choose the shortest and most suitable path to avoid the obstacle, which is beneficial to avoid the situation where the robot is prone to wandering between multiple obstacles, and further helps to avoid affecting the robot's handling efficiency;

[0059] In this solution, the servo motor of the robot chassis is driven and controlled by the control drive module, so that the robot can move according to the predetermined path and obstacle avoidance path. The robot can rotate on the spot to turn in a small space and fully scan the surrounding environment.

[0060] The three-dimensional space model of the environment is established in the recognition and positioning module, specifically:

[0061] Step 1: Set the scanning resolution, scanning angle range and point cloud density, execute the scanning task and scan the surrounding environment three times to obtain three sets of environmental 3D point clouds respectively;

[0062] Step 2: Compare the three groups of environmental 3D point clouds, mark all common point clouds in the three groups of environmental 3D point clouds to obtain the environmental 3D point cloud set, establish a 3D space coordinate system with the shooting coordinates as the origin, mark the 3D coordinates of the environmental 3D point cloud set, mark the position of the 3D point cloud set in the 3D space coordinate system, perform triangulation and surface fitting on the 3D point cloud set in the coordinate system to achieve surface reconstruction of the environmental 3D space model, and finally obtain the environmental 3D space model.

[0063] In this embodiment, a three-dimensional point cloud is obtained by multiple scans, and then multiple groups of three-dimensional point clouds are compared, so as to facilitate data cleaning and subsequent more accurate modeling. The three-dimensional point cloud set is triangulated and surface fitted, so as to reconstruct the surface of the three-dimensional point cloud set, so as to facilitate the formation of the final three-dimensional space model of the environment, so as to facilitate the subsequent calculation of the actual distance between the obstacle and the robot, so as to facilitate the robot to select a shorter obstacle avoidance route, and finally to improve the robot's handling efficiency.

[0064] In the recognition and positioning module, the robot positioning information is obtained by comparing the map information with the three-dimensional space model information of the environment, marking the current position of the robot in the map information, specifically:

[0065] Step 1: Identify multiple map information and establish a map 3D space model through common points in the multiple map information, convert the map 3D space model into a map 3D coordinate system point cloud set, obtain an environment 3D point cloud set, compare the map 3D point cloud set with the environment 3D point cloud set, and mark all common points in the map 3D point cloud set;

[0066] Step 2: Obtain the common point cloud cluster with the largest density, restore the three-dimensional space model of the map at the common point cloud cluster to obtain the reference landmark, obtain the coordinates of the reference landmark, and obtain the robot positioning coordinate information through the Euclidean distance formula in the environment three-dimensional space coordinate system, the map three-dimensional space coordinate system, the origin coordinates of the environment three-dimensional space coordinate system and the reference landmark coordinates.

[0067] In this embodiment, a three-dimensional space model of the map is established through map information, and the common points of the two models are found by comparing the point cloud sets in the three-dimensional space model of the map and the three-dimensional space model of the environment. Then, the common point cloud cluster with the largest density is selected and the three-dimensional space model is restored, thereby confirming the common reference objects of the three-dimensional space model of the map and the three-dimensional space model of the environment, and finally, the positioning information of the robot is determined based on the common reference objects.

[0068] In the information calibration module, the three-dimensional space model of the environment is corrected by shooting pictures with multiple cameras, specifically:

[0069] Step 1: obtain multiple pictures taken by cameras at different positions, establish a shooting 3D space model using the MVS algorithm through multiple groups of shooting pictures, obtain the environment 3D space model, compare the environment 3D space model with the shooting 3D space model, and mark the common models in the environment 3D space model and the shooting 3D space model, the unique models in the shooting 3D space model, and the unique models in the environment 3D space model;

[0070] Step 2: copy the unique model in the photographed three-dimensional space model to the environmental three-dimensional space model according to the coordinate position relationship of the shared model to obtain a proofread environmental three-dimensional space model, and mark the unique model in the environmental three-dimensional space model;

[0071] Step 3: Scan the surrounding environment again through the identification and positioning module to confirm whether the unique model in the marked environmental three-dimensional space model actually exists. If the unique model in the marked environmental three-dimensional space model actually exists, the proofread environmental three-dimensional space model is defined as the final environmental three-dimensional space model. If the unique model in the marked environmental three-dimensional space model does not actually exist, delete the unique model in the environmental three-dimensional space model marked in the proofread environmental three-dimensional space model, and define the deleted environmental three-dimensional space model as the final environmental three-dimensional space model.

[0072] In this embodiment, the picture taken by the camera is used to generate a three-dimensional space model, and the three-dimensional space model is then compared with the environmental three-dimensional space model, so that the models that have not been scanned in the environmental three-dimensional space model are supplemented, and the models that are incorrectly scanned in the environmental three-dimensional simplified model are deleted, thereby proofreading and confirming the authenticity and accuracy of the environmental three-dimensional space model, which facilitates the subsequent planning of obstacle avoidance paths.

[0073] In the information calibration module, the robot positioning information is corrected by shooting pictures with multiple cameras, specifically:

[0074] Step 1: Obtain the three-dimensional space model of the shooting, take the workshop entrance as the origin coordinate to obtain the three-dimensional space coordinates of the robot in the shooting three-dimensional space model, obtain the three-dimensional space model of the environment, take the workshop entrance as the origin to obtain the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment;

[0075] Step 2: The positioning error is obtained by subtracting the three-dimensional space coordinates of the robot in the three-dimensional space model of the shooting and the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment. If the positioning error is 0, the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment are used as the positioning information of the robot as the final positioning information. If the positioning error is not 0, the three-dimensional space coordinates of the robot in the three-dimensional space model of the shooting are replaced by the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment to obtain the final positioning information.

[0076] The calculation method of positioning error is as follows:

[0077]

[0078] Among them, Wh represents the positioning error, Represented as the X-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the X-axis coordinate of the robot in the three-dimensional space model. Represented as the Y-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the Y-axis coordinate of the robot in the three-dimensional space model. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model of the environment. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model.

[0079] In this embodiment, the position information of the robot in the three-dimensional space model is compared with the position information of the robot in the three-dimensional model of the environment, and the positioning error is obtained by subtracting the two. The positioning error is used to determine whether there is any deviation in the positioning information of the robot, thereby facilitating timely correction of the robot's erroneous positioning information, thereby facilitating the robot to accurately move on the predetermined path, and also facilitating the subsequent movement of the robot on the obstacle avoidance path.

[0080] In the path decision module, the starting point and end point of the robot's obstacle avoidance path in the predetermined path are determined, specifically:

[0081] Step 1: Set the emergency stop safety distance when scanning obstacles, define the position of the robot after emergency stop as the starting point of the obstacle avoidance path, obtain the calibrated three-dimensional space model of the environment, mark all obstacles in front of the robot in the predetermined path, and measure the distance and limit height of every two adjacent obstacles;

[0082] Step 2: Get the length, height and width of the robot, set the obstacle safety distance, sum the obstacle safety distance with the length, height and width of the robot to get the safety length, safety height and safety width respectively, compare the size relationship between the limit height and the safety height, if the safety height is less than the limit height, mark it as passable in height, if the safety height is greater than the limit height, mark it as inaccessible in height, compare the size relationship between the safety length and the safety width, select the smaller one as the passing direction, compare the size relationship between the length of the passing direction and the obstacle spacing, if the length of the passing direction is less than the obstacle spacing, mark it as passable in spacing, if the length of the passing direction is greater than the obstacle spacing, mark it as inaccessible in spacing;

[0083] Step 3: Determine whether the robot can pass between two adjacent obstacles. If the height is acceptable and the distance between the obstacles is acceptable, it is marked as eventually passable. Otherwise, it is marked as eventually inaccessible. The intersection of the passing path and the predetermined path is taken as the end point of the obstacle avoidance path.

[0084] In this embodiment, by calculating the size of the robot and the gap size between multiple obstacles, it is convenient to determine whether the robot can pass between multiple obstacles, thereby providing a variety of options for the robot's obstacle avoidance path, thereby facilitating the robot to pass through and avoid obstacles in crowded and narrow spaces, which is further beneficial for the robot to select a suitable obstacle avoidance route, and further beneficial for improving the robot's handling efficiency.

[0085] In the path decision module, the robot's obstacle avoidance path is formulated according to the starting point and end point of the obstacle avoidance path, specifically:

[0086] Step 1: If the end point of the obstacle avoidance path is not between two obstacles, measure the distance from the starting point of the obstacle avoidance path to the left and right sides of the obstacle respectively, and mark them as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the obstacle from the left. , the robot will bypass the obstacle from the right side;

[0087] Step 2: If the end point of the obstacle avoidance path is between two obstacles, the end point of the obstacle avoidance path passing through the second obstacle is defined as the virtual end point. The distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle is measured and marked as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the first obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the first obstacle. , the robot will bypass the right side of the first obstacle and measure the distance from the end point of the obstacle avoidance path to the left and right sides of the second obstacle, respectively, marked as and , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the second obstacle. , the robot will go around the right side of the second obstacle and finally get the first obstacle avoidance path;

[0088] Step 3: If the end point of the obstacle avoidance path is between two obstacles, the end point of the obstacle avoidance path passing through the second obstacle is defined as the virtual end point. The distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle is measured and marked as and , measure the distance from the left side of the first obstacle to the left side of the second obstacle, marked as , measure the distance from the right side of the first obstacle to the right side of the second obstacle, marked as , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right , and Ask for ,right , and Ask for ,Compare and If , the robot will go around the left side of the two obstacles and directly reach the virtual end point. , the robot will go around the right side of the two obstacles and directly reach the virtual end point, and finally obtain the second obstacle avoidance path;

[0089] Step 4: Get the total length of the first obstacle avoidance path , get the total length of the second obstacle avoidance path ,Compare and If , then the first obstacle avoidance path is selected as the final obstacle avoidance path. , then the second obstacle avoidance path is selected as the final obstacle avoidance path.

[0090] In this embodiment, by comparing and judging the distances of two obstacle avoidance paths, the robot can select a shorter obstacle avoidance distance, thereby facilitating the robot to intelligently determine whether to pass between two obstacles or to cross two obstacles at once to return to the predetermined route. In this scheme, multiple obstacles can be compared simultaneously in this way, thereby facilitating the robot to select a shorter obstacle avoidance path between multiple obstacles for obstacle avoidance transportation, thereby further improving the robot's transportation efficiency.

[0091] See also Figure 1-Figure 2 The present invention provides a control method based on a robot controller, comprising the following steps:

[0092] Step 1: Scan the robot's surrounding environment through the laser sensor installed on the top of the robot, establish a three-dimensional space model of the environment, obtain the factory workshop map information, and mark the robot's current position in the map information by comparing the map information with the three-dimensional space model information of the environment to obtain the robot positioning information;

[0093] Step 2: Obtain images taken by multiple cameras in the factory workshop, calibrate and correct the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrate and correct the robot positioning information through the images taken by multiple cameras;

[0094] Step 3: Obtain the length, height and width of the robot, set the obstacle safety distance, obtain the calibrated three-dimensional space model of the environment, obtain the calibrated robot positioning information, obtain the robot's predetermined path, determine the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulate the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path;

[0095] Step 4: The controller transmits data instructions to the servo motor to drive the robot to move forward, backward and rotate on the spot according to the predetermined path and obstacle avoidance path.

[0096] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented using software, the above embodiments may be implemented in whole or in part in the form of a computer program product. A person of ordinary skill in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution.

[0097] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0098] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A control system based on a robot controller, characterized in that: Includes the following modules: The identification and positioning module uses a laser sensor installed on the top of the robot to scan the robot's surrounding environment, establish a three-dimensional spatial model of the environment, obtain factory workshop map information, and compare the map information with the three-dimensional spatial model information of the environment to mark the robot's current position in the map information to obtain the robot's positioning information; The information calibration module obtains images taken by multiple cameras in the factory workshop, calibrates and corrects the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrates and corrects the robot positioning information through the images taken by multiple cameras; The path decision module obtains the length, height and width of the robot, sets the obstacle safety distance, obtains the calibrated three-dimensional space model of the environment, obtains the calibrated robot positioning information, obtains the robot's predetermined path, determines the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulates the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path; The control drive module transmits data instructions to the servo motor through the controller, driving the robot to move forward, backward and rotate on the spot according to the predetermined path and obstacle avoidance path; The three-dimensional space model of the environment is established in the recognition and positioning module, specifically: Step 1: Set the scanning resolution, scanning angle range and point cloud density, execute the scanning task and scan the surrounding environment three times to obtain three sets of environmental 3D point clouds respectively; Step 2: Compare the three sets of environmental 3D point clouds, mark all common point clouds in the three sets of environmental 3D point clouds to obtain an environmental 3D point cloud set, establish a 3D space coordinate system with the shooting coordinates as the origin, mark the 3D coordinates of the environmental 3D point cloud set, mark the position of the 3D point cloud set in the 3D space coordinate system, perform triangulation and surface fitting on the 3D point cloud set in the coordinate system to achieve surface reconstruction of the environmental 3D space model, and finally obtain the environmental 3D space model; In the recognition and positioning module, the robot positioning information is obtained by comparing the map information with the three-dimensional space model information of the environment, marking the current position of the robot in the map information, specifically: Step 1: Identify multiple map information and establish a map 3D space model through common points in the multiple map information, convert the map 3D space model into a map 3D coordinate system point cloud set, obtain an environment 3D point cloud set, compare the map 3D point cloud set with the environment 3D point cloud set, and mark all common points in the map 3D point cloud set; Step 2: Obtain the common point cloud cluster with the largest density, restore the three-dimensional space model of the map at the common point cloud cluster to obtain the reference landmark, obtain the coordinates of the reference landmark, and obtain the robot positioning coordinate information through the Euclidean distance formula in the environment three-dimensional space coordinate system, the map three-dimensional space coordinate system, the origin coordinates of the environment three-dimensional space coordinate system and the reference landmark coordinates.

2. A control system based on a robot controller according to claim 1, characterized in that: In the information calibration module, the three-dimensional space model of the environment is corrected by shooting pictures with multiple cameras, specifically: Step 1: obtain multiple pictures taken by cameras at different positions, establish a shooting 3D space model using the MVS algorithm through multiple groups of shooting pictures, obtain the environment 3D space model, compare the environment 3D space model with the shooting 3D space model, and mark the common models in the environment 3D space model and the shooting 3D space model, the unique models in the shooting 3D space model, and the unique models in the environment 3D space model; Step 2: copy the unique model in the photographed three-dimensional space model to the environmental three-dimensional space model according to the coordinate position relationship of the shared model to obtain a proofread environmental three-dimensional space model, and mark the unique model in the environmental three-dimensional space model; Step 3: Scan the surrounding environment again through the identification and positioning module to confirm whether the unique model in the marked environmental three-dimensional space model actually exists. If the unique model in the marked environmental three-dimensional space model actually exists, the proofread environmental three-dimensional space model is defined as the final environmental three-dimensional space model. If the unique model in the marked environmental three-dimensional space model does not actually exist, delete the unique model in the environmental three-dimensional space model marked in the proofread environmental three-dimensional space model, and define the deleted environmental three-dimensional space model as the final environmental three-dimensional space model.

3. A control system based on a robot controller according to claim 2, characterized in that: In the information calibration module, the robot positioning information is corrected by shooting pictures with multiple cameras, specifically: Step 1: Obtain the three-dimensional space model of the shooting, take the workshop entrance as the origin coordinate to obtain the three-dimensional space coordinates of the robot in the shooting three-dimensional space model, obtain the three-dimensional space model of the environment, take the workshop entrance as the origin to obtain the three-dimensional space coordinates of the robot in the three-dimensional space model of the environment; Step 2: The positioning error is obtained by subtracting the three-dimensional spatial coordinates of the robot in the captured three-dimensional spatial model from the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment. If the positioning error is 0, the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment are used as the positioning information of the robot as the final positioning information. If the positioning error is not 0, the three-dimensional spatial coordinates of the robot in the captured three-dimensional spatial model are replaced by the three-dimensional spatial coordinates of the robot in the three-dimensional spatial model of the environment to obtain the final positioning information.

4. A control system based on a robot controller according to claim 3, characterized in that: The calculation method of positioning error is as follows: Among them, Wh represents the positioning error, Represented as the X-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the X-axis coordinate of the robot in the three-dimensional space model. Represented as the Y-axis coordinate of the robot in the three-dimensional space model of the environment, It is expressed as the Y-axis coordinate of the robot in the three-dimensional space model. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model of the environment. It is expressed as the Z-axis coordinate of the robot in the three-dimensional space model.

5. A control system based on a robot controller according to claim 1, characterized in that: In the path decision module, the starting point and end point of the robot's obstacle avoidance path in the predetermined path are determined, specifically: Step 1: Set the emergency stop safety distance when scanning obstacles, define the position of the robot after emergency stop as the starting point of the obstacle avoidance path, obtain the calibrated three-dimensional space model of the environment, mark all obstacles in front of the robot in the predetermined path, and measure the distance and limit height of every two adjacent obstacles; Step 2: Get the length, height and width of the robot, set the obstacle safety distance, sum the obstacle safety distance with the length, height and width of the robot to get the safety length, safety height and safety width respectively, compare the size relationship between the limit height and the safety height, if the safety height is less than the limit height, mark it as passable in height, if the safety height is greater than the limit height, mark it as inaccessible in height, compare the size relationship between the safety length and the safety width, select the smaller one as the passing direction, compare the size relationship between the length of the passing direction and the obstacle spacing, if the length of the passing direction is less than the obstacle spacing, mark it as passable in spacing, if the length of the passing direction is greater than the obstacle spacing, mark it as inaccessible in spacing; Step 3: Determine whether the robot can pass between two adjacent obstacles. If the height is acceptable and the distance between the obstacles is acceptable, it is marked as eventually passable. Otherwise, it is marked as eventually inaccessible. The intersection of the passing path and the predetermined path is taken as the end point of the obstacle avoidance path.

6. A control system based on a robot controller according to claim 5, characterized in that: In the path decision module, the robot's obstacle avoidance path is formulated according to the starting point and end point of the obstacle avoidance path, specifically: Step 1: If the end point of the obstacle avoidance path is not between two obstacles, measure the distance from the starting point of the obstacle avoidance path to the left and right sides of the obstacle respectively, and mark them as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the obstacle from the left. , the robot will bypass the obstacle from the right side; Step 2: If the end point of the obstacle avoidance path is between two obstacles, the end point of the obstacle avoidance path passing through the second obstacle is defined as the virtual end point. The distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle is measured and marked as and , respectively measure the distance from the end point of the obstacle avoidance path to the left and right sides of the first obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the first obstacle. , the robot will bypass the right side of the first obstacle and measure the distance from the end point of the obstacle avoidance path to the left and right sides of the second obstacle, respectively, marked as and , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right and The sum is ,right and The sum is ,judge and If , the robot will bypass the left side of the second obstacle. , the robot will go around the right side of the second obstacle and finally get the first obstacle avoidance path; Step 3: If the end point of the obstacle avoidance path is between two obstacles, define the end point of the obstacle avoidance path passing through the second obstacle as the virtual end point, and measure the distance from the starting point of the obstacle avoidance path to the left and right sides of the first obstacle, respectively, and mark them as and , measure the distance from the left side of the first obstacle to the left side of the second obstacle, marked as , measure the distance from the right side of the first obstacle to the right side of the second obstacle, marked as , respectively measure the distance from the virtual end point of the obstacle avoidance path to the left and right sides of the second obstacle, marked as and ,right , and Ask for ,right , and Ask for ,Compare and If , the robot will go around the left side of the two obstacles and directly reach the virtual end point. , the robot will go around the right side of the two obstacles and directly reach the virtual end point, and finally obtain the second obstacle avoidance path; Step 4: Get the total length of the first obstacle avoidance path , get the total length of the second obstacle avoidance path ,Compare and If , then the first obstacle avoidance path is selected as the final obstacle avoidance path. , then the second obstacle avoidance path is selected as the final obstacle avoidance path.

7. A control method based on a robot controller, applied to a control system based on a robot controller as claimed in any one of claims 1 to 6, characterized in that: The steps include: Step 1: Scan the robot's surrounding environment through the laser sensor installed on the top of the robot, establish a three-dimensional space model of the environment, obtain the factory workshop map information, and mark the robot's current position in the map information by comparing the map information with the three-dimensional space model information of the environment to obtain the robot positioning information; Step 2: Obtain images taken by multiple cameras in the factory workshop, calibrate and correct the three-dimensional space model of the environment through the images taken by multiple cameras, and calibrate and correct the robot positioning information through the images taken by multiple cameras; Step 3: Obtain the length, height and width of the robot, set the obstacle safety distance, obtain the calibrated three-dimensional space model of the environment, obtain the calibrated robot positioning information, obtain the robot's predetermined path, determine the starting point and end point of the robot's obstacle avoidance path in the predetermined path, and formulate the robot's obstacle avoidance path according to the starting point and end point of the obstacle avoidance path; Step 4: The controller transmits data instructions to the servo motor to drive the robot to move forward, backward and rotate on the spot according to the predetermined path and obstacle avoidance path.

Images