Robot three-dimensional scanning positioning method and device, electronic equipment and storage medium

Abstract

The application provides a robot three-dimensional scanning positioning method and device, electronic equipment and storage medium. The robot three-dimensional scanning positioning method comprises the following steps: acquiring a standard distance and an initial imaging position, measuring the distance between the end of a mechanical arm and a task target as a preliminary distance, calculating the difference between the standard distance and the preliminary distance to obtain a target offset distance, determining a target imaging position according to the initial imaging position and the target offset distance, controlling the end of the mechanical arm to move to the target imaging position until a point cloud collection device reaches the target imaging position, performing three-dimensional scanning on a mark block through the point cloud collection device to obtain a mark block point cloud, and positioning the task target based on the mark block point cloud. When the position of the task target changes greatly in the depth direction, the target imaging position can be adaptively adjusted, and the point cloud collection device can reach the target imaging position, thereby effectively avoiding the problem of mark block defocusing, ensuring the accuracy of the mark block point cloud, and improving the accuracy of the positioning of the task target.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot positioning, and in particular to a robot three-dimensional scanning positioning method and device, electronic equipment and storage medium. BACKGROUND

[0002] There are many tasks with harsh working environments such as high temperature and high dust in the metallurgical field, such as replacing ladle oil cylinders and medium pipelines in continuous casting areas, which are usually performed by robots instead of manual work. The existing visual devices basically use ordinary optical cameras and laser cameras, but in the harsh environment of metallurgy, the light changes dramatically under the influence of molten metal, and the optical camera is easily affected, so when precise spatial position sensing is required, a laser camera is generally used. Under the influence of hoisting position deviation, mechanical structure error and other reasons, the target of many processes in the metallurgical process is not fixed, but the imaging range of the laser camera is relatively narrow, generally within a few tens of centimeters, if a laser camera with a larger imaging range is selected, there are disadvantages such as too large camera volume and high price. When the target position changes greatly in the depth of field direction, it is impossible to determine whether the distance between the camera and the target is within the effective imaging distance, and fixed position shooting scanning may exist out of focus. SUMMARY

[0003] In view of the above-mentioned shortcomings of the prior art, the present application provides a robot three-dimensional scanning positioning method, device, electronic equipment and storage medium to solve the technical problem that when the position of the task target to be positioned changes greatly in the depth of field direction, fixed position shooting scanning may exist out of focus.

[0004] The robot three-dimensional scanning positioning method provided by the present application comprises: acquiring a standard distance and an initial imaging position, and measuring the distance between the end of a mechanical arm and a task target as a preliminary distance; calculating the difference between the standard distance and the preliminary distance to obtain a target offset distance, and determining a target imaging position according to the initial imaging position and the target offset distance; controlling the end of the mechanical arm to move to the target imaging position until a point cloud collection device reaches the target imaging position, the point cloud collection device being arranged on the end of the mechanical arm; performing three-dimensional scanning on a mark block by the point cloud collection device to obtain mark block point clouds, the mark block being arranged on the task target; and positioning the task target based on the mark block point clouds.

[0005] In an embodiment of the present application, the distance between the end of the mechanical arm and the task target is measured as a preliminary distance, including: in response to a task instruction, controlling the end of the mechanical arm to move to a preset distance measuring position until a distance measuring device reaches the preset distance measuring position, the distance measuring device being arranged at the end of the mechanical arm; sending a distance measuring instruction to make the distance measuring device measure an end plane distance between the end of the mechanical arm and a target plane arranged on the task target, and taking the end plane distance as the preliminary distance.

[0006] In an embodiment of the present application, after the task target is positioned based on the landmark block point cloud, the method includes: in response to a next task instruction, if a next task target is different from the task target, matching a next preset distance measuring position and a next initial imaging position corresponding to the next task target, the next task target being determined based on the next task instruction, the next task instruction including identification information of the next task target; controlling the end of the mechanical arm to move to the next preset distance measuring position until the distance measuring device reaches the next preset distance measuring position; sending a next distance measuring instruction to make the distance measuring device measure a next end plane distance between the end of the mechanical arm and a target plane of the next task target, and taking the next end plane distance as a next preliminary distance; performing difference calculation on the standard distance and the next preliminary distance to obtain a next target offset distance, determining a next target imaging position according to the next initial imaging position and the next target offset distance; controlling the end of the mechanical arm to move to the next target imaging position until the point cloud acquisition device reaches the next target imaging position; performing three-dimensional scanning on the landmark block of the next task target through the point cloud acquisition device to obtain a next landmark block point cloud; and positioning the next task target based on the next landmark block point cloud.

[0007] In an embodiment of the present application, the positioning of the task target based on the landmark block point cloud includes: performing segmentation on the landmark block point cloud according to an image segmentation method to obtain point clouds of at least three target objects, the target objects being arranged on the landmark block, and the target centers of the target objects being on different straight lines; performing calculation on the point clouds of the target objects according to a grid search algorithm to obtain target center coordinate positions of the target objects; determining three reference target objects from the at least three target objects, calculating a center coordinate position of the task target based on the target center coordinate positions of the three reference target objects and a preset relative distance between the target centers of the three reference target objects and the center of the task target, and calculating a normal vector of the task target based on the target center coordinate positions of the three reference target objects to obtain a pose of the task target.

[0008] In an embodiment of the present application, after the end of the mechanical arm moves to the target imaging position, the method further comprises: sending a scanning instruction to make the point cloud acquisition device perform multiple three-dimensional scans on the mark block to obtain multiple sets of mark block point clouds; and repeatedly positioning the task target based on the multiple sets of mark block point clouds.

[0009] In an embodiment of the present application, the target object includes any one of a target hole, a target ball or a target block.

[0010] In an embodiment of the present application, the ranging device is a laser range finder, and the point cloud acquisition device is a three-dimensional laser camera, the laser range finder and the three-dimensional laser camera are arranged in a protective box, and the protective box is used for introducing external cooling air.

[0011] In an embodiment of the present application, a robot three-dimensional scanning and positioning device is further provided, which includes: a ranging module configured to obtain a standard distance and an initial imaging position, and measure a distance between an end of a mechanical arm and a task target as a preliminary distance; a first processing module configured to calculate a difference between the standard distance and the preliminary distance to obtain a target offset distance, and determine a target imaging position according to the initial imaging position and the target offset distance; a scanning module configured to control the end of the mechanical arm to move to the target imaging position until a point cloud acquisition device reaches the target imaging position, perform a three-dimensional scan on a mark block by the point cloud acquisition device to obtain a mark block point cloud, the point cloud acquisition device is arranged at the end of the mechanical arm, and the mark block is arranged on the task target; and a second processing module configured to position the task target based on the mark block point cloud.

[0012] In an embodiment of the present application, an electronic device is further provided, which includes: one or more processors; and a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the electronic device is caused to implement the robot three-dimensional scanning and positioning method as described above.

[0013] In an embodiment of the present application, a computer readable storage medium having a computer program stored thereon is further provided, when the computer program is executed by a processor of a computer, the computer is caused to execute the robot three-dimensional scanning and positioning method as described above.

[0014] The beneficial effects of the present application: the present application provides a robot three-dimensional scanning positioning method, device, electronic equipment and storage medium, the robot three-dimensional scanning positioning method measures the distance between the end of the mechanical arm and the task target as the preliminary distance, calculates the target imaging position according to the preliminary distance, the standard distance and the initial imaging position, and controls the movement of the mechanical arm, so that the point cloud collection device at the end of the mechanical arm reaches the target imaging position, and then scans the mark block, which can realize adaptive adjustment of the target imaging position when the task target position changes greatly in the depth direction, and set the point cloud collection device at the end of the mechanical arm, drive the point cloud collection device to reach the target imaging position by controlling the movement of the mechanical arm, effectively avoid the problem that the mark block is out of focus when the target position changes greatly in the depth direction, ensure the accuracy of the mark block point cloud, and improve the accuracy of the task target positioning. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flow chart of a robot three-dimensional scanning positioning method according to an example embodiment of the present application;

[0016] Figure 2 is a schematic diagram of robot ranging according to a specific embodiment of the present application;

[0017] Figure 3 is a schematic diagram of the imaging range of a laser camera according to a specific embodiment of the present application;

[0018] Figure 4 is a schematic diagram of the movement of the end of the mechanical arm to the target imaging position according to a specific embodiment of the present application;

[0019] Figure 5 is a mark block point cloud diagram according to a specific embodiment of the present application;

[0020] Figure 6 is a block diagram of a robot three-dimensional scanning positioning device according to an example embodiment of the present application;

[0021] Figure 7 is a schematic diagram of a robot three-dimensional scanning positioning device according to another example embodiment of the present application;

[0022] Figure 8 is a schematic diagram of the end of the mechanical arm according to a specific embodiment of the present application.

[0023] Reference signs: 1-mechanical arm; 2-flange; 3-photographing module; 4-ranging module; 5-mark block; 6-target plane; 7-execution module; 8-task target. DETAILED DESCRIPTION

[0024] Following make the embodiments of the present application by specific examples, the person skilled in the art can easily understand the other advantages and effects of the present application from the disclosure of the specification. The present application can also be implemented or applied by other different specific embodiments, and the details in the specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following examples and features in the examples can be combined with each other without conflict.

[0025] It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the present application in a schematic manner, and only the components related to the present application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component may be a random change, and the component layout pattern may be more complex.

[0026] It should be noted that in the present application, "first", "second", etc. are only for the differentiation of similar objects, and are not limited in order or sequence. The described "includes", "has", etc. means that the subject of the word covers the range in addition to the examples shown by the word, and is not exclusive.

[0027] It can be understood that the various numbers, step numbers, etc. in the present application are distinguished for convenience of description, and are not used to limit the scope of the present application. The size of the present application does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic.

[0028] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present application, however, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details, and in other embodiments, the known structures and devices are shown in the form of block diagrams rather than in detail, to avoid making the embodiments of the present application difficult to understand.

[0029] It should be noted that in the metallurgical field operation, when the robot replaces manual tasks, when the target position of the task changes greatly in the direction of the camera depth of field, the laser scanning camera scanning range is too narrow, resulting in out-of-focus recognition failure. Although the laser scanning camera with large depth of field can avoid the above problems to some extent, but the laser scanning camera with large depth of field is usually large in size, heavy in load, and more expensive in price.

[0030] To solve these problems, embodiments of the present application respectively provide a robot three-dimensional scanning positioning method, a robot three-dimensional scanning positioning device, an electronic device, a computer readable storage medium and a computer program product, which will be described in detail below.

[0031] Referring to Figure 1 , Figure 1 is a flow chart of a robot three-dimensional scanning positioning method shown in an exemplary embodiment of the present application. As Figure 1 shown, in an exemplary embodiment, the robot three-dimensional scanning positioning method comprises at least steps S110 to S150, which are described in detail as follows:

[0032] Step S110, acquire the standard distance and the initial imaging position, and measure the distance between the end of the mechanical arm and the task target as the preliminary distance.

[0033] In an embodiment of the present application, the task target refers to the task target corresponding to the task performed by the robot, for example, when replacing the oil cylinder, the oil cylinder is inserted into the groove of the steel ladle, and then the groove of the steel ladle is the task target. Before the robot performs the task, it needs to perceive the position of the task target in space. First, the standard distance and the initial imaging position are acquired, and the distance between the end of the mechanical arm and the task target is measured, which is named as the preliminary distance. The robot in this embodiment is a mechanical arm, which can be a six-degree-of-freedom mechanical arm. The standard distance refers to a pre-set distance threshold between the end of the mechanical arm and the task target. The initial imaging position refers to an ideal initial imaging position of the point cloud acquisition device. It should be noted that the robot performs the task through an execution mechanism, for example, by grabbing the oil cylinder through the execution mechanism, and the execution mechanism is arranged on the end of the mechanical arm.

[0034] In an embodiment of the present application, measuring the distance between the end of the mechanical arm and the task target as the preliminary distance comprises the following steps:

[0035] Step S111, in response to the task instruction, controlling the end of the mechanical arm to move to the pre-set distance measuring position until the distance measuring device reaches the pre-set distance measuring position.

[0036] In an embodiment of the present application, the distance measuring device is arranged on the end of the mechanical arm. After receiving the task instruction, the robot plans a motion trajectory according to the pre-set distance measuring position, and controls the motion of the mechanical arm according to the motion trajectory, so that the distance measuring device on the end of the mechanical arm reaches the pre-set distance measuring position.

[0037] Step S112, sending a distance measuring instruction to make the distance measuring device measure the end plane distance between the end of the mechanical arm and the target plane, and taking the end plane distance as the preliminary distance.

[0038] In one embodiment of the present application, the target plane is arranged on the task target. When the ranging device at the end of the mechanical arm reaches the preset ranging position, the robot sends a ranging instruction to cause the ranging device to measure the end plane distance between the end of the mechanical arm and the target plane, and take the end plane distance as the preliminary distance between the end of the mechanical arm and the task target. The ranging device can be any one of a laser ranging device, an infrared ranging device and an ultrasonic ranging device. The target plane is fixedly arranged at the bottom of the task target, and the target plane can also be fixedly arranged at other positions on the task target that can be measured by the ranging device. It should be noted that, since the position of the task target is not fixed, the size of the target plane should be large enough to ensure the accuracy of the measured distance. The specific size of the target plane is determined according to the specific task to be performed. For example, for replacing the ladle oil cylinder, since the position of the ladle stops each time has a deviation, the width of the target plane can be greater than or equal to 30 cm, and the height can be greater than or equal to 10 cm. In addition, the robot can directly send a ranging instruction to the ranging device, or send a ranging instruction to the industrial computer to control the ranging device to measure the end plane distance between the end of the mechanical arm and the target plane. This is not limited here.

[0039] Please refer to Figure 2 , Figure 2 is a schematic diagram of robot ranging according to one specific embodiment of the present application. As shown in Figure 2 , the black straight line represents the laser emitted by the laser ranging device 4. The laser ranging device 4 is fixedly arranged on the flange 2 at the end of the mechanical arm 1. When the robot receives a task execution instruction, it first controls the end of the mechanical arm 1 to move to a preset ranging position, which is an artificially taught fixed point. When the end of the mechanical arm 1 is located at the point, the laser emitted by the laser ranging device 4 is perpendicular to the target plane 6, and the robot sends a ranging instruction to cause the industrial computer to read the distance between the target plane 6 and the end of the robot (end plane distance) fed back by the laser ranging device 4 at this time.

[0040] In step S120, the standard distance and the preliminary distance are subtracted to obtain a target offset distance, and the target imaging position is determined according to the initial imaging position and the target offset distance.

[0041] In one embodiment of the present application, since the task target can change greatly in the depth of field direction, in order to avoid the marker block from being out of focus, the target imaging position should be changed accordingly. The difference between the standard distance and the preliminary distance is calculated, and the difference is taken as the target offset distance. The initial imaging position is corrected using the target offset distance to obtain the robot end coordinates of the current ideal shooting point (target imaging position).

[0042] Taking the replacement of the oil cylinder as an example, a preliminary distance d between the end of the mechanical arm and the task target groove is obtained by measurement, the preliminary distance d is compared with a stored standard distance d0, a target offset distance Δd is obtained, and a target imaging position is calculated according to the target offset distance Δd and an initial imaging position, and the calculation manner is as follows:

[0043]

[0044] Wherein, (x, y, z) is the initial imaging position, (x', y', z') is the target imaging position, R is the distance from the center of the rotary axis of the continuous casting rotary table to the storage position of the ladle oil cylinder, and Δd is the target offset distance.

[0045] It should be noted that the calculation process of the target imaging position can be completed by the robot or the industrial computer, and the present application is not limited in this regard.

[0046] In step S130, the end of the mechanical arm is controlled to move to the target imaging position until the point cloud acquisition device reaches the target imaging position.

[0047] In an embodiment of the present application, the point cloud acquisition device is arranged at the end of the mechanical arm, the robot plans a motion trajectory according to the target imaging position, controls the motion of the mechanical arm according to the motion trajectory, so that the end of the mechanical arm moves to the target imaging position until the point cloud acquisition device reaches the target imaging position.

[0048] In step S140, the marker block is three-dimensionally scanned by the point cloud acquisition device to obtain a marker block point cloud.

[0049] In an embodiment of the present application, the marker block is arranged on the task target and is used to calibrate the positional relationship of the task target. After the point cloud acquisition device at the end of the mechanical arm reaches the target imaging position, the marker block is located at the optimal imaging distance of the point cloud acquisition device, the robot can send a scanning instruction to make the point cloud acquisition device three-dimensionally scan the marker block to generate a marker block point cloud. The point cloud acquisition device can be any one of a three-dimensional laser camera, a laser radar, etc. The marker block is fixedly arranged at the top of the task target, the marker block can also be fixedly arranged at other positions on the task target which can be scanned by the point cloud acquisition device, and the number of the marker blocks can be one or more. In addition, the robot can directly send the scanning instruction to the point cloud acquisition device, or can send the scanning instruction to the industrial computer to control the point cloud acquisition device to three-dimensionally scan the marker block, and the present application is not limited in this regard.

[0050] Please refer to Figure 3 , Figure 3 is a schematic diagram of the imaging range of the laser camera according to an embodiment of the present application. As shown in Figure 3As shown, the effective imaging range of the laser camera is between the maximum imaging distance and the minimum imaging distance, and generally there is an optimal imaging distance in the range, which is calibrated by the camera manufacturer and provided to the user, and at the optimal imaging distance, the most accurate point cloud can be obtained by scanning.

[0051] Before responding to the task instruction, the ranging point (preset ranging position) is fixed by manual teaching, and the robot is taught to range. When the task target is static at the initial position, the end of the mechanical arm is controlled to move to the preset ranging position until the ranging device at the end of the mechanical arm reaches the preset ranging position. The distance d0 between the end of the mechanical arm and the task target is measured by the ranging device, and d0 is stored as a standard distance. When calibrating the robot program, an ideal shooting point (initial imaging position) is taught. When the point cloud acquisition device is located at the ideal shooting point and the task target is static at the initial position, the marker block is located at the optimal imaging distance of the point cloud acquisition device.

[0052] In an embodiment of the present application, the ranging device is a laser range finder, and the point cloud acquisition device is a three-dimensional laser camera. The laser range finder and the three-dimensional laser camera are arranged in a protective box, and the protective box is used to introduce external cooling air.

[0053] Since the effective range of the laser sensor (laser range finder) is generally 10cm-1200cm, which is much larger than the imaging range of the three-dimensional laser camera (the imaging range of the three-dimensional laser camera is generally 30cm-80cm), even if there is a large position change in the radial range of the task target, it will not exceed the effective range of the laser range finder, providing offset data for adaptive adjustment of the shooting point. In addition, the laser range finder and the three-dimensional laser camera are placed in the protective box and cooled by cooling air, and the material of the protective box can be a high-temperature resistant material to adapt to the high-temperature dust environment on site and prolong the service life of the laser range finder and the three-dimensional laser camera.

[0054] Please refer to Figure 4 , Figure 4 is a schematic diagram of the movement of the end of the mechanical arm to the target imaging position according to an embodiment of the present application. As Figure 4 shown, the end of the mechanical arm 1 is located at the target imaging position, the laser camera 3 (three-dimensional laser camera) is fixed on the flange 2 at the end of the mechanical arm 1, and two virtual straight lines indicate the phase field of the laser camera, showing the imaging range of the laser camera 3. The marker block 5 is located within the imaging range and near the optimal imaging distance. At this time, the robot sends a scanning instruction to start three-dimensional scanning of the marker block 5 by the laser camera 3.

[0055] Step S150, positioning the task target based on the marker block point cloud.

[0056] In an embodiment of the present application, the image segmentation method can be used to process the landmark block point cloud, and the grid search algorithm can be used to calculate the pose of the landmark block, so that the robot can locate the task target according to the pose of the landmark block, and then control the end effector of the robot to perform the task.

[0057] In an embodiment of the present application, step S140 comprises the following steps:

[0058] According to the image segmentation method, the point cloud of the at least three target objects is segmented, the target objects are arranged on the landmark block, and the target centers of the target objects are on different straight lines.

[0059] According to the grid search algorithm, the target center coordinate positions of the target objects are calculated.

[0060] Three reference target objects are determined from the at least three target objects, the center coordinate position of the task target is calculated based on the target center coordinate positions of the three reference target objects and the preset relative distances between the target centers of the three reference target objects and the center of the task target, and the normal vector of the task target is calculated based on the target center coordinate positions of the three reference target objects, so as to obtain the pose of the task target.

[0061] In this embodiment, 3 or more target objects are arranged on the landmark block, the image segmentation method is used to identify and segment the edges of the landmark block point cloud to obtain the point cloud containing all the target objects, the grid search algorithm is used to identify the edges of the target objects, and the target object point cloud formed by each target object is extracted to calculate the target center coordinate position of each target object, so as to obtain the target center coordinate positions of all the target objects. The pose of the task target can be calculated according to the three-point positioning method. Since the relative distances between the target centers of each target object and the center of the task target are fixed, the relative distances between the target centers of each target object and the center of the task target can be measured by the person skilled in the art in advance and stored as the preset relative distances. Three reference target objects can be determined from all the target objects, the center coordinate position of the task target is calculated based on the target center coordinate positions of the three reference target objects and the preset relative distances between the target centers of the three reference target objects and the center of the task target, and the normal vector of the landmark block is calculated based on the target center coordinate positions of the three reference target objects. Since the landmark block is relatively fixed with the task target, the normal vector of the landmark block can be used as the normal vector of the task target. The pose of the task target includes the center coordinate position and the normal vector of the task target.

[0062] It should be noted that the above processing process of the landmark block point cloud can be completed by the robot or by the industrial computer, which is not limited here.

[0063] In one embodiment of the present application, the target object includes any one of a target hole, a target ball or a target block. At least three holes can be formed on the marker block as the target hole, the target hole can be a circular hole or a polygonal hole, or at least three spherical objects can be welded on the marker block as the target ball, or at least three block-shaped objects can be welded on the marker block as the target block, the target block can be a square block, a triangular block or other polygonal blocks, in addition, the target object can also be a circular mark, a square mark, a triangular mark or the like which is obviously different from the color of the marker block, for example, when the color of the marker block is gray white, at least three circles can be marked on the marker block in black, and the centers of the three circles are not on the same straight line.

[0064] The technical scheme of the embodiment can measure the distance between the end of the mechanical arm and the task target as a preliminary distance when the task target position changes greatly in the depth direction, adjust the initial imaging position by the difference between the preliminary distance and the standard distance to obtain a target imaging position, and then control the end of the mechanical arm to move to the target imaging position, so that the marker block of the task target is located at the best imaging distance of the point cloud collection device, and then the marker block of the task target is scanned in three dimensions by the point cloud collection device, even if the task target position changes greatly in the depth direction, the marker block can be kept in focus and not out of focus, the accuracy of the marker block point cloud is ensured, and the positioning accuracy of the task target is improved. Figure 5 , Figure 5 is a marker block point cloud diagram shown in a specific embodiment of the present application. As shown in Figure 5 , the marker block point cloud has three circular holes, and the industrial computer obtains the three-dimensional coordinates of the centers of the three circular holes in space according to the image segmentation method after obtaining the point cloud data (marker block point cloud), and further calculates the three-dimensional coordinates and attitude of the task target in space by using equation (three-point positioning method). Since the scanning is located at the ideal shooting point, i.e. the best imaging distance, the point cloud quality is extremely high, and the calculated position coordinates have high accuracy (up to millimeter level), which can meet the needs of the loading task of the robot.

[0065] In another embodiment of the present application, after step S130, the following steps are further included:

[0066] sending a scanning instruction to make the point cloud collection device perform multiple three-dimensional scanning on the marker block to obtain multiple sets of marker block point clouds;

[0067] repeatedly positioning the task target based on the multiple sets of marker block point clouds.

[0068] In this embodiment, after the point cloud collection device at the end of the mechanical arm reaches the target imaging position, the robot sends a scanning instruction to make the point cloud collection device perform multiple three-dimensional scans on the marker block to obtain multiple sets of marker block point clouds, calculates the target center coordinate positions of the multiple sets of marker block positions according to the above calculation method, each set of marker block position includes the target center coordinate positions of at least three targets, and repeatedly positions the task target based on the multiple sets of marker block positions according to the above calculation method to obtain multiple sets of poses of the task target. The average value or weighted average value of the multiple sets of poses of the task target is calculated, and the calculation result is determined as the accurate pose of the task target, which can further improve the accuracy of positioning the task target. In addition, before calculating the average value or weighted average value, the multiple sets of poses of the task target can also be screened to eliminate the discrete poses of the task target.

[0069] In one embodiment of the application, after step S150, the following steps are included:

[0070] In response to the next task instruction, if the next task target is different from the task target, the next preset ranging position corresponding to the next task target and the next initial imaging position are matched, the next task target is determined based on the next task instruction, and the next task instruction includes identification information of the next task target.

[0071] The end of the mechanical arm is controlled to move to the next preset ranging position until the ranging device reaches the next preset ranging position.

[0072] A next ranging instruction is sent to make the ranging device measure a next end plane distance between the end of the mechanical arm and the target plane of the next task target, and the next end plane distance is taken as a next preliminary distance.

[0073] The standard distance and the next preliminary distance are subjected to difference calculation to obtain a next target offset distance, and a next target imaging position is determined according to the next initial imaging position and the next target offset distance.

[0074] The end of the mechanical arm is controlled to move to the next target imaging position until the point cloud collection device reaches the next target imaging position.

[0075] The marker block of the next task target is subjected to three-dimensional scanning by the point cloud collection device to obtain a next marker block point cloud.

[0076] The next task target is positioned based on the next marker block point cloud.

[0077] In this embodiment, the robot can also perform tasks of different task targets, for example, one robot needs to be responsible for replacing the oil cylinder for multiple steel ladles. When receiving a task instruction, the task target can be determined according to the task instruction. The task instruction includes identification information capable of identifying different task targets. Each task target is preconfigured with a corresponding preset distance measuring position and an initial imaging position. When the next task target determined according to the next task instruction is different from the previous task target (the task target in the foregoing embodiment is taken as the previous task target), the next preset distance measuring position and the next initial imaging position corresponding to the next task target are matched to measure the next preliminary distance between the end of the mechanical arm and the next task target, and the next target imaging position is calculated accordingly. The next target imaging position is used for three-dimensional scanning of the mark block of the next task target based on the next target imaging position to obtain the next mark block point cloud, and the next task target is positioned based on the next mark block point cloud, so that the robot performs the next task. For details of the implementation process, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0078] Please refer to Figure 6 , Figure 6 is a block diagram of a robot three-dimensional scanning and positioning device according to an exemplary embodiment of the present application. As shown in Figure 6 , the exemplary robot three-dimensional scanning and positioning device includes:

[0079] The distance measuring module 610 is configured to obtain a standard distance and an initial imaging position, and measure the distance between the end of the mechanical arm and the task target as a preliminary distance. The first processing module 620 is configured to calculate the target offset distance by performing a difference calculation on the standard distance and the preliminary distance, and determine the target imaging position according to the initial imaging position and the target offset distance. The scanning module 630 is configured to control the end of the mechanical arm to move to the target imaging position until the point cloud acquisition device reaches the target imaging position, and perform three-dimensional scanning on the mark block by the point cloud acquisition device to obtain the mark block point cloud. The point cloud acquisition device is arranged on the end of the mechanical arm, and the mark block is arranged on the task target. The second processing module 640 is configured to position the task target based on the mark block point cloud.

[0080] In another embodiment of the present application, a robot three-dimensional scanning and positioning device is also provided, which includes:

[0081] The mechanical arm is a six-degree-of-freedom mechanical arm. The mark block is arranged on the task target and used for position calibration of the task target. The shooting module (point cloud acquisition device) is arranged on the free end (end) of the mechanical arm and used for shooting the mark block to position the task target. The distance measuring module (distance measuring device) is arranged on the free end of the mechanical arm and used for measuring the distance between the free end of the mechanical arm and the task target.

[0082] In one embodiment of the present application, the device further comprises an execution module (actuator) arranged on the free end of the mechanical arm, for docking with the task target.

[0083] In one embodiment of the present application, a flange is arranged on the free end of the mechanical arm, and the shooting module, the distance measuring module and the execution module are arranged on the flange.

[0084] In one embodiment of the present application, the device further comprises a calculation module, and the shooting module, the distance measuring module and the mechanical arm are connected with the calculation module (industrial computer).

[0085] In one embodiment of the present application, the marker block is provided with at least three target objects, and the center points of the target objects are on different straight lines.

[0086] In one embodiment of the present application, the cross section of the target object is circular.

[0087] In one embodiment of the present application, the device further comprises a target plane, which is arranged on the task target, and the distance measuring module measures the distance from the target plane to determine the distance between the free end of the mechanical arm and the task target.

[0088] In one embodiment of the present application, a protective box is arranged on the mechanical arm, and the distance measuring module and the shooting module are arranged in the protective box, and the protective box is used for introducing external cooling air.

[0089] In one embodiment of the present application, the distance measuring module comprises one of a laser range finder, an infrared range finder and an ultrasonic range finder.

[0090] In one embodiment of the present application, the shooting module comprises a three-dimensional laser camera.

[0091] Please refer to Figure 7 and Figure 8 , Figure 7 is a schematic diagram of a robot three-dimensional scanning and positioning device according to another exemplary embodiment of the present application, Figure 8 is a schematic diagram of the end of a mechanical arm according to one embodiment of the present application. As Figure 7 and Figure 8As shown, the robot three-dimensional scanning positioning device comprises a mechanical arm 1, a flange plate 2, a three-dimensional laser camera 3, a laser range finder 4, a marker block 5, a target plane 6, an actuator 7 and a task target 8, wherein the actuator 7, the three-dimensional laser camera 3 and the laser range finder 4 are fixed on the flange plate 2 at the end of the mechanical arm 1, the marker block 5 is fixedly arranged on the top of the task target 8, the target plane 6 is fixedly arranged on the bottom of the task target 8, and three round holes are formed in the marker block 5. The mechanical arm 1 is a six-degree-of-freedom mechanical arm, so that the end of the mechanical arm moves to a preset ranging position and a target imaging position. In addition, the robot three-dimensional scanning positioning device further comprises an industrial computer, which is used for communicating with the mechanical arm 1, the three-dimensional laser camera 3 and the laser range finder 4, calculating the target imaging position, processing the marker block point cloud and the like, and the industrial computer is omitted in the figure. Figure 7 In this embodiment, the robot will grab the oil cylinder into the task target 8, i.e. the groove on the ladle, which will be offset in x, y, z directions and pose angle r x , r y , r z with the position of the ladle loaded and the rotation error of the ladle turret, and the embodiment of the present application uses the laser range finder and the three-dimensional laser camera for secondary sampling to accurately identify the specific position and pose angle of the groove in space, so as to accurately insert the oil cylinder into the groove.

[0092] Through the secondary information collection of the laser range finder and the three-dimensional laser scanning camera, the adaptability and accuracy of point cloud modeling and positioning are effectively improved. First, the relative distance (initial distance) between the end of the robot (the end of the mechanical arm) and the task target is fed back by the laser range finder, then the position of the robot for taking a picture is adaptively adjusted, the end of the mechanical arm moves to an ideal picture taking position, and the three-dimensional scanning modeling of the marker block of the task target is completed by the three-dimensional laser scanning camera to obtain the marker block point cloud, so as to calculate the accurate position and pose of the task target in space. Since the best picture taking distance is adopted, the accuracy can reach millimeter level. The device can effectively cope with the situation that the imaging range of the laser scanning camera is too narrow to cause out-of-focus recognition failure when the target position changes greatly in the depth direction.

[0093] It should be noted that the robot three-dimensional scanning positioning device provided in the above embodiment and the robot three-dimensional scanning positioning method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, which will not be repeated here. In actual application, the above functions can be distributed to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above, and this is not limited herein.

[0094] The embodiment also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the robot three-dimensional scanning positioning method provided in each of the above embodiments.

[0095] The embodiment also provides a computer readable storage medium, which stores a computer program, and the computer program, when executed by a processor of a computer, causes the computer to perform the robot three-dimensional scanning positioning method as described above. The computer readable storage medium can be included in the electronic device described in the above embodiments, or can exist separately and not be assembled into the electronic device.

[0096] The embodiment also provides a computer program product or computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device performs the robot three-dimensional scanning positioning method provided in each of the above embodiments.

[0097] The electronic device provided in the embodiment comprises a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected with the processor and the transceiver and complete communication between each other. The memory is used for storing a computer program, and the communication interface is used for communication. The processor and the transceiver are used for running the computer program, so that the electronic device performs each step of the method as described above.

[0098] In the embodiment, the memory can include a random access memory (RAM) and can also include a non-volatile memory, for example, at least one disk memory.

[0099] The processor described above can be a general processor, including a central processing unit (CPU), a network processor (NP) and the like; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component.

[0100] Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by computer program related hardware. The foregoing computer program can be stored in a computer readable storage medium. The program executes the steps of the foregoing method embodiments when executed; and the foregoing storage medium includes ROM (read only memory), RAM (random access memory), magnetic disk or optical disk, and various media that can store program codes.

[0101] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not intended to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by those skilled in the art without departing from the spirit and technical ideas disclosed by the present application should be covered by the claims of the present application.

Claims

1. A robot three-dimensional scanning and positioning method, characterized in that: The method comprises: Obtain a standard distance and an initial imaging position, wherein the standard distance is the distance between the end of the robotic arm and the task target measured by a distance measuring device when the task target is stationary at the initial position, and the initial imaging position is the ideal photographing point taught when calibrating the robot program. When the point cloud acquisition device is located at the ideal photographing point and the task target is stationary at the initial position, the marker block is located at the optimal imaging distance of the point cloud acquisition device; Measure the distance between the end of the robotic arm and the mission target as the preliminary distance; performing a difference calculation between the standard distance and the preliminary distance to obtain a target offset distance, and determining a target imaging position according to the initial imaging position and the target offset distance; Controlling the end of the robotic arm to move toward the target imaging position until a point cloud acquisition device reaches the target imaging position, wherein the point cloud acquisition device is disposed at the end of the robotic arm; Performing a three-dimensional scan on the marker block by the point cloud acquisition device to obtain a point cloud of the marker block, wherein the marker block is set on the task target; The task target is positioned based on the marker block point cloud, wherein the marker block point cloud is segmented according to an image segmentation method to obtain point clouds of at least three targets, the targets are set on the marker block, and the bull's eyes of the targets are on different straight lines; the point cloud of the targets is calculated according to a grid search algorithm to obtain the coordinate position of the bull's eye of the targets; three reference targets are determined from the at least three targets, and the center coordinate position of the task target is calculated based on the coordinate positions of the bull's eyes of the three reference targets and the preset relative distances between the bull's eyes of the three reference targets and the center of the task target, and the normal vector of the task target is calculated based on the coordinate positions of the bull's eyes of the three reference targets to obtain the posture of the task target.

2. The robot three-dimensional scanning and positioning method according to claim 1, characterized in that: Measure the distance between the end of the robotic arm and the mission target as a preliminary distance, including: In response to a task instruction, the end of the manipulator arm is controlled to move toward a preset distance measurement position until a distance measurement device reaches the preset distance measurement position, wherein the distance measurement device is provided at the end of the manipulator arm; A distance measurement instruction is sent to enable the distance measurement device to measure the end plane distance between the end of the robot arm and the target plane, and the end plane distance is used as the preliminary distance. The target plane is set on the task target.

3. The robot three-dimensional scanning and positioning method according to claim 2, characterized in that: After locating the task target based on the marker block point cloud, the method includes: In response to a next task instruction, if the next task target is different from the task target, matching a next preset ranging position and a next initial imaging position corresponding to the next task target, the next task target being determined based on the next task instruction, the next task instruction including identification information of the next task target; Controlling the end of the robotic arm to move toward the next preset distance measurement position until the distance measurement device reaches the next preset distance measurement position; Sending a next distance measurement instruction to enable the distance measurement device to measure a next end plane distance between the end of the manipulator and the target plane of the next task target, and using the next end plane distance as a next preliminary distance; performing a difference calculation between the standard distance and the next preliminary distance to obtain a next target offset distance, and determining a next target imaging position according to the next initial imaging position and the next target offset distance; Controlling the end of the robotic arm to move toward the next target imaging position until the point cloud acquisition device reaches the next target imaging position; Performing a three-dimensional scan on the marker block of the next task target by the point cloud acquisition device to obtain a point cloud of the next marker block; The next task target is located based on the next marker block point cloud.

4. The robot three-dimensional scanning and positioning method according to claim 1, characterized in that: Controlling the end of the robotic arm to move toward the target imaging position until the point cloud acquisition device reaches the target imaging position, the method further comprising: Sending a scanning instruction to enable the point cloud acquisition device to perform multiple three-dimensional scans on the marker block to obtain multiple groups of marker block point clouds; The task target is repeatedly positioned based on the multiple groups of marker block point clouds.

5. The robot three-dimensional scanning and positioning method according to claim 1, characterized in that: The target object includes any one of a target hole, a target ball or a target block.

6. The robot three-dimensional scanning and positioning method according to any one of claims 2 or 3, characterized in that: The distance measuring device is a laser rangefinder, the point cloud acquisition device is a three-dimensional laser camera, the laser rangefinder and the three-dimensional laser camera are arranged in a protective box, and the protective box is used to allow external cold air to enter.

7. A robot three-dimensional scanning and positioning device, characterized in that: The device comprises: A distance measurement module is configured to obtain a standard distance and an initial imaging position, wherein the standard distance is the distance between the end of the robotic arm and the task target measured by a distance measurement device when the task target is stationary at the initial position; the initial imaging position is the ideal photographing point taught when calibrating the robot program; when the point cloud acquisition device is located at the ideal photographing point and the task target is stationary at the initial position, the marker block is located at the optimal imaging distance of the point cloud acquisition device; and the distance between the end of the robotic arm and the task target is measured as the preliminary distance; a first processing module, configured to perform a difference calculation between the standard distance and the preliminary distance to obtain a target offset distance, and determine a target imaging position according to the initial imaging position and the target offset distance; a scanning module, configured to control the end of the robotic arm to move toward the target imaging position until the point cloud acquisition device reaches the target imaging position, and perform a three-dimensional scan of the marker block by the point cloud acquisition device to obtain a point cloud of the marker block; The second processing module is used to locate the task target based on the marker block point cloud, wherein the marker block point cloud is segmented according to an image segmentation method to obtain point clouds of at least three targets, the targets are set on the marker block, and the centers of the targets are on different straight lines; the point cloud of the targets is calculated according to a grid search algorithm to obtain the coordinate position of the center of the targets; three reference targets are determined from the at least three targets, and the center coordinate position of the task target is calculated based on the coordinate positions of the centers of the three reference targets and the preset relative distances between the centers of the three reference targets and the center of the task target, and the normal vector of the task target is calculated based on the coordinate positions of the centers of the three reference targets to obtain the posture of the task target.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the robot three-dimensional scanning and positioning method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the robot three-dimensional scanning and positioning method according to any one of claims 1 to 6.

Images