Visual positioning method and system for mechanical arm machining

Abstract

The invention provides a visual positioning method and system for mechanical arm machining, and relates to the technical field of visual image processing.The translation deviation and rotation deviation of a workpiece are obtained by photographing the workpiece twice before and after operation through a depth camera, and a deviation model is trained in combination with positioning information of a mechanical arm; providing position information and posture information of a plurality of workpieces for the manipulator according to translation deviation and rotation deviation output by the deviation model, scoring the translation deviation and rotation, and forming positioning information by adopting the translation deviation and rotation deviation with the lowest score; the system comprises an information collection module, a model training module, a calculation statistics module and an alarm module, more accurate positioning information is provided for the manipulator, and an alarm is given for failed operation of the manipulator.

Description

Technical Field

[0001] The present invention relates to the technical field of visual image processing, and in particular to a visual positioning method and system for robot processing. Background Art

[0002] In the robot processing process, positioning accuracy is one of the key factors affecting processing quality and efficiency. Traditional robot positioning methods usually rely on a fixed coordinate system and precise control of the robot arm. However, due to the positioning error of the workpiece, posture changes and the operation error of the robot itself in the actual processing environment, the positioning accuracy is often not ideal. Especially in complex processing tasks, the error will further accumulate and affect the final processing effect. In addition, most of the visual positioning methods in the prior art lack an effective feedback mechanism and cannot correct errors in real time, which may cause position deviations of the robot during the processing process, thereby affecting the accuracy of the processing.

[0003] In the prior art, publication number CN202410757152.X discloses a visual positioning method and system for robot processing, which obtains an image of the area to be grasped, and obtains the image to be detected based on the image of the area to be grasped; determines the probability index that the edge pixel points of the connected domain in the image to be detected belong to the workpiece based on the edge features of the workpiece in the image to be detected; determines the area where the workpiece is located in the image to be detected based on the probability index that the edge pixel points of the connected domain in the image to be detected belong to the workpiece; determines the grasping priority index of the workpiece in the image to be detected based on the grayscale value of the area where the workpiece is located in the image to be detected; determines the workpiece with the largest grasping priority index as the current grasping target, and controls the robot to grasp the current grasping target.

[0004] Although the disclosed technical document realizes the positioning and operation of the workpiece by the robot, it cannot avoid the influence of the error of the robot itself on the operation of the workpiece, and there is no judgment and warning when the robot operates incorrectly.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0006] The purpose of the present invention is to provide a visual positioning method and system for robot processing to solve the problems raised in the above background technology.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A visual positioning method for robot processing, the specific steps include:

[0009] Step 1: Take a photo of the workpiece before the robot operates it, obtain the first position information and the first posture information, and obtain the positioning information of the robot for the workpiece. After the robot completes the operation, take a photo of the workpiece again to obtain the second position information and the second posture information. Use the IPC algorithm to obtain the translation deviation and rotation deviation of the workpiece in the two photos, and add the robot positioning information to the deviation data set formed by the translation deviation and rotation deviation.

[0010] Step 2: Preprocess the deviation data set and input it into the linear regression model for training. Use the translation deviation and rotation deviation as labels to train the positioning information. Set the loss function as the root mean square of the translation deviation and the root mean square of the rotation deviation to obtain the deviation model.

[0011] Step 3: Deploy the deviation model in the system, obtain the first position information and the first posture information of the workpiece through the depth camera, and the manipulator generates positioning information, input the positioning information generated by the manipulator into the deviation model, obtain the translation deviation and rotation deviation, correct the first position information and the first posture information according to the translation deviation and rotation deviation, and generate new position information and posture information, input the deviation model again, and obtain five translation deviations and rotation deviations in total;

[0012] Step 4: Score the five deviations respectively, and use the position information and posture information formed by the translation deviation and rotation deviation with the lowest score to perform manipulator operation;

[0013] Step 5: After the robot operation is completed, the depth camera takes a second photo of the workpiece, and forms the final translation deviation, final posture deviation and final score based on the first photo, sets the threshold, and issues an alarm if the threshold is exceeded.

[0014] Furthermore, before the manipulator officially starts working, the depth camera is used to obtain the position data of the workpiece operated by the manipulator. The depth camera is set at a fixed position next to the workpiece. After the workpiece reaches the specified position, the manipulator locates the workpiece and reaches the position to be operated. At this time, the depth camera is used to take a first photo of the workpiece to obtain the position information and posture information of the workpiece.

[0015] The logic for obtaining the workpiece position information and posture information is as follows:

[0016] The photo taken by the depth camera is smoothed by a Gaussian filter. Each pixel in the photo contains a depth value. The pixel points in the picture are converted into three-dimensional coordinates according to the following formula:

[0017]

[0018] Z=D(x,y)

[0019] Among them, (x, y) is the pixel coordinate in the image, D(x, y) is the depth value of the pixel, (X, Y, Z) is the three-dimensional coordinate corresponding to the pixel point, and f x is the horizontal focal length of the depth camera, f y is the vertical focal length of the depth camera;

[0020] The photo obtained by the first shooting is calibrated as the first photo, and the position information and posture information obtained from the first photo are calibrated as the first position information and the first posture information. Then the manipulator starts to operate the workpiece. When the manipulator completes the operation, the workpiece is photographed for the second time through the same depth camera, and the photo obtained by the second shooting is calibrated as the second photo. The position information and posture information obtained from the second photo are calibrated as the second position information and the second posture information, and the second position information and the second posture information in the second photo are obtained. The occluded part of the manipulator in the workpiece picture is removed, and the point cloud of the workpiece in the two photos is obtained respectively. The point cloud is a set of three-dimensional coordinates of the pixel points of the workpiece in each photo. The translation deviation and rotation deviation of the workpiece in the two photos are obtained by the IPC algorithm, and the translation deviation and rotation deviation are summarized into a deviation data set;

[0021] When the robot positions the workpiece, the positioning information of the robot is generated based on the workpiece. The positioning information includes the position data of each joint of the robot and the load data of the robot's actuator. The load data includes the operation type, weight and force. The operation types are encoded with different four-dimensional unique hot encodings, and the positioning information of the robot is mapped in the deviation data set. Each positioning information sample corresponds one-to-one to the translation deviation and rotation deviation caused at that time.

[0022] Furthermore, the deviation data set is preprocessed, and samples with failed operations are manually screened and eliminated. All sample data are normalized, and the deviation data set is input into the linear regression model for training. The positioning data is trained with translation deviation and rotation deviation as labels to obtain different translation deviations and rotation deviations caused by the manipulator under positioning information. The loss function is set to the root mean square of the translation deviation and the root mean square of the rotation deviation, and the root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are set. The model training is completed when the root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are met at the same time. The trained model is calibrated as the deviation model, and the translation deviation and rotation deviation of the manipulator are output each time it performs an operation.

[0023] Furthermore, the deviation model is deployed in the robot operating system. After the workpiece reaches the specified position, the first photo is taken by the depth camera to obtain the first position information and the first posture information of the workpiece, and the first position information and the first posture information are calibrated as position information I and posture information I respectively. The robot is positioned according to the position information I and the posture information I to obtain the robot positioning information. The robot positioning information obtained at this time is calibrated as the robot positioning information I, and the robot positioning information I is mapped to the deviation data set and input into the deviation model to obtain the translation deviation I and the rotation deviation I. The translation deviation I and the rotation deviation I are used to correct the position information I and the posture information I to obtain the position information II and the posture information II. The formula for the correction is:

[0024]

[0025] Among them, X n is the value on the x-axis after the workpiece is corrected, X o is the value on the x-axis before the workpiece is corrected, ΔX is the translation deviation of the x-axis, and Y n is the value on the y-axis after the workpiece is corrected, Y o is the value on the y-axis before the workpiece is corrected, ΔY is the translation deviation of the y-axis, and Z n is the value on the z-axis after the workpiece is corrected, Z o is the value on the z-axis before the workpiece is corrected, ΔZ is the translation deviation of the z-axis, α n is the value on the a-axis after the workpiece is corrected, α o is the value on the a-axis before the workpiece is corrected, Δα is the rotation deviation of the a-axis, β n is the value on the b-axis after the workpiece is corrected, β o is the value on the b-axis before the workpiece is corrected, Δβ is the rotation deviation of the b-axis, and γ n is the value on the c-axis after the workpiece is corrected, γ o is the value on the c-axis before the workpiece is corrected, Δγ is the rotation deviation of the c-axis;

[0026] According to position information II and posture information II, the manipulator positioning information II is obtained again, the manipulator positioning information II is mapped to the deviation data set and input into the deviation model to obtain translation deviation II and rotation deviation II, and the translation deviation II and rotation deviation II are used to correct position information I and posture information I again to obtain position information III and posture information III, and this step is repeated until five translation deviations and rotation deviations are obtained respectively. The last three times are calibrated as translation deviation III and rotation deviation III, translation deviation IV and rotation deviation IV, and translation deviation V and rotation deviation V, respectively.

[0027] Furthermore, the five obtained translation deviations and rotation deviations are scored respectively according to the following formula:

[0028] PF=a×(|ΔX|+|ΔY|+|ΔZ|)+b×(|Δα|+|Δβ|+|Δγ|)

[0029] Where PF is the score value, ΔX is the translation deviation of the x-axis, ΔY is the translation deviation of the y-axis, ΔZ is the translation deviation of the z-axis, Δα is the rotation deviation of the a-axis, Δβ is the rotation deviation of the b-axis, Δγ is the rotation deviation of the c-axis, a and b are weight coefficients, a>b>0, and a 2 +b 2 =1;

[0030] The position information and posture information formed by the translation deviation and rotation deviation with the lowest score are selected to form the robot positioning information, and the robot operates the workpiece according to the positioning information.

[0031] Furthermore, when the robot completes the operation, the depth camera takes a second photo of the workpiece and obtains the second position information and the second posture information, analyzes the translation deviation and posture deviation of the workpiece in the two photos, calibrates the translation deviation and posture deviation as the final translation deviation and final posture deviation, and scores the final translation deviation and final posture deviation to form a final score, and sets the final translation deviation threshold, final posture deviation threshold and final score threshold respectively. When at least one of the values ​​exceeds the corresponding threshold, an alarm is issued and the robot operation is deemed to have failed.

[0032] The present invention also includes a visual positioning system for robot processing, and the visual positioning system is used to execute the above-mentioned visual positioning method for robot processing, including:

[0033] Information collection module: obtains the position information and posture information of the workpiece through the depth camera, obtains the positioning information of the robot, and obtains the translation deviation and rotation deviation of the workpiece when taking two photos;

[0034] Model training module: The translation deviation, rotation deviation and positioning information are aggregated into a deviation data set and input into a linear regression model to obtain a deviation model. The positioning information of the manipulator is input into the deviation model to obtain the translation deviation and rotation deviation.

[0035] Calculation and statistics module: calculates different position information and posture information according to translation deviation and rotation deviation, and makes statistical scores to instruct the manipulator to operate;

[0036] Alarm module: Alarm based on the final translation deviation, final posture deviation and final score obtained.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention obtains the translation deviation and rotation deviation caused by the manipulator to the workpiece according to the depth camera, trains the deviation model, and forms the positioning of the manipulator under different deviation correction conditions, thereby reducing the influence of the manipulator's own error on the workpiece operation, providing the manipulator with more accurate positioning information, and judging whether the operation of the manipulator is successful based on two photos taken by the depth camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0040] Figure 2 It is a schematic diagram of the system composition of the present invention. DETAILED DESCRIPTION

[0041] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0042] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0043] Example:

[0044] See also Figure 1 , the present invention provides a technical solution:

[0045] A visual positioning method for robot processing, the specific steps include:

[0046] Step 1: Take a photo of the workpiece before the robot operates it, obtain the first position information and the first posture information, and obtain the positioning information of the robot for the workpiece. After the robot completes the operation, take a photo of the workpiece again to obtain the second position information and the second posture information. Use the IPC algorithm to obtain the translation deviation and rotation deviation of the workpiece in the two photos, and add the robot positioning information to the deviation data set formed by the translation deviation and rotation deviation.

[0047] The step 1 includes the following contents:

[0048] Before the robot officially starts working, the depth camera is used to obtain the position data of the workpiece operated by the robot. The depth camera is set at a fixed position next to the workpiece. After the workpiece reaches the specified position, the robot locates the workpiece and reaches the position to be operated. At this time, the depth camera is used to take the first photo of the workpiece to obtain the position information and posture information of the workpiece.

[0049] The positioning of the workpiece by the robot can refer to but is not limited to the technology disclosed in a visual positioning method and system for robot processing with publication number CN118322217A, and should also include other technologies for obtaining robot positioning information by identifying the position of the workpiece.

[0050] The logic for obtaining the workpiece position information and posture information is as follows:

[0051] The photo taken by the depth camera is smoothed by a Gaussian filter. Each pixel in the photo contains a depth value. The pixel points in the picture are converted into three-dimensional coordinates according to the following formula:

[0052]

[0053] Z=D(x,y)

[0054] Among them, (x, y) is the pixel coordinate in the image, D(x, y) is the depth value of the pixel, (X, Y, Z) is the three-dimensional coordinate corresponding to the pixel point, and f x is the horizontal focal length of the depth camera, f y is the vertical focal length of the depth camera;

[0055] The upper left corner of the two-dimensional image is taken as the pixel coordinate origin, the horizontal right direction is the positive direction of the pixel coordinate x-axis, the vertical downward direction is the positive direction of the pixel coordinate y-axis, the optical center of the depth camera is taken as the three-dimensional coordinate origin, the horizontal right direction is the positive direction of the three-dimensional coordinate x-axis, the vertical upward direction is the positive direction of the three-dimensional coordinate y-axis, and the depth direction from the camera to the object is the positive direction of the three-dimensional coordinate z-axis. According to the internal parameters that have been set for the depth camera, the pixel coordinates are converted into three-dimensional coordinates according to the two-dimensional image.

[0056] This method of converting two-dimensional information into three-dimensional coordinates has been maturely used in this field. This formula is a commonly used formula in this field and will not be described here.

[0057] By converting the two-dimensional information of each pixel in the photo into three-dimensional coordinates, the three-dimensional information of the workpiece can be constructed, which facilitates the next step of analyzing the translation deviation and rotation deviation of the workpiece between the two photos.

[0058] The photo obtained by the first shooting is calibrated as the first photo, and the position information and posture information obtained from the first photo are calibrated as the first position information and the first posture information. Then the manipulator starts to operate the workpiece. When the manipulator completes the operation, the workpiece is photographed for the second time through the same depth camera, and the photo obtained by the second shooting is calibrated as the second photo. The position information and posture information obtained from the second photo are calibrated as the second position information and the second posture information, and the second position information and the second posture information in the second photo are obtained. The occluded part of the manipulator in the workpiece picture is removed, and the point cloud of the workpiece in the two photos is obtained respectively. The point cloud is a set of three-dimensional coordinates of the pixel points of the workpiece in each photo. The translation deviation and rotation deviation of the workpiece in the two photos are obtained by the IPC algorithm, and the translation deviation and rotation deviation are summarized into a deviation data set;

[0059] As a mature algorithm, the IPC algorithm is widely used in this field, such as the correction device and method of the 3D camera and robotic arm coordinate system with publication number CN111716340A for determining the target deviation in the image.

[0060] When the robot positions the workpiece, the positioning information of the robot is generated based on the workpiece. The positioning information includes the position data of each joint of the robot and the load data of the robot's actuator. The load data includes the operation type, weight and force. The operation types are encoded with different four-dimensional unique hot codes, such as clamping as [0,0,0,1], lifting as [0,0,1,0], etc. The positioning information of the robot is mapped in the deviation data set, and each positioning information sample corresponds one-to-one to the translation deviation and rotation deviation caused at that time.

[0061] Step 2: Preprocess the deviation data set and input it into the linear regression model for training. Use the translation deviation and rotation deviation as labels to train the positioning information. Set the loss function as the root mean square of the translation deviation and the root mean square of the rotation deviation to obtain the deviation model.

[0062] The step 2 includes the following contents:

[0063] The deviation data set is preprocessed, and samples with failed operations are manually screened and eliminated. All sample data are normalized, and the deviation data set is input into the linear regression model for training. The positioning data is trained with translation deviation and rotation deviation as labels to obtain the different translation deviations and rotation deviations caused by the manipulator under the positioning information. The loss function is set to the root mean square of the translation deviation and the root mean square of the rotation deviation. The root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are set. The model training is completed when the root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are met at the same time. The trained model is calibrated as the deviation model, and the translation deviation and rotation deviation of the manipulator are output each time it performs an operation.

[0064] Step 3: Deploy the deviation model in the system, obtain the first position information and the first posture information of the workpiece through the depth camera, and the manipulator generates positioning information, input the positioning information generated by the manipulator into the deviation model, obtain the translation deviation and rotation deviation, correct the first position information and the first posture information according to the translation deviation and rotation deviation, and generate new position information and posture information, input the deviation model again, and obtain five translation deviations and rotation deviations in total;

[0065] The step 3 includes the following contents:

[0066] The deviation model is deployed in the robot operating system. After the workpiece reaches the specified position, the first photo is taken by the depth camera to obtain the first position information and the first posture information of the workpiece, and the first position information and the first posture information are calibrated as position information I and posture information I respectively. The robot is positioned according to the position information I and the posture information I to obtain the robot positioning information. The robot positioning information obtained at this time is calibrated as the robot positioning information I. The robot positioning information I is mapped to the deviation data set and input into the deviation model to obtain the translation deviation I and the rotation deviation I. The translation deviation I and the rotation deviation I are used to correct the position information I and the posture information I to obtain the position information II and the posture information II. The formula for the correction is:

[0067]

[0068] Among them, X n is the value on the x-axis after the workpiece is corrected, X o is the value on the x-axis before the workpiece is corrected, ΔX is the translation deviation of the x-axis, and Y n is the value on the y-axis after the workpiece is corrected, Y o is the value on the y-axis before the workpiece is corrected, ΔY is the translation deviation of the y-axis, and Z n is the value on the z-axis after the workpiece is corrected, Z o is the value on the z-axis before the workpiece is corrected, ΔZ is the translation deviation of the z-axis, αn is the value on the a-axis after the workpiece is corrected, α o is the value on the a-axis before the workpiece is corrected, Δα is the rotation deviation of the a-axis, β n is the value on the b-axis after the workpiece is corrected, β o is the value on the b-axis before the workpiece is corrected, Δβ is the rotation deviation of the b-axis, and γ n is the value on the c-axis after the workpiece is corrected, γ o is the value on the c-axis before the workpiece is corrected, Δγ is the rotation deviation of the c-axis;

[0069] The translation deviation and rotation deviation are caused by the positioning accuracy of the robot itself, and the workpiece is subjected to force and translation and rotation before and after the operation. The robot has its own high-precision operating range, and it cannot guarantee that every workpiece is located in the high-precision operating range of the robot. The translation deviation and rotation deviation of the workpiece before and after the operation are used to evaluate the accuracy of the robot for this operation, and the value of the workpiece deviation itself represents the deviation of the robot in this operation. Therefore, correcting the first position information and the first rotation information with the translation deviation and rotation deviation can make the robot compensate for the deviation caused by its own accuracy. During the correction, when calculating the spatial position of the workpiece, the first position information is added with the translation deviation value on the x, y and z axes respectively. When calculating the posture of the workpiece, the first posture information is added with the rotation deviation on the a, b and c axes respectively.

[0070] According to position information II and posture information II, the manipulator positioning information II is obtained again, the manipulator positioning information II is mapped to the deviation data set and input into the deviation model to obtain translation deviation II and rotation deviation II, and the translation deviation II and rotation deviation II are used to correct position information I and posture information I again to obtain position information III and posture information III, and this step is repeated until five translation deviations and rotation deviations are obtained respectively. The last three times are calibrated as translation deviation III and rotation deviation III, translation deviation IV and rotation deviation IV, and translation deviation V and rotation deviation V, respectively.

[0071] When the workpiece is in different positions, the robot can adopt different ways to position the workpiece, and different ways will cause different deviations of the robot. Since we don’t know where the best positioning information is, we use the first positioning information to input the deviation model to obtain the translation deviation and rotation deviation as the starting point, and use a limited number of attempts to obtain different translation deviations and rotation deviations, which provide reference conditions for selecting the best positioning information in the next step.

[0072] Step 4: Score the five deviations respectively, and use the position information and posture information formed by the translation deviation and rotation deviation with the lowest score to perform manipulator operation;

[0073] The step 4 includes the following contents:

[0074] The five translation deviations and rotation deviations obtained are scored respectively according to the following formula:

[0075] PF=a×(|ΔX|+|ΔY|+|ΔZ|)+b×(|Δα|+|Δβ|+|Δγ|)

[0076] Where PF is the score value, ΔX is the translation deviation of the x-axis, ΔY is the translation deviation of the y-axis, ΔZ is the translation deviation of the z-axis, Δα is the rotation deviation of the a-axis, Δβ is the rotation deviation of the b-axis, Δγ is the rotation deviation of the c-axis, a and b are weight coefficients, a>b>0, and a 2 +b 2 =1;

[0077] The absolute values ​​of the translation deviations of the x, y and z axes and the absolute values ​​of the rotation deviations of the a, b and c axes are weightedly added. The absolute value of the deviation represents the impact of the robot on the position and posture of the workpiece before and after the operation. The greater the impact, the greater the absolute values ​​of the translation deviation and rotation deviation, which means that the less accurate the positioning of the workpiece by the robot is, and the higher the score is. When the spatial position and posture of the workpiece deviate within a small range, the deviation caused by the displacement is greater than the deviation caused by the rotation. Therefore, the cumulative coefficient of the absolute value of the translation deviation is greater than the cumulative coefficient of the absolute value of the rotation deviation.

[0078] The position information and posture information formed by the translation deviation and rotation deviation with the lowest score are selected to form the robot positioning information, and the robot operates the workpiece according to the positioning information.

[0079] The lowest score means that the robot has the least impact on the position and posture of the workpiece before and after the operation, which means that the robot is in the most accurate state under this positioning information, making it easier to obtain the best operating effect.

[0080] Step 5: After the robot operation is completed, the depth camera takes a second photo of the workpiece, and forms the final translation deviation, final posture deviation and final score based on the first photo, sets the threshold, and issues an alarm if the threshold is exceeded.

[0081] The step 5 includes the following contents:

[0082] When the robot completes the operation, the depth camera takes a second photo of the workpiece and obtains the second position information and the second posture information. The translation deviation and posture deviation of the workpiece in the two photos are analyzed, and the translation deviation and posture deviation are calibrated as the final translation deviation and final posture deviation. The final translation deviation and final posture deviation are scored to form a final score. The final translation deviation threshold, final posture deviation threshold and final score threshold are set respectively. When at least one value exceeds the corresponding threshold, an alarm is issued and the robot operation is deemed to have failed.

[0083] By taking photos twice, the robot can obtain successful operation results under the most accurate positioning state. For alarms of operation failure, personnel can be reminded to take measures in time.

[0084] like Figure 2 As shown, the present invention also includes a visual positioning system for robot processing, and the visual positioning system is used to execute the above-mentioned visual positioning method for robot processing, including:

[0085] Information collection module: obtains the position information and posture information of the workpiece through the depth camera, obtains the positioning information of the robot, and obtains the translation deviation and rotation deviation of the workpiece when taking two photos;

[0086] Model training module: The translation deviation, rotation deviation and positioning information are aggregated into a deviation data set and input into a linear regression model to obtain a deviation model. The positioning information of the manipulator is input into the deviation model to obtain the translation deviation and rotation deviation.

[0087] Calculation and statistics module: calculates different position information and posture information according to translation deviation and rotation deviation, and makes statistical scores to instruct the manipulator to operate;

[0088] Alarm module: Alarm based on the final translation deviation, final posture deviation and final score obtained.

[0089] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0090] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0091] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A visual positioning method for robot processing, characterized in that: The specific steps include: Step 1: Take a photo of the workpiece before the robot operates it, obtain the first position information and the first posture information, and obtain the positioning information of the robot for the workpiece. After the robot completes the operation, take a photo of the workpiece again to obtain the second position information and the second posture information. The translation deviation and rotation deviation of the workpiece in the two photos are obtained through the IPC algorithm, and the robot positioning information is added to the deviation data set formed by the translation deviation and rotation deviation. Step 2: Preprocess the deviation data set and input it into the linear regression model for training. Use the translation deviation and rotation deviation as labels to train the positioning information. Set the loss function as the root mean square of the translation deviation and the root mean square of the rotation deviation to obtain the deviation model. Step 3: Deploy the deviation model in the system, obtain the first position information and the first posture information of the workpiece through the depth camera, and the manipulator generates positioning information, input the positioning information generated by the manipulator into the deviation model, obtain the translation deviation and rotation deviation, correct the first position information and the first posture information according to the translation deviation and rotation deviation, and generate new position information and posture information, input the deviation model again, and obtain five translation deviations and rotation deviations in total; Step 4: Score the five deviations respectively, and use the position information and posture information formed by the translation deviation and rotation deviation with the lowest score to perform manipulator operation; Step 5: After the robot operation is completed, the depth camera takes a second photo of the workpiece, and forms the final translation deviation, final posture deviation and final score based on the first photo, sets the threshold, and issues an alarm if the threshold is exceeded.

2. The visual positioning method for robot processing according to claim 1, characterized in that: Before the robot officially starts working, the depth camera is used to obtain the position data of the workpiece operated by the robot. The depth camera is set at a fixed position next to the workpiece. After the workpiece reaches the specified position, the robot locates the workpiece and reaches the position to be operated. At this time, the depth camera is used to take the first photo of the workpiece to obtain the position information and posture information of the workpiece. The logic for obtaining the workpiece position information and posture information is as follows: The photo taken by the depth camera is smoothed by a Gaussian filter. Each pixel in the photo contains a depth value. The pixel points in the picture are converted into three-dimensional coordinates according to the following formula: Z=D(x,y) Among them, (x, y) is the pixel coordinate in the image, D(x, y) is the depth value of the pixel, (X, Y, Z) is the three-dimensional coordinate corresponding to the pixel point, and f x is the horizontal focal length of the depth camera, f y is the vertical focal length of the depth camera; The photo obtained by the first shooting is calibrated as the first photo, and the position information and posture information obtained from the first photo are calibrated as the first position information and the first posture information. Then the manipulator starts to operate the workpiece. When the manipulator completes the operation, the workpiece is photographed for the second time through the same depth camera, and the photo obtained by the second shooting is calibrated as the second photo. The position information and posture information obtained from the second photo are calibrated as the second position information and the second posture information, and the second position information and the second posture information in the second photo are obtained. The occluded part of the manipulator in the workpiece picture is removed, and the point cloud of the workpiece in the two photos is obtained respectively. The point cloud is a set of three-dimensional coordinates of the pixel points of the workpiece in each photo. The translation deviation and rotation deviation of the workpiece in the two photos are obtained by the IPC algorithm, and the translation deviation and rotation deviation are summarized into a deviation data set; When the robot positions the workpiece, the positioning information of the robot is generated based on the workpiece. The positioning information includes the position data of each joint of the robot and the load data of the robot's actuator. The load data includes the operation type, weight and force. The operation types are encoded with different four-dimensional unique hot encodings, and the positioning information of the robot is mapped in the deviation data set. Each positioning information sample corresponds one-to-one to the translation deviation and rotation deviation caused at that time.

3. The visual positioning method for robot processing according to claim 2, characterized in that: The deviation data set is preprocessed, and samples with failed operations are manually screened and eliminated. All sample data are normalized, and the deviation data set is input into the linear regression model for training. The positioning data is trained with translation deviation and rotation deviation as labels to obtain the different translation deviations and rotation deviations caused by the manipulator under the positioning information. The loss function is set to the root mean square of the translation deviation and the root mean square of the rotation deviation. The root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are set. The model training is completed when the root mean square threshold of the translation deviation and the root mean square threshold of the rotation deviation are met at the same time. The trained model is calibrated as the deviation model, and the translation deviation and rotation deviation of the manipulator are output each time it performs an operation.

4. The visual positioning method for robot processing according to claim 3 is characterized in that: The deviation model is deployed in the robot operating system. After the workpiece reaches the specified position, the first photo is taken by the depth camera to obtain the first position information and the first posture information of the workpiece, and the first position information and the first posture information are calibrated as position information I and posture information I respectively. The robot is positioned according to the position information I and the posture information I to obtain the robot positioning information. The robot positioning information obtained at this time is calibrated as the robot positioning information I. The robot positioning information I is mapped to the deviation data set and input into the deviation model to obtain the translation deviation I and the rotation deviation I. The translation deviation I and the rotation deviation I are used to correct the position information I and the posture information I to obtain the position information II and the posture information II. The formula for the correction is: Among them, X n is the value on the x-axis after the workpiece is corrected, X o is the value on the x-axis before the workpiece is corrected, ΔX is the translation deviation of the x-axis, and Y n is the value on the y-axis after the workpiece is corrected, Y o is the value on the y-axis before the workpiece is corrected, ΔY is the translation deviation of the y-axis, and Z n is the value on the z-axis after the workpiece is corrected, Z o is the value on the z-axis before the workpiece is corrected, ΔZ is the translation deviation of the z-axis, α n is the value on the a-axis after the workpiece is corrected, α o is the value on the a-axis before the workpiece is corrected, Δα is the rotation deviation of the a-axis, β n is the value on the b-axis after the workpiece is corrected, β o is the value on the b-axis before the workpiece is corrected, Δβ is the rotation deviation of the b-axis, and γ n is the value on the c-axis after the workpiece is corrected, γ o is the value on the c-axis before the workpiece is corrected, Δγ is the rotation deviation of the c-axis; According to position information II and posture information II, the manipulator positioning information II is obtained again, the manipulator positioning information II is mapped to the deviation data set and input into the deviation model to obtain translation deviation II and rotation deviation II, and the translation deviation II and rotation deviation II are used to correct position information I and posture information I again to obtain position information III and posture information III, and this step is repeated until five translation deviations and rotation deviations are obtained respectively. The last three times are calibrated as translation deviation III and rotation deviation III, translation deviation IV and rotation deviation IV, and translation deviation V and rotation deviation V, respectively.

5. The visual positioning method for robot processing according to claim 4 is characterized in that: The five translation deviations and rotation deviations obtained are scored respectively according to the following formula: PF=a×(|ΔX|+|ΔY|+|ΔZ|)+b×(|Δα|+|Δβ|+|Δγ|) Where PF is the score value, ΔX is the translation deviation of the x-axis, ΔY is the translation deviation of the y-axis, ΔZ is the translation deviation of the z-axis, Δα is the rotation deviation of the a-axis, Δβ is the rotation deviation of the b-axis, Δγ is the rotation deviation of the c-axis, a and b are weight coefficients, a>b>0, and a 2 +b 2 =1; The position information and posture information formed by the translation deviation and rotation deviation with the lowest score are selected to form the robot positioning information, and the robot operates the workpiece according to the positioning information.

6. The visual positioning method for robot processing according to claim 5, characterized in that: When the robot completes the operation, the depth camera takes a second photo of the workpiece and obtains the second position information and the second posture information. The translation deviation and posture deviation of the workpiece in the two photos are analyzed, and the translation deviation and posture deviation are calibrated as the final translation deviation and final posture deviation. The final translation deviation and final posture deviation are scored to form a final score. The final translation deviation threshold, final posture deviation threshold and final score threshold are set respectively. When at least one value exceeds the corresponding threshold, an alarm is issued and the robot operation is deemed to have failed.

7. A visual positioning system for robot processing, the visual positioning system is used to execute a visual positioning method for robot processing according to any one of claims 1 to 6, characterized in that: include: Information collection module: obtains the position information and posture information of the workpiece through the depth camera, obtains the positioning information of the robot, and obtains the translation deviation and rotation deviation of the workpiece when taking two photos; Model training module: The translation deviation, rotation deviation and positioning information are aggregated into a deviation data set and input into a linear regression model to obtain a deviation model. The positioning information of the manipulator is input into the deviation model to obtain the translation deviation and rotation deviation. Calculation and statistics module: calculates different position information and posture information according to translation deviation and rotation deviation, and makes statistical scores to instruct the manipulator to operate; Alarm module: Alarm based on the final translation deviation, final posture deviation and final score obtained.

Images