Automatic kilogram group weight verification method and system based on coordination of three manipulators

Abstract

The invention discloses a kilogram group weight automatic verification method and system based on cooperation of three manipulators. The method comprises the following steps: collecting a natural light image and a depth image of stacked kilogram group weights; based on a Faster R-CNN algorithm, the positions of weights and handles of the weights are recognized in real time, and three-dimensional coordinates of each weight are output; through cooperative scheduling of the three manipulators, grabbing, carrying and placing of unpicked weights, standard weights and detected weights are completed; generating a traceable code containing weight basic information, real-time position information and a verification result through a detected object information coding technology; and calibrating operation is coordinated through the master control upper computer, and a calibrating report is generated to finish calibrating. The system comprises a weight real-time identification and positioning module, a three-manipulator cooperative scheduling module, a detected object information coding module and a master control upper computer module.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology, and in particular to a kilogram weight automatic calibration method and system based on the collaboration of three manipulators. Background Art

[0002] As a benchmark for mass measurement, weights are widely used in industrial production, commercial trade, scientific research, and other fields, and their accuracy is directly related to the reliability of measurement results. Due to their large weight and volume, kilogram-set weights are often used in applications requiring high weighing precision. Traditional kilogram-set weight verification methods rely primarily on manual operation. Research on automated verification of kilogram-set weights with the collaboration of a robotic arm is of great significance.

[0003] Automatic weight verification primarily involves weight recognition and weight verification technologies. Weight recognition technology, such as patent application publication number CN112233078A, utilizes deep learning networks to identify weights. Weight verification technology, such as patent publication numbers CN119595080A and CN119642950A, utilizes different verification devices and structures to achieve weight verification.

[0004] The above-mentioned specific patent reference documents are:

[0005] 1) "A method for identifying and segmenting key parts of stacked kilogram weights", patent publication number CN112233078A. This patent relates to a method for identifying and segmenting key parts of stacked kilogram weights, including: determining the characteristics and key parts of the kilogram weights; labeling the collected data set of stacked kilogram weights; using an image enhancement algorithm to perform image enhancement and data enhancement processing on the labeled data; using a ResNet+FPN network as a feature extractor for the stacked kilogram weights and their key parts; segmenting the key parts of the stacked kilogram weights and identifying and locating the kilogram weight instances in the stacked kilogram weights; using the enhanced stacked kilogram weights data to train the network, with the objective function being the cross-entropy loss function of the image, and using the gradient descent method to solve the loss function. When the global minimum or local minimum is obtained, the corresponding model parameters are obtained to complete the establishment of the neural network model. The present invention can quickly and accurately identify and segment stacked kilogram weights and their key parts, and is suitable for the recognition and segmentation of partially occluded low-contrast objects. The weight recognition technology of the present invention is different from the above invention. It uses the deep learning Faster R-CNN network to identify the weight, verifies the correctness of the identification points through key points, and finally outputs the world coordinates of the weight key points after conversion.

[0006] 2) "An Automatic Loading Mechanism for Tiny Weights," patent publication number CN119642950A. This patent relates to an automatic loading mechanism for tiny weights, comprising a transparent barrel, a barrel cover, a comparator, a support column, and a carrying plate. The support column is placed on the comparator and is connected to a plurality of carrying columns. The top wall of the transparent barrel is provided with a first annular groove, in which an annular slider is slidably disposed. The inner circumferential wall of the transparent barrel is provided with a second annular groove, in which a sliding block is slidably disposed. The sliding block is connected to the annular slider, and the carrying plate is connected to the sliding block. The bottom wall of the carrying plate is provided with a first cylinder, the output shaft of the first cylinder is connected to a second cylinder, the output shaft of the second cylinder is connected to a moving plate, the inner wall of the moving plate is provided with two placing columns, the output shaft of the third cylinder is connected to a fourth cylinder, and the output shaft of the fourth cylinder is connected to a toggle plate. The barrel cover can be placed on the top wall of the transparent barrel, completely covering the inner cavity of the transparent barrel, and the barrel cover is located on the inner side of the annular slider. This invention can solve the problem of low efficiency in the calibration of miniature square weights. The detection device and process of the present invention are different from the above inventions. The present invention uses three manipulators to collaboratively complete the transportation of kilogram group weights to achieve fully automatic calibration.

[0007] 3) "A precise weight calibration device and system thereof", patent publication number CN119595080A. This patent relates to a precise weight calibration device and system thereof, including: a base and a workbench; the workbench is installed on the upper surface of the base; a fixed structure, the fixed structure is installed on the lower surface of the base, and the fixed structure is used for fixing; a supporting structure, the supporting structure is installed on the upper surface of the workbench, and the supporting structure is used for supporting; a first adjustment structure, the first adjustment structure is installed on the supporting structure, and the first adjustment structure is used for adjustment. The present invention relates to the field of weight calibration, and prevents errors caused by personnel during operation or errors caused by misreading data from seriously affecting the calibration results. The structure is relatively simple and easy to use. The detection object and device of the present invention are different from the above inventions. This article is aimed at kilogram-group heavy weights, and adopts multiple manipulators to complete the weight handling to achieve automated calibration. Summary of the Invention

[0008] In order to solve the above technical problems, the purpose of the present invention is to provide a method and system for automatic calibration of kilogram group weights based on the collaboration of three manipulators.

[0009] The purpose of the present invention is achieved through the following technical solutions:

[0010] A method for automatically verifying a kilogram weight set based on the collaboration of three manipulators comprises the following steps:

[0011] Step A collects natural light images and depth images of the stacked kilogram weights;

[0012] Step B uses the Faster R-CNN algorithm to identify the position of the weights and their handles in real time and output the three-dimensional coordinates of each weight;

[0013] Step C uses the coordinated scheduling of three manipulators to complete the grabbing, transportation and placement of the unpicked weights, standard weights and checked weights;

[0014] Step D generates a traceable code containing basic information of the weight, real-time location information and verification results through the information coding technology of the object being tested;

[0015] Step E coordinates the calibration operation through the master control host computer and generates a calibration report to complete the calibration.

[0016] An automatic calibration system for kilogram weights based on the collaboration of three manipulators, comprising:

[0017] Weight real-time identification and positioning module, three-manipulator collaborative scheduling module, inspected object information encoding module and master control host computer module;

[0018] The weight real-time identification and positioning module is used to collect natural light images and depth images of the stacked kilogram weights, and identify the weight positions in real time based on the Faster R-CNN algorithm, and output the three-dimensional coordinates of each weight;

[0019] The three-manipulator collaborative scheduling module is used to complete the tasks of grabbing, carrying and placing unchecked weights, standard weights and checked weights respectively through the three manipulators;

[0020] The inspected object information encoding module is used to perform label-free traceability encoding on the weight information during the verification process, and generate a traceable code containing the basic information of the weight, real-time location information and verification results;

[0021] The master control host computer module is used to coordinate the coordinated operation of the weight real-time identification and positioning module, the three-manipulator collaborative scheduling module and the inspected object information encoding module, send control instructions and monitor the inspection process in real time.

[0022] Compared with the prior art, one or more embodiments of the present invention may have the following advantages:

[0023] Through the collaborative scheduling of three manipulators and Faster R-CNN recognition technology, the automatic grabbing, handling and calibration of kilogram weights can be achieved, significantly improving calibration efficiency and reducing manpower consumption; combined with the weight calibration task scheduling algorithm, the calibration process is optimized and the calibration time is shortened; using information coding technology, full traceability is achieved, improving the reliability of calibration results and management efficiency, which has significant practical application value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1This is a flow chart of the automatic verification method of kilogram group weights based on the cooperation of three manipulators;

[0025] Figure 2 This is the Faster R-CNN kilogram group weight detection model diagram;

[0026] Figure 3 It is the coordinate conversion flow chart of the key points of weight grabbing;

[0027] Figure 4 This is the flow chart of the three-manipulator collaborative verification algorithm. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be described in further detail below with reference to embodiments and accompanying drawings.

[0029] like Figure 1 The figure shows the process of the automatic calibration method of kilogram weights based on the collaboration of three manipulators, which includes the following steps:

[0030] Step 10: collecting natural light images and depth images of the stacked kilogram weights;

[0031] Step 20: Identify the positions of the weights and their handles in real time based on the Faster R-CNN algorithm, and output the three-dimensional coordinates of each weight;

[0032] Step 30: The three manipulators coordinate and dispatch to complete the grabbing, transporting and placement of the unpicked weights, standard weights and checked weights;

[0033] Step 40 generates a traceable code including basic information of the weight, real-time location information and verification results through the information coding technology of the object being tested;

[0034] Step 50 coordinates the calibration operation through the master control host computer and generates a calibration report to complete the calibration.

[0035] In step 10, the depth camera is used to capture the natural light image and the depth image of the stacked kilogram weights.

[0036] The above step 20 includes: using the Faster R-CNN algorithm to perform target detection on the collected natural light image and identify the bounding box of each weight and its handle, such as Figure 2 The figure shows the Faster R-CNN kilogram group weight detection model diagram.

[0037] The Faster R-CNN weight detection network loss function includes RPN loss L RPN ROI loss L ROI , that is:

[0038] L total =LRPN +L ROI (1)

[0039] RPN loss includes RPN classification loss L RPN_cls , RPN bounding box regression loss L RPN_box , let the total number of samples be N cls , the probability that sample i is predicted to be the target is p i , the positive sample label probability of sample i is The weight balance parameter is λ and the feature map size is N reg , the predicted offset is t i , the actual offset of the anchor box is have:

[0040] L RPN =L RPN_cls +L RPN_box (2)

[0041]

[0042] in,

[0043] ROI loss includes ROI classification loss L ROI_cls , ROI bounding box regression loss L ROI_box ,have:

[0044] L ROI =L ROI_cls +L ROI_box (4)

[0045] L ROI_cls 、L ROI_box The calculation formula is the same as the above L RPN_cls 、L RPN_box The calculation formula is the same.

[0046] Let the coordinates of the upper left corner and lower right corner of the bounding box of the weight handle be detected respectively (u h-left ,v h-left )、(u h-right ,v h-right ), then the weight grabbing center (u hand ,v hand ) can be calculated by the following formula:

[0047]

[0048] At the same time, let the coordinates of the upper left corner and lower right corner of the overall bounding box of the weight be detected respectively (u w-left ,v w-left )、(u w-right ,v w-right), the key point verification rules for real-time identification of the handle center of the kilogram group weight are as follows:

[0049]

[0050] Figure 3 The overall flow chart of the coordinate conversion of the key points for weight grasping is as follows: it needs to go through three conversions: pixel coordinates → image coordinates → camera coordinates → world coordinates, and obtain the world coordinates to guide the robot arm to grasp the weight.

[0051] The above step 30 specifically includes: the unchecked weight manipulator grabs the unchecked weight and transports it to the calibration balance; the standard weight manipulator grabs the standard weight and compares the mass; the checked weight manipulator places the checked weight in the qualified / unqualified area; the calibration task scheduling is optimized by the weight calibration task scheduling algorithm, and the weight calibration task scheduling algorithm is to minimize the maximum calibration time T max As the target, its mathematical expression is:

[0052] [T max ] min =min[max(T1,T2,T3)] (7)

[0053] Among them, T1, T2, and T3 are the total operating time of the unchecked weight manipulator, the standard weight manipulator, and the checked weight manipulator respectively;

[0054] The weight calibration task scheduling algorithm includes the following steps: ① generating a feasible solution to the scheduling problem through an initial solution construction algorithm (algorithm 1); ② optimizing the initial solution through an insertion iteration algorithm (algorithm 2); and ③ further optimizing the scheduling solution through an exchange iteration algorithm (algorithm 3) to obtain the optimal solution.

[0055] The pseudo code of Algorithm 1 is as follows:

[0056]

[0057] The pseudo code of Algorithm 2 is as follows:

[0058]

[0059]

[0060] 3: Output the optimal solution δ′ for the weight verification task scheduling.

[0061]

[0062] like Figure 4 The figure shows the algorithm flow for scheduling weight verification tasks.

[0063] The above step 40 specifically includes: the traceable code of the inspected object information is composed of an 8-digit verification date code (year / month / day), a 2-digit entrusting unit number, a 4-digit verification time (the time when the batch of weights enters the verification site), a 2-digit weight specification number, a 3-digit placement information of the kilogram group weights before verification (row / column / layer number), a 12-digit position information of the kilogram group weights before verification (the world coordinates of each weight in the unverified weight placement area), a 2-digit kilogram group weight verification scheduling information, a 2-digit kilogram group weight verification result information (pass 00 / fail 01), a 3-digit placement information of the kilogram group weights after verification (row / column / layer number), a 12-digit position information of the kilogram group weights after verification (the world coordinates of each weight in the verified weight placement area), and a 1-digit check code, totaling 50 decimal digits and 1 hexadecimal digit. Through this technology, a traceable code including basic weight information, real-time position information, and verification results is generated.

[0064] This embodiment also provides an automatic calibration system for kilogram-group weights based on the collaboration of three manipulators, including a real-time weight identification and positioning module, a three-manipulator collaborative scheduling module, a tested object information encoding module, and a master control host computer module; the real-time weight identification and positioning module is used to collect natural light images and depth images of stacked kilogram-group weights through a depth camera, and identify the weight position in real time based on the Faster R-CNN algorithm, and output the three-dimensional coordinates of each weight; the three-manipulator collaborative scheduling module is used to complete the grasping, carrying and placing tasks of uncalibrated weights, standard weights and verified weights through three manipulators respectively, thereby realizing multi-manipulator collaborative operation; the tested object information encoding module is used to perform label-free traceability encoding on the weight information during the calibration process, and generate a traceable code containing basic weight information, real-time position information and calibration results; the master control host computer module is used to coordinate the operation of each module, send control instructions and monitor the calibration process in real time.

[0065] Although the embodiments disclosed herein are as described above, the contents described herein are merely embodiments for facilitating understanding of the present invention and are not intended to limit the present invention. Any person skilled in the art may make any modifications and variations in the form and details of the embodiments without departing from the spirit and scope of the present invention. However, the scope of patent protection of the present invention shall remain subject to the scope defined by the appended claims.

Claims

1. A method for automatic calibration of kilogram weights based on the collaboration of three manipulators, characterized in that: The following steps are involved: Step A collects natural light images and depth images of the stacked kilogram weights; Step B uses the Faster R-CNN algorithm to identify the position of the weights and their handles in real time and output the three-dimensional coordinates of each weight; Step C uses the coordinated scheduling of three manipulators to complete the grabbing, transportation and placement of the unpicked weights, standard weights and checked weights; Step D generates a traceable code containing basic information of the weight, real-time location information and verification results through the information coding technology of the object being tested; Step E coordinates the calibration operation through the master control host computer and generates a calibration report to complete the calibration.

2. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 1 is characterized in that: In step B, the Faster R-CNN algorithm is used to perform target detection on the collected natural light image to identify the bounding box of each weight and its handle; the key point verification rule is used to verify whether the center point of the identified weight handle is accurate; the pixel coordinates are converted into world coordinates through a coordinate conversion algorithm, and the three-dimensional position information of the weight is output.

3. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 2 is characterized in that: The loss function of the Faster R-CNN in the weight detection network includes the RPN loss L RPN ROI loss L ROI ,Right now: L total =L RPN +L ROI (1) RPN loss includes RPN classification loss L RPN_cls , RPN bounding box regression loss L RPN_box , let the total number of samples be N cls , the probability that sample i is predicted to be the target is p i , the positive sample label probability of sample i is The weight balance parameter is λ and the feature map size is N reg , the predicted offset is t i , the actual offset of the anchor box is Then L RPN : L RPN =L RPN_cls +L RPN_box (2) in, ROI loss includes ROI classification loss L ROI_cls , ROI bounding box regression loss L ROI_box , then L ROI : L ROI =L ROI_cls +L ROI_box (4) L ROI_cls 、L ROI_box The calculation formula is the same as the above L RPN_cls 、L RPN_box The calculation formula is the same; Let the coordinates of the upper left corner and lower right corner of the bounding box of the weight handle be detected respectively (u h-left ,v h-left )、(u h-right ,v h-right ), then the weight grabs the center (u hand ,v hand ) is calculated by the following formula: At the same time, let the coordinates of the upper left corner and lower right corner of the overall bounding box of the weight be detected respectively (u w-left ,v w-left )、(u w-right ,v w-right ), then the key point verification rule for real-time identification of the handle center of the kilogram group weight is:

4. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 3 is characterized in that: The coordinate transformation of the weight key point includes: the center pixel coordinate (u hand ,v hand ) is first converted into image coordinates (x hand ,y hand ), and then converted into camera coordinates (X hand , Y hand ,Z hand ), and finally converted to world coordinates for the third time 5. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 1 is characterized in that: In the step C, The uncalibrated weights are grabbed by the uncalibrated weight manipulator and sent to the calibration balance; Grab the standard weights through the standard weight manipulator and compare their masses; The checked weights are placed in the qualified / unqualified area by the checked weight manipulator; The calibration task scheduling is optimized by the weight calibration task scheduling algorithm, which minimizes the maximum calibration time T max as the goal.

6. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 5 is characterized in that: The minimum test maximum time T max The calculation formula is: [T max ] min =min[max(T1,T2,T3)] (7) Among them, T1, T2, and T3 are the total operating times of the unchecked weight manipulator, standard weight manipulator, and checked weight manipulator, respectively.

7. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 5 is characterized in that: The weight verification task scheduling algorithm includes: 1) By entering the coordinates of the key points of the weight handle center Obtain one of the feasible solutions δ in the verification task scheduling; 2) Insert each weight in the scheduling feasible solution δ into an iterative loop in turn to optimize the initial solution; 3) Perform exchange iterations in the feasible scheduling solution δ to obtain the optimal solution δ′ for the weight verification task scheduling.

8. The automatic calibration method of kilogram weights based on three-manipulator collaboration according to claim 1 is characterized in that: In the step D, the traceable code of the inspected object information consists of an 8-digit calibration date code, a 2-digit entrusting unit number, a 4-digit calibration time, a 2-digit weight specification number, a 3-digit placement information of the kilogram group weights before calibration, a 12-digit position information of the kilogram group weights before calibration, a 2-digit calibration scheduling information of the kilogram group weights, a 2-digit calibration result information of the kilogram group weights, a 3-digit placement information of the kilogram group weights after calibration, a 12-digit position information of the kilogram group weights after calibration, and a 1-digit check code, totaling 50 decimal digits + 1 hexadecimal digit.

9. An automatic calibration system for kilogram weights based on the collaboration of three manipulators, characterized in that: include: Weight real-time identification and positioning module, three-manipulator collaborative scheduling module, inspected object information encoding module and master control host computer module; The weight real-time identification and positioning module is used to collect natural light images and depth images of the stacked kilogram weights, and identify the weight positions in real time based on the Faster R-CNN algorithm, and output the three-dimensional coordinates of each weight; The three-manipulator collaborative scheduling module is used to complete the tasks of grabbing, carrying and placing unchecked weights, standard weights and checked weights respectively through the three manipulators; The inspected object information encoding module is used to perform label-free traceability encoding on the weight information during the verification process, and generate a traceable code containing the basic information of the weight, real-time location information and verification results; The master control host computer module is used to coordinate the coordinated operation of the weight real-time identification and positioning module, the three-manipulator collaborative scheduling module and the inspected object information encoding module, send control instructions and monitor the inspection process in real time.

Images