A Chip Positioning Algorithm Applicable to Turret Sorting Machines

Abstract

The present invention relates to the field of chip positioning, and specifically to a chip positioning algorithm suitable for a turret type sorter, which includes the following steps: S1. Inclining the vision module to obtain several consecutive first inclined images in real time; S2. Screening the several first inclined images, and marking the screened first inclined images as second inclined images; S3. Analyzing the motion state of the target chip according to the second inclined images; S4. Calculating the actual position of the target chip according to the motion state of the target chip and the second inclined images; The present invention takes into account that there are still deviations after determining the position of the chip according to the motion direction and speed of the chip, so double constraints of the angle between the key edges and the key edges are used, and the key points with low reliability will be automatically downweighted to avoid the influence of some key points with large errors on the whole. In addition, the calculation method of the present invention only optimizes the key points instead of the whole image, which can improve the calculation accuracy while reducing the calculation amount.

Description

Technical Field

[0001] The present invention relates to the field of chip positioning, and specifically to a chip positioning algorithm for a turret type sorter. Background Art

[0002] Chips are generally fed by vibrating feeding, and then the sorter picks up the chips. When the sorter picks up the chips, it generally takes pictures of the chip positions through a camera to calculate the chip positions, and then controls the suction nozzle to pick up the chips. However, when the camera takes pictures of the chips, the vibrating feeding tray is very likely to be still in a vibrating state, so there are large errors in the taken pictures, and there will also be chip gaps, which is not conducive to determining the chip positions. When the suction nozzle sucks the chips, it is easy to suck them obliquely or suck nothing. Summary of the Invention

[0003] Aiming at the deficiencies of the prior art, the present invention provides a chip positioning algorithm for a turret type sorter, which solves the technical problems in the above background art.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] A chip positioning algorithm for a turret type sorter includes the following steps:

[0006] S1. Incline the vision module to obtain several consecutive first inclined images in real time;

[0007] S2. Screen several first inclined images, and mark the selected first inclined images as second inclined images;

[0008] In step S2, it specifically includes the following steps:

[0009] S21. Take the center point of the picking position as the origin 0 to construct a real coordinate system. In the real coordinate system, the east-west direction is the x-axis, the north-south direction is the y-axis, and the vertical direction is the z-axis;

[0010] S22. Construct a standard chip edge map according to the chip shape. The standard chip edge map includes several standard key points and standard key edges, and mark the coordinate positions of each standard key point in the real coordinate system;

[0011] S23. Use the openpose algorithm to extract the to-be-measured key points in the first inclined image, use the to-be-measured key edges to connect the to-be-measured key points to generate a to-be-measured map, and calculate the coordinate positions of each to-be-measured key point in the real coordinate system;

[0012] S24. Divide several first inclined images into multiple image sets to be calculated in chronological order, and calculate the average value of the image set similarity of each image set to be calculated;

[0013] S25. Select the set of images to be calculated with the highest average similarity, and mark the first tilted image among them as the second tilted image;

[0014] S3. Analyze the motion state of the target chip based on the second tilted image;

[0015] S4. Calculate the actual position of the target chip based on the motion state of the target chip and the second tilted image.

[0016] Further, in step S1, the first tilted images are sorted in chronological order, and the time interval between every two adjacent first tilted images is t.

[0017] Further, in step S23, it specifically includes the following steps:

[0018] S231. Take the center point of the first tilted image as the origin Construct a camera coordinate system, with the horizontal and vertical directions of the first tilted image as the horizontal and vertical axes of the camera coordinate system;

[0019] S232. Obtain the camera internal parameter matrix K of the vision module, and its expression is: , where and respectively represent the focal lengths of the vision module on the horizontal and vertical axes of the camera; and respectively represent the coordinates of the plane center of the first tilted image;

[0020] S233. Determine the homogeneous coordinates of the key points to be measured in the camera coordinate system , and its expression is: , where and respectively represent the horizontal and vertical axis coordinates of the key points to be measured in the camera coordinate system; represents the transpose operation of the matrix;

[0021] S234. Calculate the coordinate position of the key points to be measured in the real coordinate system according to the camera internal parameter matrix K and the homogeneous coordinates , and its calculation formula is: , where represents the three-dimensional coordinates of the key points to be measured in the real coordinate system, ; represents the depth value of the key points to be measured from the optical axis of the vision module; in the present invention, is obtained by detecting with a depth sensor provided in the vision module.

[0022] Further, in step S24, it specifically includes the following steps:

[0023] S241. Calculate the key point position difference between each key point to be measured and the corresponding standard key point , and its calculation formula is: , where in the formula, and respectively represent the Euclidean distances of the i-th standard key point and the key point to be measured;

[0024] S242. Calculate the average key edge difference between each key edge to be measured and the corresponding standard key edge , and its calculation formula is: , where in the formula, represents the total number of standard key points; represents the Euclidean distance between the i-th standard key point and the j-th standard key point in the standard chip edge map; represents the Euclidean distance between the i-th key point to be measured and the j-th key point to be measured in the map to be measured;

[0025] S243. Calculate the average difference between the included angles of every two key edges to be measured and the included angles of the corresponding two standard key edges , and its calculation formula is: , where in the formula, represents the included angle between the i-th standard key edge and the j-th standard key edge in the standard chip map; represents the included angle between the i-th key edge to be measured and the j-th key edge to be measured in the map to be measured;

[0026] S244. Calculate the similarity between the map to be measured and the standard chip edge map according to the key point position difference , the average key edge difference and the average included angle difference , and its calculation formula is: , where in the formula, , , and respectively represent the first, second, and third weight coefficients with respect to ;

[0027] S245. Set a first sliding window in a number of frames of the first tilted image. The length of the first sliding window is b, and the first sliding window slides one frame each time to obtain a number of image sets to be calculated;

[0028] S246. Calculate the similarity between each map to be measured and the standard chip edge map in each image set to be calculated , and calculate the average value of the image set similarity according to it.

[0029] Furthermore, in step S3, it specifically includes the following steps:

[0030] S31. Obtain several consecutive historical tilted images captured by the vision module, set a second sliding window with a length of b, and slide the second sliding window by one frame each time to obtain b historical tilted images as training samples;

[0031] S32. Mark the movement direction and movement speed of the chip at the next moment in each training sample through manual marking, and use them as sample labels;

[0032] S33. Train the long short-term memory network with the training samples and sample labels to obtain the target model;

[0033] S34. Input all the second tilted images into the target model, and output the movement direction and movement speed of the target chip at this time.

[0034] Further, in step S4, it specifically includes the following steps:

[0035] S41. Obtain the system delay time of the vision module ;

[0036] S42. According to the system delay time and the movement state of the target chip, correct the three-dimensional coordinates of each key point to be measured in the real coordinate system to obtain the corrected key points, and its calculation formula is:

[0037] ;

[0038] Expand it to: ,

[0039] In the formula, represents the three-dimensional coordinates of the corrected key point; ; represents the three-dimensional coordinates of the key point to be measured; ; represents the three-dimensional velocity vector of the chip in the real coordinate system; represents the polar angle of the movement direction of the chip; represents the azimuth angle of the movement direction of the chip;

[0040] S43. Construct corrected key edges according to the corrected key points;

[0041] S44. Calculate the comprehensive reliability of each corrected key point according to the length of the corrected key edge and the angle between adjacent corrected key edges ;

[0042] S45. Select the two corrected key points with the highest comprehensive reliability as the basic key points, and construct a determined chip map according to the basic key points to determine the actual position of the target chip with the determined chip map.

[0043] Further, in step S44, it specifically includes the following steps:

[0044] S441. Calculate the side length deviation of each calibration key edge , and its calculation formula is: , where in the formula represents the length of the calibration key edge ij; represents the length of the standard key edge ij;

[0045] S442. Calculate the included angle deviation of the included angle between every two adjacent calibration key edges , and its calculation formula is: ; where , in the formula represents the included angle between the calibration key edge ij and the calibration key edge ik; represents the included angle between the standard key edge ij and the standard key edge ik; represents function; , and respectively represent the three-dimensional coordinates of the i-th, j-th, and k-th calibration key points;

[0046] S443. Calculate the side length reliability of each calibration key edge according to the side length deviation , and its calculation formula is: ; where , in the formula , represents the average length deviation of all adjacent calibration key edges of the i-th calibration key point; represents about attenuation control parameter; represents the number of adjacent calibration key edges of the i-th calibration key point; represents the set of all adjacent calibration key edges of the i-th calibration key point; represents function;

[0047] S444. Calculate the angle reliability of each calibration key point according to the included angle deviation , and its calculation formula is: ; where , in the formula , represents the average angle deviation between all adjacent calibration key edges of the i-th calibration key point; represents about attenuation control parameter; represents the number of all adjacent calibration key edges of the i-th calibration key point; Denote the set of all adjacent calibration key edges of the $i$-th calibration key point;

[0048] S445. According to the side length reliability and the angle reliability calculate the comprehensive reliability of each calibration key point .

[0049] Furthermore, in step S445, the calculation formula of the comprehensive reliability is: , where in the formula, denotes the weight coefficient with respect to .

[0050] Compared with the prior art, the present invention provides a sorting machine chip positioning algorithm applicable to a turret type, having the following beneficial effects:

[0051] 1. The present invention takes into account that there are still deviations after determining the position of the chip according to the movement direction and speed of the chip. Therefore, double constraints of the angle between the key edges and the key edges are used, and the key points with low reliability will be automatically de-weighted to avoid the influence of some key points with large errors on the whole. In addition, the calculation method of the present invention only optimizes for key points rather than the whole graph, which can improve the calculation accuracy of the chip position while reducing the calculation amount to avoid the situation of the nozzle sucking empty or sucking skew.

[0052] 2. When calculating the similarity between the measured graph and the standard chip edge graph, the present invention not only considers the accuracy of the position of each key point, but also considers the accuracy of the length and included angle of the key edges. In addition, the present invention also considers the factor of motion blur. Compared with common calculation methods, the calculation accuracy of the similarity between the measured graph and the standard chip edge graph is higher, and the first tilted image in which the chip is moving rapidly can be excluded to facilitate screening out the measured graph closest to the standard chip edge graph.

[0053] 3. The vision module of the present invention can avoid the influence of the vision module on the number of nozzle components by being tilted. Compared with common vision module setting methods, nozzle components can be added, thereby improving the chip picking efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0055] Figure 1 is a flowchart of a sorting machine chip positioning algorithm applicable to a turret type of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] In order to make the above objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Thereby, the implementation process of how this application uses technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0057] Those of ordinary skill in the art can understand that all or part of the steps in the following embodiment methods can be completed by instructing relevant hardware through a program. Therefore, this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0058] When loading the chips onto the chip, a vibrating feeder can be used to sequentially transfer the chips from a disordered state to the picking position, and then the vision module positions the chips. Finally, the nozzle assembly is fine-tuned according to the chip position to facilitate the nozzle to pick up the chips. However, since the vibrating feeder tray feeds the chips through vibration, when the vibrating feeder tray stops, the entire device may still be vibrating. If the above factors are not considered, when using the picking scheme mentioned above, problems such as the nozzle not being firmly adsorbed and the adsorption position deviation are very likely to occur. Therefore, as Figure 1 shown, the present invention proposes a chip positioning algorithm applicable to a turret sorter, including the following steps:

[0059] S1. Incline the vision module to obtain several consecutive first inclined images in real time; specifically, since the nozzle assembly is generally installed on a rotating disk, if the vision module is vertically arranged, it will affect the number of rotating disks. Therefore, the vision module is inclined to facilitate increasing the number of nozzle assemblies. It should be noted that in addition, in step S1, the first inclined images are sorted in chronological order, and the time interval between every two adjacent first inclined images is t.

[0060] By inclining the vision module of the present invention, the influence of the vision module on the number of nozzle assemblies can be avoided. Compared with the common vision module setting method, the nozzle assembly can be added, thereby improving the chip picking efficiency.

[0061] S2. Screen several first tilted images, and mark the selected first tilted images as second tilted images. Specifically, when the vision module takes pictures of the picking position, only the first tilted images in which the chip reaches the picking position are available. Under the action of the vibrating feeding tray, both the chip and the picking position are in a vibrating state, so the images obtained by the vision module are likely to be motion blurred. Therefore, in step S2, it specifically includes the following steps:

[0062] S21. Construct a real-world coordinate system with the center point of the picking position as the origin 0. In the real-world coordinate system, the east-west direction is the x-axis, the north-south direction is the y-axis, and the vertical direction is the z-axis.

[0063] S22. Construct a standard chip edge map according to the chip shape. The standard chip edge map includes several standard key points and standard key edges, and mark the coordinate positions of each standard key point in the real-world coordinate system.

[0064] S23. Use the openpose algorithm to extract the key points to be measured in the first tilted image, use the key edges to be measured to connect the key points to be measured to generate a to-be-measured map, and calculate the coordinate positions of each key point to be measured in the real-world coordinate system. Specifically, in step S23, it specifically includes the following steps:

[0065] S231. Construct a camera coordinate system with the center point of the first tilted image as the origin Take the horizontal direction and vertical direction of the first tilted image as the horizontal axis and vertical axis of the camera coordinate system.

[0066] S232. Obtain the camera internal parameter matrix K of the vision module, and its expression is: , where and respectively represent the focal lengths of the vision module on the horizontal axis and vertical axis of the camera; and respectively represent the coordinates of the plane center of the first tilted image;

[0067] S233. Determine the homogeneous coordinates of the key points to be measured in the camera coordinate system, and its expression is: , where and respectively represent the horizontal axis coordinate and vertical axis coordinate of the key point to be measured in the camera coordinate system; represents the transpose operation of the matrix;

[0068] S234. Calculate the coordinate position of the key point to be measured in the real-world coordinate system according to the camera internal parameter matrix K and the homogeneous coordinates , and its calculation formula is: , where Represents the three-dimensional coordinates of the key point to be measured in the real coordinate system, ; Represents the depth value of the key point to be measured from the optical axis of the vision module; in the present invention, It is obtained by detecting with a depth sensor provided in the vision module.

[0069] S24. Divide a number of first tilted images into multiple image sets to be calculated according to the time sequence, and calculate the average similarity of each image set to be calculated; specifically, due to physical factors of the vision module (including perspective distortion, lens distortion, and insufficient resolution) and dynamic factors during movement (including motion blur and frame rate mismatch), etc., when using the openpose algorithm to extract the key points to be measured in the first tilted image and then construct the image to be measured, there will be a certain degree of deformation. The physical factors of the vision module are often approximated, while the dynamic factors will exacerbate the generation of deformation. Therefore, in step S24, it specifically includes the following steps:

[0070] S241. Calculate the key point position difference between each key point to be measured and the corresponding standard key point , and its calculation formula is: , where in the formula, and respectively represent the Euclidean distances of the i-th standard key point and the key point to be measured;

[0071] S242. Calculate the average key edge difference between each key edge to be measured and the corresponding standard key edge , and its calculation formula is: , where in the formula, represents the total number of standard key points; represents the Euclidean distance between the i-th standard key point and the j-th standard key point in the standard chip edge map; represents the Euclidean distance between the i-th key point to be measured and the j-th key point to be measured in the image to be measured;

[0072] S243. Calculate the average difference between the included angle of every two key edges to be measured and the included angle between the corresponding two standard key edges , and its calculation formula is: , where in the formula, represents the included angle between the i-th standard key edge and the j-th standard key edge in the standard chip map; represents the included angle between the i-th key edge to be measured and the j-th key edge to be measured in the image to be measured; it should be noted that the i-th standard key edge and the j-th standard key edge are adjacent;

[0073] S244. According to the key point position difference , the average key edge difference And the average difference in included angle Calculate the similarity between the to-be-tested image and the edge image of the standard chip , and its calculation formula is: , where in the formula , and respectively represent the first, second, and third weight coefficients regarding ; in the present invention, , and are 0.45, 0.3, and 0.25 respectively;

[0074] S245. Set a first sliding window in several frames of first inclined images. The length of the first sliding window is b, and the first sliding window slides one frame each time to obtain several sets of to-be-calculated images;

[0075] S246. Calculate the similarity between each to-be-tested image in each set of to-be-calculated images and the edge image of the standard chip , and calculate the average value of the image set similarity based on this.

[0076] S25. Select the set of to-be-calculated images with the highest average similarity value, and mark the first inclined image therein as the second inclined image.

[0077] When calculating the similarity between the to-be-tested image and the edge image of the standard chip in step S2 of the present invention, not only the accuracy of the position of each key point is considered, but also the accuracy of the key edge length and included angle is considered. In addition, the present invention also considers the factor of motion blur. Compared with common calculation methods, the calculation accuracy of the similarity between the to-be-tested image and the edge image of the standard chip is higher, and the first inclined images in which the chip is in fast motion can be excluded, so as to facilitate the screening of the to-be-tested image closest to the edge image of the standard chip.

[0078] S3. Analyze the motion state of the target chip according to the second inclined image; specifically, since the vibrating feeding tray may be in a continuous vibrating state, it is necessary to analyze the actual motion state of the chip in order to calculate the actual position of the chip based on this later. Therefore, in step S3, it specifically includes the following steps:

[0079] S31. Obtain several consecutive historical inclined images captured by the vision module, set a second sliding window, the length of the second sliding window is b, and the second sliding window slides one frame each time to obtain b frames of historical inclined images as training samples;

[0080] S32. Mark the motion direction and motion speed of the chip at the later moment of each training sample by means of manual marking, and use them as sample labels;

[0081] S33. Train the long short - term memory network using training samples and sample labels to obtain the target model;

[0082] S34. Input all the second tilted images into the target model and output the movement direction and movement speed of the target chip at this time.

[0083] S4. Calculate the actual position of the target chip according to the movement state of the target chip and the second tilted image; specifically, in step S4, it specifically includes the following steps:

[0084] S41. Obtain the system delay time of the vision module ;

[0085] S42. According to the system delay time and the movement state of the target chip, correct the three - dimensional coordinates of each key point to be measured in the real - world coordinate system to obtain the corrected key points, and its calculation formula is:

[0086] ;

[0087] Expand it to: ,

[0088] In the formula, represents the three - dimensional coordinates of the corrected key point; ; represents the three - dimensional coordinates of the key point to be measured; ; represents the three - dimensional velocity vector of the chip in the real - world coordinate system; represents the polar angle of the movement direction of the chip; represents the azimuth angle of the movement direction of the chip;

[0089] S43. Construct corrected key edges according to the corrected key points;

[0090] S44. Calculate the comprehensive reliability of each corrected key point according to the length of the corrected key edge and the angle between adjacent corrected key edges ; specifically, since the calculation method mentioned above is a simplified calculation model and cannot cover all movements of the chip, there are still errors in its calculation results. Therefore, in step S44, it specifically includes the following steps:

[0091] S441. Calculate the side - length deviation of each corrected key edge , and its calculation formula is: ;

[0092] In the formula, represents the length of the corrected key edge ij; represents the length of the standard key edge ij; specifically, It can be directly calculated through the three-dimensional coordinates of the i-th and j-th calibration key points; the calibration key edge ij represents the calibration key edge between the i-th and j-th calibration key points;

[0093] S442. Calculate the angle deviation between the angles of every two adjacent calibration key edges , and its calculation formula is: ; where , in the formula, represents the angle between the calibration key edge ij and the calibration key edge ik; represents the angle between the standard key edge ij and the standard key edge ik; represents function; , and respectively represent the three-dimensional coordinates of the i-th, j-th, and k-th calibration key points;

[0094] S443. Calculate the side length reliability of each calibration key edge according to the side length deviation , and its calculation formula is: ; where ; in the formula, , represents the average length deviation of all adjacent calibration key edges of the i-th calibration key point; represents the attenuation control parameter with respect to ; represents the number of adjacent calibration key edges of the i-th calibration key point; represents the set of all adjacent calibration key edges of the i-th calibration key point; represents represents function; in the present invention, is 0.1;

[0095] S444. Calculate the angle reliability of each calibration key point according to the angle deviation , and its calculation formula is: ; where ; in the formula, , represents the average angle deviation between all adjacent calibration key edges of the i-th calibration key point; represents the attenuation control parameter with respect to ; represents the number of all adjacent calibration key edges of the i-th calibration key point; represents the set of all adjacent calibration key edges of the i-th calibration key point; in the present invention, is 1; ;

[0096] S445. According to the side length reliability and angular reliability Calculate the comprehensive reliability of each calibration key point Specifically, in step S445, the comprehensive reliability The calculation formula is: , where represents the weight coefficient with respect to ; in the present invention, is 0.7.

[0097] S45. Select the two calibration key points with the highest comprehensive reliability as the basic key points, and construct and determine the chip map based on the basic key points, and use the determined chip map as the actual position of the target chip.

[0098] In step S4 of the present invention, considering that there are still deviations after determining the position of the chip according to the movement direction and speed of the chip, so the double constraint of the angle between the key edges and the key edges is used, and the key points with low reliability will be automatically down-weighted to avoid the influence of some key points with large errors on the whole. In addition, the calculation method of the present invention only optimizes the key points rather than the whole map, which can reduce the calculation amount while improving the calculation accuracy.

[0099] The above embodiments have introduced the present invention in detail. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A chip positioning algorithm applicable to a turret type sorting machine, characterized in that, Including the following steps: S1. Set the vision module obliquely to obtain several consecutive first oblique images in real time; S2. Screen several first oblique images, and mark the selected first oblique images as second oblique images; In step S2, it specifically includes the following steps: S21. Construct a real-world coordinate system with the center point of the picking position as the origin 0. In the real-world coordinate system, the east-west direction is the x-axis, the north-south direction is the y-axis, and the vertical direction is the z-axis; S22. Construct a standard chip edge map according to the chip shape. The standard chip edge map includes several standard key points and standard key edges, and mark the coordinate positions of each standard key point in the real-world coordinate system; S23. Use the openpose algorithm to extract the key points to be measured in the first oblique image, use the key edges to be measured to connect the key points to be measured to generate a to-be-measured map, and calculate the coordinate positions of each key point to be measured in the real-world coordinate system; S24. Divide several first oblique images into multiple image sets to be calculated in chronological order, and calculate the average similarity of each image set to be calculated; In step S24, it specifically includes the following steps: S241. Calculate the key-point position difference between each key point to be measured and the corresponding standard key point , and its calculation formula is: , where in the formula, and respectively represent the Euclidean distances of the i-th standard key point and the key point to be measured; S242. Calculate the average key-edge difference between each key-edge to be measured and the corresponding standard key-edge , and its calculation formula is: , where in the formula, represents the total number of standard key points; represents the Euclidean distance between the i-th and j-th standard key points in the standard chip edge map; represents the Euclidean distance between the i-th and j-th key points to be measured in the map to be measured; S243. Calculate the average difference between the angles of every two key edges to be measured and the angles between the corresponding two standard key edges. , and its calculation formula is: , where in the formula, represents the angle between the i-th standard key edge and the j-th standard key edge in the standard chip diagram; represents the angle between the i-th key edge to be measured and the j-th key edge to be measured in the diagram to be measured. S244. Calculate the similarity between the image to be measured and the edge image of the standard chip according to the position difference of key points , the average difference of key edges and the average difference of included angles . The calculation formula is as follows: , where , in the formula, , and respectively represent the first, second and third weight coefficients with respect to ; S245. Set a first sliding window in several frames of first oblique images. The length of the first sliding window is b, and the first sliding window slides one frame each time to obtain several image sets to be calculated; S246. Calculate the similarity between each test image in each image set to be calculated and the standard chip edge image, and calculate the average value of the image set similarity based on this; S25. Select the image set to be calculated with the highest average similarity, and mark the first oblique image therein as the second oblique image; S3. Analyze the motion state of the target chip according to the second oblique image; S4. Calculate the actual position of the target chip according to the motion state of the target chip and the second oblique image.

2. The sorting machine chip positioning algorithm applicable to a turret according to claim 1, wherein, In step S1, the first oblique images are sorted in chronological order, and the time interval between every two adjacent first oblique images is t.

3. The sorting machine chip positioning algorithm applicable to a turret according to claim 1, characterized in that In step S23, it specifically includes the following steps: S231. Taking the center point of the first tilted image as the origin Construct a camera coordinate system, with the horizontal direction and the vertical direction of the first tilted image as the horizontal axis and the vertical axis of the camera coordinate system; S232. Obtain the camera internal parameter matrix K of the vision module, and its expression is: , where and respectively represent the focal lengths of the vision module on the horizontal axis and the vertical axis of the camera; and respectively represent the coordinates of the plane center of the first tilted image; S233. Determine the homogeneous coordinates of the key point to be measured in the camera coordinate system , and its expression is: , where and respectively represent the horizontal axis coordinate and the vertical axis coordinate of the key point to be measured in the camera coordinate system; represents the transpose operation of the matrix; S234. Calculate the coordinate position of the key point to be measured in the real coordinate system according to the camera internal parameter matrix K and the homogeneous coordinates . The calculation formula is as follows: , where represents the three-dimensional coordinates of the key point to be measured in the real coordinate system, ; represents the depth value of the key point to be measured from the optical axis of the vision module; in the present invention, is obtained by detecting with a depth sensor provided in the vision module.

4. The sorting machine chip positioning algorithm applicable to a turret according to claim 1, characterized in that, In step S3, it specifically includes the following steps: S31. Obtain several consecutive historical oblique images taken by the vision module, set a second sliding window, the length of the second sliding window is b, and the second sliding window slides one frame each time to obtain b historical oblique images as training samples; S32. Manually mark the motion direction and motion speed of the chip in each training sample at the next moment of the training sample, and use it as the sample label; S33. Use the training samples and sample labels to train the long short-term memory network to obtain the target model; S34. Input all the second oblique images into the target model, and output the motion direction and motion speed of the target chip at this time.

5. The sorting machine chip positioning algorithm applicable to a turret according to claim 1, characterized in that In step S4, it specifically includes the following steps: S41. Obtain the system latency time of the vision module ; S42. According to the system delay time and the motion state of the target chip, correct the three-dimensional coordinates of each key point to be measured in the real coordinate system to obtain the corrected key points, and its calculation formula is: ； Expand it to: , In the formula, represents the three-dimensional coordinates of the calibration key point; ; represents the three-dimensional coordinates of the key point to be measured; ; represents the three-dimensional velocity vector of the chip in the real coordinate system; represents the polar angle of the movement direction of the chip; represents the azimuth angle of the movement direction of the chip; S43. Construct a corrected key edge according to the corrected key points; S44. Calculate the comprehensive reliability of each calibration key point based on the length of the calibrated key edge and the angle between adjacent calibrated key edges ; S45. Select the comprehensive reliability Use the two calibration key points with the highest reliability as the basic key points, and construct a chip map based on the basic key points to determine the actual position of the target chip with the determined chip map.

6. The sorting machine chip positioning algorithm applicable to a turret according to claim 5, characterized in that In step S44, it specifically includes the following steps: S441. Calculate the side length deviation of each calibration key edge , and its calculation formula is: , where represents the length of the calibration key edge ij; represents the length of the standard key edge ij; S442. Calculate the angular deviation of the angle between every two adjacent calibration key edges , and its calculation formula is: ; where , in the formula, represents the angle between calibration key edge ij and calibration key edge ik; represents the angle between standard key edge ij and standard key edge ik; represents function; , and respectively represent the three-dimensional coordinates of the i-th, j-th, and k-th calibration key points; S443. Calculate the side length reliability of each calibration key edge according to the side length deviation The calculation formula is as follows: , and the formula is: where, in the formula, represents the average length deviation of all adjacent calibration key edges of the i-th calibration key point; represents the attenuation control parameter with respect to ; represents the number of adjacent calibration key edges of the i-th calibration key point; represents the set of all adjacent calibration key edges of the i-th calibration key point; represents function; S444. Calculate the angular reliability of each calibration key point according to the included angle deviation The calculation formula is as follows: ; where In the formula, represents the average angular deviation between all adjacent calibration key edges of the i-th calibration key point; represents the attenuation control parameter with respect to ; represents the number of all adjacent calibration key edges of the i-th calibration key point; represents the set of all adjacent calibration key edges of the i-th calibration key point;​ S445. Calculate the comprehensive reliability of each calibration key point according to the side length reliability and the angle reliability . .

7. The sorting machine chip positioning algorithm applicable to a turret according to claim 6, characterized in that, In step S445, the comprehensive reliability is calculated by the formula: , where represents the weight coefficient with respect to .

Images