Method and system for visual positioning of a moving object

Abstract

The embodiment of the application relates to the technical field of automatic production, and discloses a visual positioning method and a positioning system for a moving object, the method comprising: setting at least two positioning reference points on the moving object, and acquiring geometric position relationships between the at least two positioning reference points and a reference point of the moving object; in the process of movement of the moving object, recognizing positions of the at least two positioning reference points; and according to the recognized positions of the positioning reference points and the geometric position relationships, determining the position of the reference point of the moving object at an imaging moment. The embodiment of the application establishes the rigidly connected positioning reference points on the moving object, reverses the accurate position of the reference point of the moving object according to the actual position of the positioning reference point at the imaging moment, eliminates the influence of communication delay and imaging delay on positioning accuracy, improves the accuracy of dynamic visual positioning, simultaneously realizes accurate positioning of the moving object in the process of high-speed movement, shortens a single positioning cycle, and improves production efficiency.

Description

Technical Field

[0001] This application relates to the field of automated production technology, and in particular to a visual positioning method and positioning system for moving objects. Background Technology

[0002] In the field of modern automated production, robotic auxiliary material attaching equipment is widely used in industries such as electronics and communications, and precision manufacturing. It is mainly used to accurately attach various auxiliary materials to product brackets. As global manufacturing competition intensifies and cost pressures increase, enterprises continue to raise their requirements for the production efficiency and precision of automated equipment, driving the development of robot vision positioning technology from static photography to dynamic aerial photography.

[0003] In implementing the embodiments of this application, the inventors discovered that: Currently, traditional robotic attachment systems use a fixed-position photography method. The robotic arm must decelerate and stop above the camera to complete the photography and positioning before initiating the attachment action. The time consumed by the robot's deceleration, stopping, photography, and re-acceleration during this process significantly reduces the overall efficiency of the equipment. Although the flying photography technology can complete the photography during robot movement, avoiding downtime losses, due to communication delays and camera imaging delays, there is an error between the position of the robot when it triggers the photography command and the actual position of the camera when it captures the image. Especially in high-speed movement, the error can reach more than five millimeters, seriously affecting the attachment accuracy. Summary of the Invention

[0004] The main technical problem addressed by this application is to provide a visual positioning method for animals, which can solve the positional error problem caused by communication delay and imaging delay in the aerial photography process in the prior art.

[0005] To address the aforementioned technical problems, one technical solution adopted in this application embodiment is: providing a visual positioning method for a moving object, comprising setting at least two positioning reference points on the moving object, wherein the positioning reference points and a reference point of the moving object have a fixed geometric positional relationship; obtaining the geometric positional relationship between the at least two positioning reference points and the reference point of the moving object; identifying the positions of the at least two positioning reference points through a vision device during the movement of the moving object; and determining the position of the reference point of the moving object at the imaging moment based on the identified positions of the positioning reference points and the geometric positional relationship.

[0006] Optionally, the at least two positioning reference points have a fixed geometric positional relationship with the reference point of the moving object, including a distance relationship and an angular relationship.

[0007] Optionally, the distance relationship is the distance between the reference point of the moving object and the first positioning reference point among the at least two positioning reference points, and the angle relationship is the angle between the line connecting the reference point of the moving object and the first positioning reference point and the line connecting the second positioning reference point and the first positioning reference point.

[0008] Optionally, the step of obtaining the geometric positional relationship between the at least two positioning reference points and the reference point of the moving object further includes: when the moving object is in a stationary state, obtaining the position coordinates of the reference point of the moving object and the position coordinates of the at least two positioning reference points through a vision device, and calculating the geometric positional relationship based on the position coordinates.

[0009] Optionally, the step of calculating the geometric positional relationship based on the position coordinates further includes: calculating the distance based on the position coordinates of the reference point of the moving object and the position coordinates of the first positioning reference point, and calculating the included angle using the arctangent function based on the position coordinates of the reference point of the moving object, the position coordinates of the first positioning reference point, and the position coordinates of the second positioning reference point.

[0010] Optionally, the step of determining the position of the reference point of the moving object at the imaging time based on the identified positioning reference point position and the geometric position relationship further includes: calculating the direction angle of the second positioning reference point relative to the first positioning reference point based on the identified first positioning reference point position and the second positioning reference point position; adding the direction angle to the included angle to obtain the direction angle of the reference point of the moving object relative to the first positioning reference point; and calculating the position coordinates of the reference point of the moving object at the imaging time using trigonometric functions based on the direction angle of the reference point of the moving object relative to the first positioning reference point and the distance.

[0011] Optionally, the number of the at least two positioning reference points is two.

[0012] Optionally, the moving object is a robot feed nozzle, and the reference point of the moving object is the center of the robot's rotation axis.

[0013] Optionally, the method further includes: after determining the position of the robot's rotation axis center at the imaging moment, identifying the position of the auxiliary material at the imaging moment through a vision device, and determining the target position for the robot to perform the attachment operation based on the relative relationship between the position of the robot's rotation axis center at the imaging moment and the position of the auxiliary material at the imaging moment.

[0014] To solve the above-mentioned technical problems, another technical solution adopted in the embodiments of this application is: to provide a visual positioning system, which is used to perform any of the above-mentioned visual positioning methods for moving objects.

[0015] The visual positioning method for moving objects provided in this application establishes a fixed geometric positional relationship between the positioning reference points and the reference point of the moving object by setting at least two positioning reference points on the moving object. During the movement of the moving object, the position of the positioning reference points is identified by a vision device. Based on the identified position of the positioning reference points and the geometric positional relationship, the position of the reference point of the moving object at the imaging moment is determined. This solves the positional error problem caused by communication delay and imaging delay in the prior art during the aerial shooting process. By establishing a rigidly connected positioning reference point as a geometric reference on the moving object, the accurate position of the reference point of the moving object at the imaging moment can be deduced from the actual position of the positioning reference point, regardless of the time delay between the moment the vision device triggers the shooting command and the actual imaging moment. This fundamentally eliminates the impact of communication delay and imaging delay on positioning accuracy, greatly improving the accuracy of dynamic visual positioning. Furthermore, the positioning method allows the moving object to complete visual positioning during high-speed movement without deceleration or stopping. Compared with the traditional fixed-position shooting method, it eliminates the time consumed by the moving object's deceleration, stopping, shooting, and re-acceleration, significantly shortening the single positioning cycle and improving production efficiency. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0017] Figure 1 This is an environmental diagram illustrating the visual localization method for moving objects. Figure 2 It is a coordinate reference diagram for the visual positioning method of moving objects; Figure 3 This is a flowchart of a visual localization method for moving objects; Figure 4 This is a further flowchart of step S102 of the visual localization method for moving objects; Figure 5 This is a flowchart of another embodiment of step S102 of the visual positioning method for moving objects; Figure 6 This is a flowchart of another embodiment of step S102 of the visual positioning method for moving objects; Figure 7 This is a flowchart of another embodiment of the visual positioning method for moving objects.

[0018] The reference numerals in the detailed embodiments are as follows: 10, robot R-axis; 20, auxiliary material suction nozzle; 30, first positioning point; 40, second positioning point; 50, auxiliary material; 60, auxiliary material suction nozzle. Detailed Implementation

[0019] To facilitate understanding of this application, a more detailed description is provided below with reference to the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on the other element, or one or more intermediate elements may exist between them. When an element is described as "connected" to another element, it can be directly connected to the other element, or one or more intermediate elements may exist between them. The terms "upper," "lower," "inner," "outer," "vertical," "horizontal," etc., used in this specification indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0021] Furthermore, the technical features involved in the different embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0022] Please see Figure 1 and Figure 2 , Figure 1 This diagram illustrates the overall structure of the application environment in this embodiment. Figure 2 A geometric diagram illustrating the calibration phase is shown. In this embodiment, the moving object is the robot's auxiliary material suction nozzle 20, which is fixedly mounted on the robot's R-axis 10. The robot's R-axis 10 is the robot's rotation axis, capable of driving the auxiliary material suction nozzle 20 to rotate. During production, the auxiliary material suction nozzle 60 is used to pick up the auxiliary material 50. The robot's R-axis 10 drives the auxiliary material suction nozzle 20 to a predetermined position, attaching the auxiliary material 50 to the bracket product. To achieve precise attachment, it is necessary to accurately obtain the position of the auxiliary material 50 and the position of the robot during the robot's movement.

[0023] A first positioning point 30 and a second positioning point 40 are provided on the auxiliary material suction nozzle 20. The first positioning point 30 and the second positioning point 40 are fixedly installed on the auxiliary material suction nozzle 20 in the form of positioning pins, forming a rigid connection with the auxiliary material suction nozzle 20. Since the auxiliary material suction nozzle 20 is fixedly installed on the robot's R-axis 10, the relative positional relationship between the first positioning point 30 and the second positioning point 40 and the rotation axis center of the robot's R-axis 10 remains fixed. A vision device is set below the robot's motion trajectory to capture image information of the first positioning point 30, the second positioning point 40, and the auxiliary material 50 during the robot's movement.

[0024] Please continue reading. Figure 2 and Figure 3 , Figure 3 A flowchart of a visual positioning method for a moving object according to this embodiment is shown. The visual positioning method for a moving object according to this embodiment includes the following steps.

[0025] Step S101: Set at least two positioning reference points on the moving object, wherein the positioning reference points and the reference point of the moving object have a fixed geometric positional relationship; In this embodiment, a first positioning point 30 and a second positioning point 40 are set on the robot's auxiliary material suction nozzle 20 as positioning reference points, and the reference point of the moving object is the center of the rotation axis of the robot's R-axis 10. The first positioning point 30 and the second positioning point 40 are rigidly fixed on the auxiliary material suction nozzle 20 by positioning pins, so the distance and angle relationship between the first positioning point 30, the second positioning point 40 and the center of the rotation axis of the robot's R-axis 10 remains constant.

[0026] Step S102: Obtain the geometric positional relationship between the at least two positioning reference points and the reference point of the moving object; During the calibration phase before production begins, the robot's R-axis 10 is kept stationary. The position coordinates of the center of rotation of the robot's R-axis 10 in the vision device's coordinate system are captured by a vision device and denoted as follows: Simultaneously, obtain the position coordinates of the first positioning point 30, denoted as... And the position coordinates of the second positioning point 40, denoted as Based on the acquired position coordinates, the distance between the center of the robot's R-axis 10 rotation axis and the first positioning point 30 is calculated. and the angle between the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30 and the line connecting the second positioning point 40 and the first positioning point 30. and distance and included angle Stored in the system for later use.

[0027] Step S103: During the movement of the moving object, the positions of the at least two positioning reference points are identified using a vision device; In actual production, the robot's R-axis 10 drives the auxiliary material suction nozzle 20 to move at high speed. When the auxiliary material suction nozzle 20 moves into the imaging range of the vision device, the vision device triggers imaging. Due to communication and imaging delays, there is a time difference between the actual imaging moment of the vision device and the moment the imaging command is triggered. By this time, the robot has already moved to a new position. The image captured by the vision device at the imaging moment includes a first positioning point 30 and a second positioning point 40. The position coordinates of the first positioning point 30 at the imaging moment are obtained through an image recognition algorithm. and the position coordinates of the second positioning point 40 at the imaging time. .

[0028] Step S104: Determine the position of the reference point of the moving object at the imaging time based on the identified positioning reference point and the geometric positional relationship.

[0029] Based on the position coordinates of the first positioning point 30 at the imaging time The position coordinates of the second positioning point 40 at the imaging time and the distance obtained during the calibration phase. and included angle By using geometric calculations, the position coordinates of the center of the robot's R-axis rotation at the imaging moment can be deduced. Since the geometric positional relationship between the first positioning point 30, the second positioning point 40 and the rotation axis center of the robot's R-axis 10 is fixed, even if there is a time delay, as long as the positions of the first positioning point 30 and the second positioning point 40 can be identified in the image, the true position of the rotation axis center of the robot's R-axis 10 at that moment can be accurately calculated, thereby eliminating the impact of communication delay and imaging delay on positioning accuracy.

[0030] In this embodiment, at least two positioning reference points and the reference point of the moving object have a fixed geometric positional relationship, including a distance relationship and an angular relationship. The distance relationship is the distance between the reference point of the moving object and the first positioning reference point among the at least two positioning reference points, and the angular relationship is the angle between the line connecting the reference point of the moving object and the first positioning reference point and the line connecting the second positioning reference point and the first positioning reference point.

[0031] Specifically, please refer to Figure 2 During the calibration phase, when the robot's R-axis 10 is stationary, the position coordinates of the center of rotation of the robot's R-axis 10 are obtained through a vision device. The position coordinates of the first positioning point 30 and the position coordinates of the second positioning point 40. Based on the above position coordinates, the distance between the center of the robot's R-axis 10 rotation axis and the first positioning point 30 is calculated. distance The calculation is derived using the Pythagorean theorem, and the specific formula is as follows: Simultaneously, the line connecting the center of the rotation axis of the robot's R-axis 10 and the first positioning point 30 is calculated. The line connecting the second positioning point 40 and the first positioning point 30 The angle between included angle It is calculated using the arctangent function; the specific calculation formula is as follows: .distance and included angle These are stored in the system as fixed geometric parameters and used as known quantities in subsequent aerial photography positioning processes.

[0032] Based on the above embodiments, this embodiment further specifies the specific implementation method for obtaining geometric positional relationships in the calibration stage and the detailed steps for calculating geometric parameters in the calibration stage.

[0033] Please see Figure 4 and Figure 5 , Figure 4 The flowchart illustrating the process of obtaining geometric positional relationships during the calibration phase is shown. Figure 5 A detailed flowchart illustrating the calculation of geometric parameters during the calibration phase is shown. Before being put into formal use on the production line, the vision positioning system needs to be calibrated once to establish a fixed geometric relationship between the rotation axis center of the robot's R-axis 10 and the first positioning point 30 and the second positioning point 40. The calibration process is performed with the robot completely stationary to ensure that the acquired position data is not disturbed by any motion factors.

[0034] In this embodiment, the step of obtaining the geometric positional relationship between at least two positioning reference points and the reference point of the moving object further includes: Step S120: When the moving object is stationary, obtain the position coordinates of the reference point of the moving object and the position coordinates of at least two positioning reference points through a vision device; Before calibration begins, the operator adjusts the robot's R-axis 10 to a suitable posture using the robot control system, ensuring that the auxiliary material suction nozzle 20 is within the effective shooting range of the vision device, and guaranteeing that the first positioning point 30, the second positioning point 40, and the rotation axis center of the robot's R-axis 10 can all be clearly captured by the vision device. The robot's R-axis 10 remains completely stationary, and all motion axes are locked to eliminate any possible minor vibrations or displacements. Upon receiving the calibration command, the vision device performs high-precision imaging of the stationary auxiliary material suction nozzle 20, acquiring clear image data.

[0035] In the acquired images, the vision device identifies three key locations using a pre-set image recognition algorithm. First, it identifies the center position of the robot's R-axis 10 rotation axis. The rotation axis center can be identified through the outline features of the robot's R-axis 10, or through pre-set markers on the robot's R-axis 10. The recognition algorithm extracts the position coordinates of the rotation axis center in the vision device's coordinate system, denoted as […]. Then, the position of the first positioning point 30 is identified. The first positioning point 30 is set on the auxiliary material suction nozzle 20 in the form of a positioning pin, with the tip or center of the positioning pin serving as the reference position of the positioning point. The recognition algorithm extracts the position coordinates of the first positioning point 30 in the vision device coordinate system, denoted as... Finally, the position of the second positioning point 40 is identified using the same identification method as the first positioning point 30. The position coordinates of the second positioning point 40 in the visual device coordinate system are extracted and denoted as follows. .

[0036] Since the calibration process is performed while the robot is stationary, there are no positional deviation issues caused by communication or imaging delays. The three acquired position coordinates accurately reflect the true spatial relationship between the robot's R-axis rotation axis center, the first positioning point 30, and the second positioning point 40. The vision device transmits the three identified position coordinates to the computing unit for subsequent geometric parameter calculations.

[0037] Step S121: Calculate the geometric positional relationship based on the position coordinates.

[0038] The computing unit receives the center position coordinates of the robot's R-axis 10 rotation axis. The position coordinates of the first positioning point 30 and the position coordinates of the second positioning point 40 Next, the geometric positional relationship between the three points is calculated. This geometric positional relationship includes two basic elements: distance and angle, which are calculated in detail in subsequent steps S122 and S123, respectively. The results of the geometric positional relationship calculation will be permanently stored as calibration parameters for the system and will be used as known constants in all subsequent aerial photography positioning processes for position back-calculation.

[0039] Among them, combined Figure 5 The step of calculating the geometric positional relationship based on the position coordinates further includes: Step S122: Calculate the distance based on the position coordinates of the reference point of the moving object and the position coordinates of the first positioning reference point; The computing unit uses the position coordinates of the center of the robot's R-axis rotation axis 10. The position coordinates of the first positioning point 30 The straight-line distance L between the center of the rotation axis and the first positioning point 30 is calculated using the distance formula between two points in plane geometry. The calculation uses the Pythagorean theorem, and the specific formula is as follows: The calculation process first calculates the difference between the two points along the horizontal axis. Then calculate the difference between the two points along the vertical axis. Square the two differences separately, add them together, and then take the square root of the sum to obtain the Euclidean distance between the two points. .

[0040] distance This represents the spatial distance in the planar coordinate system between the center of the rotation axis of the robot's R-axis 10 and the first positioning point 30. Since the first positioning point 30 is rigidly fixed to the auxiliary material suction nozzle 20 by a positioning pin, and the auxiliary material suction nozzle 20 is rigidly fixed to the robot's R-axis 10, a rigid connection is formed between the three, therefore the distance... The distance between the robot's R-axis 10 rotation axis center and the first positioning point 30 remains constant throughout the robot's entire movement. Regardless of how the robot moves or rotates, the distance between the center of the R-axis 10 rotation axis and the first positioning point 30 is always equal to the distance measured during calibration. The calculation unit will calculate the distance. The calibration parameters are stored in the system memory for subsequent aerial photography positioning calculations.

[0041] Step S123: Calculate the included angle using the arctangent function based on the position coordinates of the reference point of the moving object, the position coordinates of the first positioning reference point, and the position coordinates of the second positioning reference point.

[0042] The computing unit uses the position coordinates of the center of the robot's R-axis rotation axis 10. The position coordinates of the first positioning point 30 and the position coordinates of the second positioning point 40 Calculate the angle between the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30, and the line connecting the second positioning point 40 and the first positioning point 30. .

[0043] included angle The calculation is divided into three sub-steps. The first sub-step calculates the direction angle of the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30 relative to the horizontal axis of the vision device coordinate system. Taking the first positioning point 30 as the reference point, the offset of the center of the robot's R-axis 10 rotation axis relative to the first positioning point 30 in the vertical axis direction is... The offset in the horizontal direction is The ratio of the two is obtained through the arctangent function. The direction angle is calculated, that is The second sub-step calculates the direction angle of the line connecting the second positioning point 40 and the first positioning point 30 relative to the horizontal axis of the visual device coordinate system. Using the same method, the offset of the second positioning point 40 relative to the first positioning point 30 in the vertical direction is... The offset in the horizontal direction is Through the arctangent function The direction angle is calculated, that is The third sub-step subtracts the two direction angles to obtain the angle between the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30, and the line connecting the second positioning point 40 and the first positioning point 30. The calculation formula is: .

[0044] included angle The angular relationships between three points—the center of the robot's R-axis 10 rotation axis, the first positioning point 30, and the second positioning point 40—are defined. Since all three points are rigidly fixed to the same mechanical structure, the included angles... The relative angles between the three points remain constant throughout the robot's entire movement. Regardless of how the robot moves or rotates, the relative angles between the three points are always equal to the included angles measured during calibration. The calculation unit will calculate the included angle. Stored as calibration parameters in system memory, along with distance. Together, they constitute complete geometric positional relationship calibration data.

[0045] After calibration, the system obtained two key geometric parameters: distance. and included angle In subsequent production processes, when the robot moves at high speed, the vision device only needs to identify the positions of the first positioning point 30 and the second positioning point 40 at the moment of imaging to utilize the pre-calibrated distance. and included angle By using geometric back-calculation, the true position of the robot's R-axis rotation center at the moment of imaging is accurately obtained. The high precision of the calibration process ensures the accuracy of subsequent aerial imaging positioning, while the fixed calibration parameters ensure the stability and reliability of the positioning method.

[0046] Through the above calibration process, the system acquires and stores the distance. and included angle Two geometric parameters. In subsequent production processes, when the robot moves at high speed, it only needs to identify the positions of the first positioning point 30 and the second positioning point 40 at the moment of imaging using a vision device, combined with the pre-calibrated distance. and included angle This allows for accurate calculation of the true position of the center of the robot's R-axis 10 rotation axis at the moment of imaging.

[0047] Based on the above embodiments, this embodiment further defines the detailed calculation process for determining the position of the reference point of the moving object based on the position of the positioning reference point and the geometric positional relationship during the aerial photography stage.

[0048] Please see Figure 6 , Figure 6 A detailed calculation flowchart for determining the reference point position during the aerial imaging phase is shown. In actual production, the robot's R-axis 10 drives the auxiliary material suction nozzle 20 to move at high speed. When the auxiliary material suction nozzle 20 enters the imaging range of the vision device, the system triggers the vision device to take a picture. Due to communication delay and imaging delay between the triggering of the imaging command and the actual completion of imaging by the vision device, the robot continues to move during this delay time, resulting in a difference between the robot's position at the triggering moment and the imaging moment. To obtain the robot's accurate position at the imaging moment, it is necessary to calculate it backwards from the positions of the first positioning point 30 and the second positioning point 40.

[0049] Step S124: Calculate the orientation angle of the second positioning reference point relative to the first positioning reference point based on the positions of the first and second positioning reference points.

[0050] The image captured by the vision device at the imaging moment contains a first positioning point 30 and a second positioning point 40. The position coordinates of the first positioning point 30 at the imaging moment are identified by an image recognition algorithm. and the position coordinates of the second positioning point 40 at the imaging time. Based on the two position coordinates mentioned above, the orientation angle of the line connecting the second positioning point 40 with respect to the first positioning point 30 in the visual device coordinate system is calculated. The orientation angle is calculated using the arctangent function, and the formula is as follows: The orientation angle reflects the tilt angle of the line connecting the second positioning point 40 and the first positioning point 30 relative to the horizontal axis of the coordinate system at the imaging moment. Since both the first positioning point 30 and the second positioning point 40 are fixed on the auxiliary material suction nozzle 20, the change in the orientation angle reflects the change in the rotation angle of the auxiliary material suction nozzle 20 at the imaging moment relative to the calibration moment.

[0051] Step S125: Add the direction angle to the included angle to obtain the direction angle of the reference point of the moving object relative to the first positioning reference point.

[0052] The orientation angle of the second positioning reference point relative to the first positioning reference point calculated in step S124. Angle obtained during the calibration phase Adding these together, we obtain the orientation angle of the robot's R-axis 10 rotation axis center relative to the first positioning point 30. The formula for calculating this orientation angle is: Due to the included angle A fixed angular relationship is defined between the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30, and the line connecting the second positioning point 40 and the first positioning point 30. Once the direction angle of the line connecting the second positioning point 40 and the first positioning point 30 at the imaging moment is known, a fixed angle is added... This allows us to obtain the orientation angle of the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30 at the imaging moment. The orientation angle determines the direction of the center of the robot's R-axis 10 rotation axis relative to the first positioning point 30.

[0053] Step S126: Calculate the position coordinates of the reference point of the moving object at the imaging time using trigonometric functions based on the direction angle of the reference point of the moving object relative to the first positioning reference point and the distance.

[0054] The position coordinates of the first positioning point 30 at the imaging time are known. The orientation angle of the rotation axis center of the robot's R-axis 10 relative to the first positioning point 30 is: And the distance between the center of the rotation axis of the robot's R-axis 10 and the first positioning point 30 is The position coordinates of the center of the robot's rotation axis 10 at the imaging time are calculated using trigonometric functions. The x-coordinate of the center of the robot's R-axis rotation axis 10. The calculation formula is , y-axis The calculation formula is Using the trigonometric functions calculated above, starting from the first positioning point 30, the distance extended along the direction of the direction angle is... This allows for precise positioning of the center of the robot's R-axis rotation at the moment of imaging. Due to the distance... and included angle The fixed geometric parameters are precisely calibrated in a static state, while the positions of the first positioning point 30 and the second positioning point 40 are the actual positions obtained by actual shooting at the imaging moment. Therefore, the calculated center position of the rotation axis of the robot R-axis 10 accurately reflects the real state at the imaging moment and is completely unaffected by communication delay and imaging delay.

[0055] Through the above calculation process, the system can accurately deduce the precise position of the rotation axis center of the robot's R-axis 10 at the imaging moment by simply identifying the positions of the first positioning point 30 and the second positioning point 40 and combining them with the pre-calibrated geometric parameters during the robot's high-speed movement, thus providing a reliable position reference for subsequent auxiliary material attachment operations.

[0056] Based on the above embodiments, this embodiment further illustrates how to use the determined center position of the robot's rotation axis to complete the precise application of auxiliary materials in the application scenario of robot auxiliary material application.

[0057] In this embodiment, the moving object is the robot's feeding nozzle, and the reference point of the moving object is the center of the robot's rotation axis. Please refer to [link / reference]. Figure 7 , Figure 7 The flowchart illustrates the process of identifying and determining the attachment position of the auxiliary material. During production, the robot's R-axis 10 drives the auxiliary material suction nozzle 20 to move, and the auxiliary material suction nozzle 60 adsorbs the auxiliary material 50 through negative pressure adsorption. Since the auxiliary material 50 may have positional deviations during adsorption—that is, the position of the auxiliary material 50 relative to the auxiliary material suction nozzle 20 is not completely fixed—it is necessary to simultaneously identify the actual position of the auxiliary material 50 during the aerial photography process and calculate the precise attachment target position by combining this with the center position of the robot's rotation axis.

[0058] Step S105: After determining the position of the robot's rotation axis center at the imaging moment, the position of the auxiliary material at the imaging moment is identified by the vision device.

[0059] During the robot's movement, the image captured by the vision device at the imaging moment includes not only the first positioning point 30 and the second positioning point 40, but also the auxiliary material 50 adsorbed by the auxiliary material suction nozzle 60. The aforementioned steps have determined the position coordinates of the center of the robot's R-axis 10 rotation axis at the imaging moment. In the same image, the vision device identifies the features of the auxiliary material 50 using an image recognition algorithm, and obtains the position coordinates of the auxiliary material 50 at the time of imaging. The position recognition of the auxiliary material 50 can be based on the shape outline, color characteristics, or preset marker points of the auxiliary material. Since the auxiliary material 50, the first positioning point 30, and the second positioning point 40 are all captured by the vision device at the same time, the position coordinates of the auxiliary material 50 are in the same state as the center position coordinates of the rotation axis of the robot's R-axis 10, thus exhibiting time synchronization.

[0060] Step S106: Determine the target position for the robot to perform the attachment operation based on the relative relationship between the position of the robot's rotation axis center at the imaging moment and the position of the auxiliary material at the imaging moment.

[0061] The coordinates of the center of the robot's rotation axis R10 at the imaging time are given. The position coordinates of auxiliary material 50 at the imaging time are: The relative positional relationship between the two is calculated, specifically the offset of the auxiliary material 50 relative to the rotation axis center of the robot's R-axis 10. A target attachment position for the auxiliary material is pre-set on the bracket product, and this target attachment position has known coordinates in the vision device's coordinate system. To accurately attach the auxiliary material 50 to the target position on the bracket product, the robot needs to be controlled to move to a specific position, causing the auxiliary material 50 to move from its current position to the target attachment position. Based on the offset relationship of the auxiliary material 50 relative to the rotation axis center of the robot's R-axis 10, and the coordinates of the target attachment position, the target position coordinates that the rotation axis center of the robot's R-axis 10 should reach are calculated in reverse. The robot control system plans the robot's motion trajectory based on the calculated target position coordinates and controls the robot's R-axis 10 to move to the target position, thereby achieving precise attachment of the auxiliary material 50.

[0062] Through the above steps, this embodiment achieves complete closed-loop control from aerial positioning to precise attachment. Since the center position of the robot's R-axis 10 is accurately calculated using the first positioning point 30 and the second positioning point 40, it is unaffected by communication and imaging delays. Meanwhile, the position of the auxiliary material 50 is the actual position identified at the same imaging moment. The relative relationship between the two accurately reflects the actual state at the imaging moment. The attachment target position calculated based on the accurate relative positional relationship ensures that the auxiliary material 50 is precisely attached to the predetermined position on the bracket product, meeting the high-precision attachment process requirements. Furthermore, since the entire process is completed during robot movement, there is no need for robot deceleration and stopping, significantly improving production efficiency.

[0063] In this embodiment, there are at least two positioning reference points. The moving object is the robot's feeding nozzle, and the reference point of the moving object is the center of the robot's rotation axis.

[0064] Please reconsider. Figure 1 and Figure 2 Two positioning reference points are set on the robot's auxiliary material suction nozzle 20, namely the first positioning point 30 and the second positioning point 40. The two positioning reference points are rigidly fixed to the auxiliary material suction nozzle 20 by positioning pins. The positioning pins are set perpendicular to the surface of the auxiliary material suction nozzle 20, and the tip of the positioning pin serves as the position reference point. The installation positions of the first positioning point 30 and the second positioning point 40 on the auxiliary material suction nozzle 20 are carefully designed to ensure that the two positioning points form a stable triangular geometric relationship with the center of the robot's R-axis 10 rotation axis.

[0065] The spacing between the first positioning point 30 and the second positioning point 40 needs to meet the accuracy requirements of visual recognition. The distance between the two positioning points cannot be too small; otherwise, the pixel distance between them will be too small when the vision device captures the image, affecting recognition accuracy and angle calculation accuracy. The distance also cannot be too large; otherwise, the two positioning points may not appear simultaneously within the field of view of the vision device in certain robot postures. In this embodiment, the spacing between the first positioning point 30 and the second positioning point 40 is set to a suitable proportion of the feature size of the auxiliary material suction nozzle 20, ensuring both sufficient angular resolution and that both positioning points can be captured simultaneously by the vision device in all working postures of the robot.

[0066] The placement of the first positioning point 30 and the second positioning point 40 also needs to consider their geometric relationship with the center of the rotation axis of the robot's R-axis 10. The three points cannot be collinear; otherwise, an effective angular relationship cannot be formed, leading to failure of the position back-calculation or a decrease in accuracy. In this embodiment, the first positioning point 30, the second positioning point 40, and the center of the rotation axis of the robot's R-axis 10 form a non-degenerate triangle, and the lines connecting the three points form a clear angular relationship, providing a stable mathematical foundation for subsequent geometric back-calculation.

[0067] The technique of using two positioning reference points for position inversion is based on the positioning principle in plane geometry. In a plane coordinate system, if the distance and angle relationships between a point to be determined and two reference points are known, the position of the point to be determined can be uniquely determined. In this embodiment, the center of the rotation axis of the robot's R-axis 10 corresponds to the point to be determined, and the first positioning point 30 and the second positioning point 40 correspond to the reference points. Through the calibration phase, the distance L between the center of the rotation axis of the robot's R-axis 10 and the first positioning point 30, and the angle θ between the line connecting the center of the rotation axis of the robot's R-axis 10 and the first positioning point 30 and the line connecting the second positioning point 40 and the first positioning point 30 are obtained, establishing a complete geometric constraint relationship.

[0068] During the aerial imaging phase, after the vision device identifies the positions of the first positioning point 30 and the second positioning point 40 at the imaging moment, it can first determine the direction of the line connecting the second positioning point 40 and the first positioning point 30. Then, it uses the included angle θ to calculate the direction of the line connecting the center of the robot's R-axis 10 rotation axis and the first positioning point 30. Finally, it uses the distance... By extending a fixed distance along the calculated direction from the first positioning point 30, the center position of the robot's R-axis 10 rotation axis can be accurately located. The two positioning reference points provide sufficient constraints, neither redundant nor missing, constituting the simplest and most efficient positioning scheme.

[0069] The configuration of two positioning reference points offers significant advantages over single-point or multi-point configurations. A single positioning reference point cannot provide angle information, only distance relationships, and is insufficient to uniquely determine the position of the point to be determined. While three or more positioning reference points can provide redundant constraints, which is beneficial for improving positioning accuracy or error verification, they increase hardware costs and computational complexity. Furthermore, arranging more positioning pins within the limited space of the auxiliary material suction nozzle 20 presents practical difficulties. The configuration of two positioning reference points achieves an optimal balance between hardware cost, computational complexity, and installation convenience while meeting positioning accuracy requirements, making it the most suitable configuration for robotic auxiliary material attachment applications.

[0070] The visual positioning method for moving objects provided in this application establishes a fixed geometric positional relationship between the positioning reference points and the reference point of the moving object by setting at least two positioning reference points on the moving object. During the movement of the moving object, the position of the positioning reference points is identified by a vision device. Based on the identified position of the positioning reference points and the geometric positional relationship, the position of the reference point of the moving object at the imaging moment is determined. This solves the positional error problem caused by communication delay and imaging delay in the prior art during the aerial shooting process. By establishing a rigidly connected positioning reference point as a geometric reference on the moving object, the accurate position of the reference point of the moving object at the imaging moment can be deduced from the actual position of the positioning reference point, regardless of the time delay between the moment the vision device triggers the shooting command and the actual imaging moment. This fundamentally eliminates the impact of communication delay and imaging delay on positioning accuracy, greatly improving the accuracy of dynamic visual positioning. Furthermore, the positioning method allows the moving object to complete visual positioning during high-speed movement without deceleration or stopping. Compared with the traditional fixed-position shooting method, it eliminates the time consumed by the moving object's deceleration, stopping, shooting, and re-acceleration, significantly shortening the single positioning cycle and improving production efficiency.

[0071] This embodiment provides an example of a visual positioning system, which is used to perform the visual positioning method for moving objects described in any of the foregoing embodiments.

[0072] Please reconsider. Figure 1 , Figure 1 A schematic diagram of the overall structure of the visual positioning system is shown. The visual positioning system consists of three main components: a moving object, a visual device, and a computing unit. These components are connected by electrical connections and mechanical installation to form a complete positioning system.

[0073] In this embodiment, the moving object is specifically the robot material suction nozzle 20, which is fixedly mounted on the robot's R-axis 10. The robot's R-axis 10 is the rotation axis of a six-axis industrial robot, capable of driving the material suction nozzle 20 to perform complex movements in three-dimensional space, including translational and rotational movements. The bottom of the material suction nozzle 20 is equipped with an adsorption structure, which is connected to a vacuum pump via a negative pressure system, generating an adsorption force on the material 50 to achieve the picking and handling of the material 50. At least two positioning reference points are provided on the material suction nozzle 20; in this embodiment, a first positioning point 30 and a second positioning point 40 are provided. The first positioning point 30 and the second positioning point 40 are rigidly fixed to the outer surface of the material suction nozzle 20 in the form of positioning pins. The positioning pins are made of wear-resistant metal material to ensure that their position remains stable during long-term use. A fixed geometric positional relationship is formed between the first positioning point 30, the second positioning point 40 and the rotation axis center of the robot's R-axis 10. The geometric positional relationship includes distance relationship and angle relationship. After system calibration, this geometric positional relationship is stored as a known parameter in the computing unit.

[0074] The vision device is positioned within the robot's working area, below or to the side of the robot's motion trajectory. The optical axis of the vision device is perpendicular to or tilted to the robot's main motion plane, ensuring clear imaging of the material suction nozzle 20 during its movement. The vision device includes an industrial camera, lens, and light source system. The industrial camera uses a high-speed CMOS image sensor, enabling it to capture clear images during high-speed robot movement and avoid motion blur. The lens's focal length and field of view are selected according to the robot's working range, ensuring clear imaging throughout the entire working area. The light source system includes LED ring or strip light sources, providing uniform and stable illumination, eliminating the impact of ambient light variations on image quality, and improving the accuracy of identifying positioning reference points and materials.

[0075] The vision device connects to the computing unit via a data cable, which uses an industrial-grade Ethernet cable or a dedicated camera data cable, supporting high-speed image data transmission. Upon receiving a trigger signal, the vision device takes an image and transmits the captured image data to the computing unit for processing in real time. The trigger signal can be automatically generated by the computing unit based on the robot's motion state, or it can be issued by the robot control system when the robot moves to a specific position. The vision device's exposure time, gain, and other parameters can be remotely adjusted via the computing unit to adapt to different working environments and shooting requirements.

[0076] The computing unit is the core control component of the vision positioning system, responsible for image processing, position calculation, and communication with the robot control system. The computing unit includes an industrial computer or embedded processor, equipped with sufficient processing power and storage space to process high-resolution images in real time and perform complex geometric calculations. Internally, the computing unit runs image recognition algorithm software and position calculation software. The image recognition algorithm software identifies the positions of the first positioning point 30, the second positioning point 40, and the auxiliary material 50 from the images transmitted by the vision device. The position calculation software calculates the position of the robot's reference point at the moment of imaging based on the identified positioning reference point positions and pre-calibrated geometric positional relationships.

[0077] The computing unit has multiple functional modules, including a calibration module, a recognition module, and a calculation module. The calibration module executes the calibration process during system initialization, acquiring the geometric positional relationships between at least two positioning reference points and the reference point of the moving object. The calibration module controls the robot to move to the calibration position and remain stationary, triggering the vision device to capture an image. It identifies the position coordinates of the robot's rotation axis center, the first positioning point 30, and the second positioning point 40 from the captured image, calculates the distance and angle relationships based on the position coordinates, and stores the calculation results in the system parameter file. The recognition module processes the images transmitted by the vision device in real time during production, identifying the positions of the first positioning point 30 and the second positioning point 40 from the images. The recognition module uses image processing algorithms such as template matching, edge detection, or feature point detection to quickly and accurately locate the pixel coordinates of the positioning reference points in the image and converts the pixel coordinates into physical coordinates in the vision device's coordinate system. The calculation module calculates the position of the reference point of the moving object at the time of imaging using a geometric back-calculation algorithm, based on the positioning reference point positions output by the recognition module and the geometric positional relationships stored in the calibration module. The calculation module calls the trigonometric function library to perform angle calculations and coordinate transformations, and outputs the position coordinates of the robot's rotation axis center in the vision device's coordinate system.

[0078] The computing unit connects to the robot control system via a communication interface. This interface utilizes industrial Ethernet, RS232 serial port, or a dedicated bus protocol to enable bidirectional data exchange between the computing unit and the robot control system. The computing unit sends the calculated robot rotation axis center position and auxiliary material position information to the robot control system. Based on the received position information, the robot control system calculates the target attachment position, plans the robot's motion trajectory, and controls the robot to precisely complete the attachment action. The robot control system can also send trigger signals, parameter setting instructions, and other control commands to the computing unit to achieve coordinated control of the vision positioning system.

[0079] The vision positioning system also includes a display unit and a human-machine interface (HMI) unit. The display unit is an LCD or industrial touchscreen, used to display images captured by the vision device, recognition results, calculation results, and system operating status in real time, facilitating operator monitoring of the system's operation and troubleshooting. The HMI unit includes a keyboard, mouse, or touchscreen, used by operators to input parameters, perform calibration, start or stop the system, and adjust algorithm parameters.

[0080] The workflow of the entire vision positioning system is as follows: During system initialization, the operator initiates the calibration program through the human-machine interface. The calibration module of the computing unit controls the robot to move to the calibration position and triggers the vision device to capture an image. It identifies key point positions from the image and calculates the geometric positional relationships, completing the calibration process. During production, the robot control system controls the robot's R-axis 10 to move the auxiliary material suction nozzle 20. When the auxiliary material suction nozzle 20 enters the field of view of the vision device, the computing unit triggers the vision device to take a flying image. The vision device captures the image at the imaging moment and transmits it to the computing unit. The recognition module of the computing unit identifies the positions of the first positioning point 30 and the second positioning point 40 from the image. Based on the recognition results and the pre-calibrated geometric positional relationships, the computing module calculates the position of the rotation axis center of the robot's R-axis 10 at the imaging moment, and simultaneously identifies the position of the auxiliary material 50. This position information is then sent to the robot control system, which performs precise attachment based on the position information.

[0081] With the above system setup, the visual positioning system can achieve precise dynamic positioning during high-speed robot movement, eliminating the impact of communication and imaging delays on positioning accuracy and meeting the requirements of high-precision and high-efficiency automated production. The system has a compact structure, with clearly defined functions for each component, working together to achieve complete visual positioning functionality.

[0082] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for visually locating a moving object, characterized in that, include: At least two positioning reference points are set on the moving object, and the positioning reference points have a fixed geometric positional relationship with the reference point of the moving object; Obtain the geometric positional relationship between the at least two positioning reference points and the reference point of the moving object; During the movement of the moving object, the positions of the at least two positioning reference points are identified using a vision device; Based on the identified positioning reference point and the geometric positional relationship, the position of the reference point of the moving object at the imaging moment is determined.

2. The visual positioning method according to claim 1, characterized in that, The at least two positioning reference points have a fixed geometric positional relationship with the reference point of the moving object, including a distance relationship and an angular relationship.

3. The visual positioning method according to claim 2, characterized in that, The distance relationship is the distance between the reference point of the moving object and the first positioning reference point among the at least two positioning reference points, and the angle relationship is the angle between the line connecting the reference point of the moving object and the first positioning reference point and the line connecting the second positioning reference point and the first positioning reference point.

4. The visual positioning method according to claim 1, characterized in that, The step of obtaining the geometric positional relationship between the at least two positioning reference points and the reference point of the moving object further includes: When the moving object is stationary, the position coordinates of the reference point of the moving object and the position coordinates of the at least two positioning reference points are obtained through a vision device. The geometric positional relationship is calculated based on the position coordinates.

5. The visual positioning method according to claim 4, characterized in that, The step of calculating the geometric positional relationship based on the position coordinates further includes: The distance is calculated based on the position coordinates of the reference point of the moving object and the position coordinates of the first positioning reference point; The included angle is calculated using the arctangent function based on the position coordinates of the reference point of the moving object, the position coordinates of the first positioning reference point, and the position coordinates of the second positioning reference point.

6. The visual positioning method according to claim 3, characterized in that, The step of determining the position of the reference point of the moving object at the imaging time based on the identified positioning reference point position and the geometric positional relationship further includes: Calculate the orientation angle of the second positioning reference point relative to the first positioning reference point based on the positions of the first and second positioning reference points identified; Adding the direction angle to the included angle yields the direction angle of the reference point of the moving object relative to the first positioning reference point; The position coordinates of the reference point of the moving object at the imaging moment are calculated using trigonometric functions based on the direction angle of the reference point of the moving object relative to the first positioning reference point and the distance.

7. The visual positioning method according to claim 1, characterized in that, The number of the at least two positioning reference points is two.

8. The visual positioning method according to claim 1, characterized in that, The moving object is a robot's feeding nozzle, and the reference point of the moving object is the center of the robot's rotation axis.

9. The visual positioning method according to claim 8, characterized in that, The method further includes: After determining the position of the robot's rotation axis center at the imaging moment, the position of the auxiliary material at the imaging moment is identified by a vision device; The target position for the robot to perform the attachment operation is determined based on the relative relationship between the position of the robot's rotation axis center at the imaging moment and the position of the auxiliary material at the imaging moment.

10. A visual positioning system, characterized in that, The visual positioning system is used to perform the visual positioning method for a moving object as described in any one of claims 1 to 9.

Images