Dynamic flying photography high-precision positioning method and device based on Mark point marking
By using the dynamic fly shot high-precision positioning method of Mark point mark in the fly shot technology, the Mark points in the image are extracted and calibrated, and the accuracy deviation and debugging problems in the fly shot technology are solved, high-precision positioning is achieved, and the flying is expanded. Application scope of shooting technology.
Patent Information
- Application Number
- CN202311447318.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-02
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-11-02
AI Technical Summary
Flying shooting technology has accuracy deviations, long debugging time and control problems in applications with high accuracy requirements, which limits its application in the field of crawling with low accuracy requirements.
Using a dynamic fly shot high-precision positioning method based on Mark point mark, the Mark points are extracted in static and real-time images, calculate and calibrate image relationships, obtain the calibrated image, and use deviation values to calibrate the coordinate system of the detected object to achieve high accuracy. position.
It improves the positioning accuracy of Feipao technology, realizes the secondary high-precision positioning of the product, solves the problems of difficult control of trigger points and long debugging time, and makes Feipao technology no longer limited to crawling applications.
Smart Images

Figure CN117557637B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of visual detection, and in particular relates to a high-precision positioning method and device for dynamic flying shooting based on Mark point marking. Background Art
[0002] Flying camera technology is to take pictures of objects while they are in motion. The objects do not need to stop, and the position of the objects is calculated through visual algorithms, which greatly improves production efficiency. It has been widely used in various fields of dynamic grasping. However, although flying camera shortens the CCD image acquisition cycle, it will cause accuracy deviation due to factors such as delay in transmission between camera trigger and control system signal. Therefore, flying camera is mostly used in grasping fields where accuracy requirements are not high.
[0003] In other related patents on flying shot counting, most of them calculate the real-time position of the object on the conveyor belt and the real-time grasping posture of the object based on the initial grasping posture of the object, the starting position, and the movement speed of the conveyor belt, but these methods all have a common problem, which is that they are difficult to debug and are greatly affected by the robot speed, sensor sensing accuracy, belt speed, etc. Therefore, in this patent, a high-precision positioning method based on dynamic flying shot is proposed, which not only solves the problems of difficult control of communication control signals, long debugging time, and low positioning accuracy that occur in the application process of flying shot technology, but also realizes secondary high-precision positioning of the product while realizing grasping, so that flying shot technology is no longer limited to grasping applications. Summary of the invention
[0004] In view of the defects of the above-mentioned prior art, the present invention proposes a dynamic flying camera high-precision positioning method and device based on Mark point marking, which can realize secondary high-precision positioning of the product while realizing flying camera grasping, that is, the product can be positioned to an application scenario with an accuracy higher than 0.25mm tolerance while grasping the product.
[0005] Specifically, to achieve the above purpose, the present invention provides a high-precision positioning method for dynamic flying based on Mark point marking, comprising:
[0006] S1: Get the image of the Mark point in a static state, recorded as image 1;
[0007] S2: The camera collects images in real time. When an image that meets the requirements is collected, it is recorded as image 2;
[0008] S3: extracting the area where the Mark point is located in the image 1 and the image 2, and calculating the relationship between the real-time image and the static image;
[0009] S4: calibrating the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3;
[0010] S5: Obtain the position of any Mark point in the image 3 and compare it with the image 1 to obtain the deviation value of the Mark point for calibrating the conversion matrix between the coordinate system of the detection object and the image coordinate system;
[0011] S6: Obtaining the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix.
[0012] The dynamic flying camera high-precision positioning method based on Mark point marking described in the present invention can effectively solve the problems of difficult control, difficult debugging and poor accuracy of the camera's shooting trigger point during the application of flying camera technology, thereby improving the positioning accuracy of flying camera and making flying camera technology not only limited to grasping applications.
[0013] The S2 also includes: the camera collects images in real time to show that there are Mark points and complete detection targets in the images.
[0014] The S3 further includes:
[0015] Use Harris corner detection to extract mark points;
[0016] Acquire the target area by using the nearest neighbor algorithm according to the same or similar feature points;
[0017] The image is corrected by using a non-exposure line correction method, the image smear is corrected, and an image correction transformation relationship is obtained. The image transformation relationship includes translation, rotation and scaling.
[0018] The S4 further comprises:
[0019] S41: obtaining a transformation matrix based on the relationship between the real-time image and the static image obtained by the previous calculation;
[0020] S42: Perform perspective transformation on image 2 using the transformation matrix; map pixel coordinates in image 2 to calibrated positions through perspective transformation so that the marker points in image 2 are aligned with the corresponding marker points in the static image.
[0021] S43: After the perspective transformation is completed, a calibrated image 3 is obtained.
[0022] The S5 further includes:
[0023] S51: Select any marking point in image 3 and obtain its pixel coordinates;
[0024] S52: Find the marking point corresponding to the marking point selected in image 3 in image 1, and obtain its pixel coordinates;
[0025] S53: Calculate the deviation value of the Mark point between the pixel coordinates of the selected mark point in image 3 and the pixel coordinates of the corresponding mark point in image 1.
[0026] S54: Calibrate the coordinate system according to the deviation value, and align the coordinate system of the detected object with the image coordinate system using a linear transformation or an affine transformation method; and convert the position of the detected object in the image into a position in the world coordinate system according to the calibrated conversion relationship.
[0027] Based on the same inventive concept, the present invention also provides a high-precision positioning device for dynamic flying shooting based on Mark point marking, and the device also includes:
[0028] Exposure unit, which performs global exposure of the camera real-time image;
[0029] The first acquisition unit acquires an image of the Mark point in a static state, which is recorded as image 1;
[0030] The second acquisition unit acquires images in real time, and when an image that meets the requirements is acquired, it is recorded as image 2;
[0031] The first calculation unit extracts the area where the Mark point is located in the image 1 and the image 2, and calculates the relationship between the real-time image and the static image
[0032] A second calculation unit calibrates the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3;
[0033] A calibration unit, which obtains the position of any Mark point in the image 3 and compares it with the image 1 to obtain a deviation value of the Mark point for calibrating a conversion matrix between a coordinate system of the detection object and an image coordinate system;
[0034] The third calculation unit obtains the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix.
[0035] Preferably, the device further comprises: an analysis unit;
[0036] The analysis unit has a built-in neural network model, which performs position difference analysis on the Mark points in the captured image and the Mark points in the target image, and outputs the position deviation value of the Mark points;
[0037] Or the product's location coordinates are input into the neural network model, and the product's location coordinates are compared with the location coordinate information in the target image to reversely verify the calibration accuracy.
[0038] Preferably, the device further comprises: a control unit;
[0039] The control unit is used to control the manipulator to perform a positioning operation according to the accurate position of the target position of the detection object in the world coordinate system.
[0040] Based on the same inventive concept, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the dynamic flying shooting high-precision positioning method based on Mark point marking as described above is implemented.
[0041] Based on the same inventive concept, the present invention also provides a visual inspection device, which adopts the positioning device as described above to implement a dynamic flying shooting high-precision positioning method based on Mark point marking.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] The present invention provides a dynamic flying camera high-precision positioning method and device based on Mark point marking, which adopts the method of real-time image acquisition, that is, no trigger point is required to trigger the photo shooting, thus avoiding the problem of difficult trigger point controller and long debugging time. When the Mark point and the product appear in the field of view at the same time, the acquisition is stopped, the Mark point in the static image and the real-time image is extracted, and the relationship between the real-time image and the static image is calculated. Then, according to the relationship between the real-time image and the static image, the real-time image is calibrated to obtain the calibrated image, and the Mark point position deviation is calculated according to the calibrated image and the static image. The compensated image information is used for global calibration, and the product position information is calculated to make up for the accuracy difference caused by the change of the photo position in the traditional flying camera capture and positioning, and improve the positioning accuracy of the flying camera, so that the flying camera technology is not limited to the application of capture. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 The flowchart of a high-precision positioning method of dynamic flying shot based on Mark point marking in one embodiment.
[0045] Figure 2 Schematic diagram of a high-precision positioning device for dynamic flying shooting based on Mark point marking in one embodiment. DETAILED DESCRIPTION
[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution will be clearly and completely described in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0047] Embodiment 1:
[0048] As attached Figure 1As shown, the present invention provides a dynamic flying shooting high-precision positioning method based on Mark point marking, comprising:
[0049] S1: Get the image of the Mark point in a static state, recorded as image 1;
[0050] The image of the mark point can be obtained in a static state through a camera or other image acquisition device. The image of the mark point can be obtained in a static state by:
[0051] (1) Place a camera or image acquisition device at the location where the image of the marked point needs to be obtained.
[0052] (2) Set camera parameters: Set the camera’s exposure time, white balance, focal length and other parameters according to specific needs to ensure clear and accurate images.
[0053] (3) Determine the marking point area: According to application requirements, determine the area where the marking point is located in a static state.
[0054] (4) Triggering image acquisition: Depending on the actual situation, you can trigger image acquisition by manually operating the camera button or using a remote control, or use a timer to set automatic acquisition.
[0055] (5) Obtaining the image of the marker point: The camera or image acquisition device will obtain the image of the marker point in a static state according to the set parameters. Ensure that the camera or device is stable to avoid image blur or shaking.
[0056] (6) Image storage: The acquired marker point images are saved to a medium such as the camera's memory card or the computer's hard disk for subsequent processing and analysis.
[0057] Images of the marker points can be acquired in a static state. These images can be used for subsequent image processing, matching, calibration and other operations to achieve specific application requirements.
[0058] S2: The camera collects images in real time. When an image that meets the requirements is collected, it is recorded as image 2;
[0059] Configure the camera: Set the camera to real-time acquisition mode and set corresponding parameters according to specific needs, such as resolution, frame rate, exposure time, etc.
[0060] Start real-time acquisition: Start the camera to start real-time image acquisition to ensure that the camera can continuously acquire images.
[0061] Image quality judgment: By setting some image quality indicators or using image processing algorithms, the real-time collected images are detected and judged to determine whether they meet the requirements.
[0062] Determine whether the requirements are met: According to the set image quality index or algorithm judgment condition, determine the image that meets the requirements, that is, image 2.
[0063] Mark image 2: When an image that meets the requirements is detected, it is marked as image 2 for subsequent processing.
[0064] Preferably, you can also set appropriate image quality indicators or use specific image processing algorithms to achieve real-time acquisition and image screening functions according to your own requirements.
[0065] The S2 also includes: the camera collects images in real time to show that there are Mark points and complete detection targets in the images.
[0066] S3: extracting the area where the Mark point is located in the image 1 and the image 2, and calculating the relationship between the real-time image and the static image;
[0067] The S3 further includes:
[0068] Use Harris corner detection to extract mark points;
[0069] Acquire the target area by using the nearest neighbor algorithm according to the same or similar feature points;
[0070] The image is corrected by using a non-exposure line correction method, the image smear is corrected, and an image correction transformation relationship is obtained. The image transformation relationship includes translation, rotation and scaling.
[0071] In order to further explain the relationship between the calculated real-time image and the static image, preferably, the calculation is as follows:
[0072] 1. Marker point extraction: Use image processing techniques, such as edge detection, feature extraction, etc., to extract the location information of the marker points from image 1 and image 2. The marker points can be markers of a specific shape, color or texture placed in the image in advance.
[0073] 2. Matching marker points: Match the marker points extracted from image 1 with the marker points extracted from image 2. Feature descriptors such as SIFT, SURF, etc. can be used for feature matching of marker points.
[0074] 3. Calculate the relationship: Based on the geometric relationship between the matched marker point pairs, calculate the relationship between the real-time image and the static image, such as translation, rotation, scaling and other transformation parameters.
[0075] 4. Optimization and calibration: Based on the initial relationship obtained, an optimization algorithm (such as the least squares method) can be used to optimize the error of the marker point matching to further improve the relationship between the real-time image and the static image.
[0076] Through the above steps, the markers in image 1 and image 2 can be extracted, and the relationship between the real-time image and the static image can be calculated. These relationships can be used for subsequent applications such as image calibration, target tracking, and image registration.
[0077] S4: calibrating the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3;
[0078] Wherein, the S4 further includes:
[0079] S41: obtaining a transformation matrix based on the relationship between the real-time image and the static image obtained by the previous calculation;
[0080] S42: Perform perspective transformation on image 2 using the transformation matrix; map pixel coordinates in image 2 to calibrated positions through perspective transformation so that the marker points in image 2 are aligned with the corresponding marker points in the static image.
[0081] S43: After the perspective transformation is completed, a calibrated image 3 is obtained.
[0082] S5: Obtain the position of any Mark point in the image 3 and compare it with the image 1 to obtain the deviation value of the Mark point for calibrating the conversion matrix between the coordinate system of the detection object and the image coordinate system;
[0083] Wherein, the S5 further includes:
[0084] S51: Select any marking point in image 3 and obtain its pixel coordinates;
[0085] S52: Find the marking point corresponding to the marking point selected in image 3 in image 1, and obtain its pixel coordinates;
[0086] S53: Calculate the deviation value of the Mark point between the pixel coordinates of the selected mark point in image 3 and the pixel coordinates of the corresponding mark point in image 1.
[0087] S54: Calibrate the coordinate system according to the deviation value, and align the coordinate system of the detected object with the image coordinate system using a linear transformation or an affine transformation method; and convert the position of the detected object in the image into a position in the world coordinate system according to the calibrated conversion relationship.
[0088] By comparing the position of any Mark point in Image 3 with Image 1, the deviation value of the Mark point can be obtained, which is further used to calibrate the conversion matrix between the coordinate system of the detected object and the image coordinate system. Through this conversion matrix, the target position of the detected object can be accurately converted to the world coordinate system.
[0089] S6: Obtaining the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix.
[0090] The present invention can achieve high-precision target positioning by using Mark point marking and image processing technology. According to the extraction and calculation of Mark points in image 1 and image 2, and the generation of calibrated image 3, the relationship between the real-time image and the static image can be obtained, and then the accurate position of the target can be correctly calculated. The camera is used to collect images in real time, and the images are screened according to the image quality requirements. The Mark points are extracted and calibrated in the images that meet the requirements. In this way, the timeliness and accuracy of the positioning process can be guaranteed.
[0091] The dynamic flying shooting high-precision positioning method based on Mark point marking of the present invention has the technical effects of high precision, real-time and accuracy, and can be applied to fields requiring high-precision positioning, such as robot navigation, industrial automation, etc.
[0092] The present invention can also effectively solve the problems of difficult control, difficult debugging and poor accuracy of the camera's shooting trigger point during the application of the flying camera technology, improve the positioning accuracy of the flying camera, and make the flying camera technology not only limited to the application of grasping.
[0093] Embodiment 2:
[0094] The present invention also provides a high-precision positioning device for dynamic flying shooting based on Mark point marking, and the device also includes:
[0095] Exposure unit, which performs global exposure of the camera real-time image;
[0096] The first acquisition unit acquires an image of the Mark point in a static state, which is recorded as image 1;
[0097] The second acquisition unit acquires images in real time, and when an image that meets the requirements is acquired, it is recorded as image 2;
[0098] The first calculation unit extracts the area where the Mark point is located in the image 1 and the image 2, and calculates the relationship between the real-time image and the static image
[0099] A second calculation unit calibrates the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3;
[0100] A calibration unit, which obtains the position of any Mark point in the image 3 and compares it with the image 1 to obtain a deviation value of the Mark point for calibrating a conversion matrix between a coordinate system of the detection object and an image coordinate system;
[0101] The third calculation unit obtains the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix.
[0102] Preferably, the device further comprises: an analysis unit;
[0103] The analysis unit has a built-in neural network model, which performs position difference analysis on the Mark points in the captured image and the Mark points in the target image, and outputs the position deviation value of the Mark points;
[0104] Or the product's location coordinates are input into the neural network model, and the product's location coordinates are compared with the location coordinate information in the target image to reversely verify the calibration accuracy.
[0105] Preferably, the device further comprises: a control unit;
[0106] The control unit is used to control the manipulator to perform a positioning operation according to the accurate position of the target position of the detection object in the world coordinate system.
[0107] Each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.
[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0109] Embodiment three:
[0110] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the dynamic flying shooting high-precision positioning method based on Mark point marking as described above is implemented.
[0111] The computer device can store the required image data and algorithm models through the memory, and the processor is responsible for executing the computer program to realize the specific steps of camera image acquisition, marker point extraction, relationship calculation, image calibration, etc. Through this device, objects in dynamic scenes can be efficiently positioned with high precision.
[0112] Embodiment 4:
[0113] The present invention also provides a visual inspection device, which adopts the positioning device as described above to implement a dynamic flying shooting high-precision positioning method based on Mark point marking.
[0114] The visual inspection equipment includes a positioning device and an image acquisition device and an image processing module that cooperate with it. The positioning device is responsible for real-time acquisition of static images and obtaining the position information of the Mark point, while acquiring the real-time image of the camera for positioning calculation. The image acquisition device is responsible for acquiring the real-time image of the camera and transmitting it to the image processing module for processing. The image processing module contains relevant computer programs, which are executed by the processor to realize the steps of extracting the Mark point, calculating the relationship between the real-time image and the static image, and image calibration.
[0115] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods when implementing them in actual applications. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0116] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only first memory (Read-Only Memory, ROM), a random access first memory (Random Access Memory, RAM), a disk or an optical disk, and other media that can store program codes.
[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A high-precision positioning method for dynamic flying based on Mark point marking, characterized in that: The method comprises: S1: Get the image of the Mark point in a static state, recorded as image 1; S2: The camera collects images in real time. When an image that meets the requirements is collected, it is recorded as image 2; S3: extracting the area where the Mark point is located in the image 1 and the image 2, and calculating the relationship between the real-time image and the static image; S4: calibrating the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3; S5: Obtain the position of any Mark point in the image 3 and compare it with the image 1 to obtain the deviation value of the Mark point for calibrating the conversion matrix between the coordinate system of the detection object and the image coordinate system; S6: Obtaining the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix; The S3 further includes: Use Harris corner detection to extract mark points; Acquire the target area using the nearest neighbor algorithm based on the same or similar feature points; Correcting the image by using a non-exposure line correction method, correcting image smear, and obtaining an image correction transformation relationship, wherein the image transformation relationship includes translation, rotation, and scaling; The S4 further comprises: S41: obtaining a transformation matrix based on the relationship between the real-time image and the static image obtained by the previous calculation; S42: Perform perspective transformation on image 2 using the transformation matrix; map pixel coordinates in image 2 to calibrated positions through perspective transformation so that the marker points in image 2 are aligned with the corresponding marker points in the static image. S43: After the perspective transformation is completed, a calibrated image 3 is obtained.
2. The Mark point marking-based dynamic flying high-precision positioning method according to claim 1 is characterized in that: The S2 also includes: the camera collects images in real time to show that there are Mark points and complete detection targets in the images.
3. The Mark point marking-based dynamic flying high-precision positioning method according to claim 2 is characterized in that: The S5 further includes: S51: Select any marking point in image 3 and obtain its pixel coordinates; S52: Find the marking point corresponding to the marking point selected in image 3 in image 1, and obtain its pixel coordinates; S53: Calculate the deviation value of the Mark point between the pixel coordinates of the selected mark point in image 3 and the pixel coordinates of the corresponding mark point in image 1. S54: Calibrate the coordinate system according to the deviation value, and align the coordinate system of the detected object with the image coordinate system using a linear transformation or an affine transformation method; and convert the position of the detected object in the image into a position in the world coordinate system according to the calibrated conversion relationship.
4. The positioning device of the Mark point-based dynamic flying high-precision positioning method according to any one of claims 1 to 3, characterized in that: The device also includes: Exposure unit, which performs global exposure of the camera real-time image; The first acquisition unit acquires an image of the Mark point in a static state, which is recorded as image 1; The second acquisition unit acquires images in real time, and when an image that meets the requirements is acquired, it is recorded as image 2; The first calculation unit extracts the area where the Mark point is located in the image 1 and the image 2, and calculates the relationship between the real-time image and the static image; A second calculation unit calibrates the image 2 according to the relationship between the real-time image and the static image to obtain a calibrated image 3; A calibration unit, which obtains the position of any Mark point in the image 3 and compares it with the image 1 to obtain a deviation value of the Mark point for calibrating a conversion matrix between a coordinate system of the detection object and an image coordinate system; The third calculation unit obtains the accurate position of the target position of the detection object in the world coordinate system according to the transformation matrix.
5. The positioning device according to claim 4, characterized in that: Also includes: An analysis unit; the analysis unit has a built-in neural network model, through which the position difference analysis of the Mark point in the captured image and the Mark point in the target image is performed, and the Mark point position deviation value is output; Or the product's location coordinates are input into the neural network model, and the product's location coordinates are compared with the location coordinate information in the target image to reversely verify the calibration accuracy.
6. The positioning device according to claim 5, characterized in that: Also includes: Control unit; the control unit is used to control the manipulator to perform a positioning operation according to the accurate position of the target position of the detection object in the world coordinate system.
7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the Mark point marking-based dynamic flying shooting high-precision positioning method according to any one of claims 1-3 is implemented.
8. A visual inspection device, characterized in that: The visual inspection equipment adopts a positioning device as described in any one of claims 4-6 to implement a dynamic flying shooting high-precision positioning method based on Mark point marking.
Citation Information
Patent Citations
Plane feature point navigation and positioning method and device
CN104766309A
Flight shot compensation method and assembly method
CN114422695A
