Image positioning method, device and related equipment

Abstract

The present invention discloses an image positioning method, device and related equipment, which relate to the field of image positioning, wherein the image positioning method comprises the following steps: obtaining an image to be tested with respect to a positioning mark; obtaining feature points to be tested; matching the template image with respect to the positioning mark and the image to be tested according to the template feature points and the feature points to be tested; when the matching fails, taking the position of the center point of the circumscribed rectangle of the first connected domain as the positioning position; and controlling the movement of the visual field of the visual system according to the deviation between the positioning position and the center position of the visual field of the visual system. The image positioning method of the present application can solve the problem that the local area of ​​the positioning mark in the image to be tested may be blocked or the positioning mark may not appear completely in the visual field, resulting in the positioning mark being unable to be fully identified, causing the template matching to fail, and then causing the image positioning to fail, thereby achieving positioning when the local area of ​​the positioning mark is blocked.

Description

Technical Field

[0001] The invention belongs to the field of image positioning, and in particular relates to an image positioning method, a device and related equipment. Background Art

[0002] The existing image positioning method for OLED inkjet printing generally involves capturing the image to be tested, identifying the positioning mark pre-set in the image to be tested and obtaining the positioning point based on the template matching method, and controlling the center of the visual system's field of view to align with the positioning point to position the image to be tested. However, the template matching method requires the positioning mark in the image to be tested to be fully identified in order to match it with the preset template. During the image positioning process, the motion mechanism used to control the movement of the field of view in the visual system may not move into place, resulting in the positioning mark not being able to appear completely in the visual field of the visual system. In addition, the ink droplets generated by the OLED inkjet printing device after working for a long time may contaminate the local area of ​​the image to be tested and block the positioning mark. Therefore, during the template matching process, the positioning mark in the image to be tested may not be fully identified, resulting in a template matching failure, which in turn leads to a failure in image positioning.

[0003] Therefore, the existing technology needs to be improved and developed. Summary of the invention

[0004] The purpose of the present application is to provide an image positioning method, device and related equipment, aiming to solve the problem that a local area of ​​a positioning mark in the image to be measured may be blocked or the positioning mark may not appear completely in the field of view, resulting in the positioning mark being unable to be fully recognized, causing template matching failure, and further leading to image positioning failure.

[0005] In a first aspect, the present application provides an image positioning method for controlling the center of the field of view of a visual system to align with an image to be measured based on a positioning mark, comprising the following steps:

[0006] S1. Obtaining the image to be tested about the positioning mark;

[0007] S2. Extract feature points from the image to be tested to obtain feature points to be tested;

[0008] S3. Matching the template image and the image to be tested about the positioning mark according to the template feature points and the feature points to be tested, the template feature points are feature points obtained by pre-processing the template image for feature point extraction;

[0009] S4. When the match is successful, the position of the positioning mark in the image to be tested is obtained based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and a regional image with the same shape as the template image is cut out from the image to be tested with the positioning mark as the center, and the positioning mark in the regional image is identified to obtain the positioning position; when the match fails, a first connected domain is obtained according to the image to be tested, and the first connected domain is the largest white connected domain in the image to be tested whose circumscribed rectangle size meets the requirements, and the position of the center point of the circumscribed rectangle of the first connected domain is the positioning position;

[0010] S5. controlling the movement of the visual field of the visual system according to the deviation between the positioning position and the center position of the visual field of the visual system;

[0011] S6. When the center of the visual system's field of view is not aligned with the positioning mark, repeat steps S1-S5 until the center of the visual system's field of view is aligned with the positioning mark.

[0012] The image positioning method of the present application can eliminate interference in the image to be tested and obtain a clear and accurate outline of the positioning mark in the image to be tested, thereby making the positioning of the image to be tested based on the positioning mark more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be tested, resulting in a matching failure, so that the complete positioning mark is located in the image to be tested, thereby making the next round of matching results successful to obtain a more accurate positioning position.

[0013] Furthermore, the feature point extraction process includes edge detection, the template feature points include first marked edge points, the feature points to be measured include first edge points, the first marked edge points are edge points obtained by edge detection of the positioning marks in the template image, and the first edge points are edge points obtained by edge detection of the image to be measured.

[0014] Furthermore, in step S2, feature point extraction is performed on the image to be tested, and the process of obtaining the feature points to be tested includes:

[0015] S21. Detecting preliminary edge points based on the image to be tested and calculating their gradient amplitudes based on the Sobel operator, where the preliminary edge points are edge points in the image to be tested;

[0016] S22. Performing double threshold detection on the preliminary edge points based on the gradient amplitude of the preliminary edge points to screen out the first strong edge points and the first weak edge points;

[0017] S23. Screen the first strong edge point and the first weak edge point based on the hysteresis boundary tracking algorithm to obtain the first edge point.

[0018] In this embodiment, the image positioning method of the present application can accurately filter out edge points corresponding to the edge of the positioning mark from the image to be tested based on dual threshold detection and hysteresis boundary tracking algorithm, thereby making the matching result performed in step S3 more accurate.

[0019] Furthermore, the process of acquiring the template feature points includes:

[0020] A1. Establish an image pyramid based on the template image. The image pyramid has multiple levels, and each level includes a level image. The level image is a template image with different sizes and resolutions. The higher the level, the smaller the level image size and the lower the resolution.

[0021] A2. Perform edge detection on the positioning mark in the highest level image to obtain the first marked edge point in the highest level image;

[0022] In step S3, the process of matching the template image with the image to be tested according to the template feature points and the feature points to be tested includes:

[0023] S31. Perform hierarchical image matching until the matching fails or the lowest hierarchical image successfully matches the image to be tested. The hierarchical image matching process includes:

[0024] S311. Matching the current level image and the image to be tested according to the first marked edge point and the first edge point in the current level image, wherein the initial current level image is the highest level image;

[0025] S312. When the match is successful, the successfully matched first marked edge point is mapped to the same position in the hierarchical image of the next level, the local area where the first marked edge point is located in the hierarchical image of the next level is used as the new current hierarchical image, and the mapped first marked edge point is used as the first marked edge point in the new current hierarchical image.

[0026] In this embodiment, the image positioning method of the present application can quickly match the image to be tested and the template image based on the image pyramid.

[0027] Furthermore, in step S4, the process of identifying the positioning mark in the regional image and obtaining the positioning position includes:

[0028] S41. Extract the edge of the regional image to obtain a second marked edge point;

[0029] S42. Calculate the positioning position according to the second marked edge point.

[0030] Furthermore, in step S42, the process of calculating the positioning position according to the second marked edge point includes:

[0031] S421. Perform circle fitting or straight line fitting on the second marked edge point to obtain a fitting contour;

[0032] S422. Calculate the positioning position according to the position of the fitted contour.

[0033] Furthermore, the process of obtaining the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and cutting out a region image having the same shape as the template image from the image to be tested with the positioning mark as the center and step S41 also includes:

[0034] B1. Perform interference removal on the regional image based on the image processing algorithm.

[0035] In a second aspect, the present application provides an image positioning device for controlling the center of the field of view of a visual system to align with an image to be measured based on a positioning mark, comprising:

[0036] An image acquisition module, used for acquiring an image to be tested with respect to the positioning mark;

[0037] A feature point extraction module is used to extract feature points from the image to be tested and obtain feature points to be tested;

[0038] A matching module, used for matching the template image with the image to be tested based on the template feature points and the feature points to be tested, wherein the template feature points are feature points obtained by performing feature point extraction processing on the template image in advance;

[0039] A positioning module is used to obtain the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points when the matching is successful, and to intercept a regional image with the same shape as the template image from the image to be tested with the positioning mark as the center, identify the positioning mark in the regional image, and obtain the positioning position; when the matching fails, obtain a first connected domain according to the image to be tested, the first connected domain is the largest white connected domain in the image to be tested whose circumscribed rectangle size meets the requirements, and the position of the center point of the circumscribed rectangle of the first connected domain is the positioning position;

[0040] A field of view movement module, used for controlling the field of view movement of the visual system according to the deviation between the positioning position and the center position of the field of view of the visual system;

[0041] The alignment module is used to repeatedly run the image acquisition module, the feature point extraction module, the matching module, the positioning module and the field of view movement module to repeatedly execute corresponding steps when the center of the field of view of the visual system is not aligned with the positioning mark until the center of the field of view of the visual system is aligned with the positioning mark.

[0042] The image positioning device of the present application can eliminate interference in the image to be tested and obtain a clear and accurate outline of the positioning mark in the image to be tested, so that the positioning of the image to be tested based on the positioning mark can be more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be tested, resulting in a matching failure, so that the complete positioning mark is located in the image to be tested, thereby making the next round of matching results successful to obtain a more accurate positioning position.

[0043] In a third aspect, the present application provides an electronic device, including a processor and a memory, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, it runs the steps in any of the above methods.

[0044] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above methods are performed.

[0045] As can be seen from the above, the present application provides an image positioning method, device and related equipment, wherein the image positioning method provided by the present application can eliminate interference in the image to be measured and obtain a clear and accurate outline of the positioning mark in the image to be measured, so that the positioning of the image to be measured based on the positioning mark can be more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in the image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be measured, resulting in a matching failure, so that the complete positioning mark is located in the image to be measured, thereby making the next round of matching results become successful to obtain a more accurate positioning position.

[0046] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 A flowchart of an image positioning method provided in an embodiment of the present application.

[0048] Figure 2 A schematic diagram of the structure of an image positioning device provided in an embodiment of the present application.

[0049] Figure 3A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0050] Explanation of reference numerals: 1. Image acquisition module; 2. Feature point extraction module; 3. Matching module; 4. Positioning module; 5. Field of view movement module; 6. Alignment module; 301. Processor; 302. Memory; 303. Communication bus. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0052] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0053] First, as Figure 1 As shown, the present application provides an image positioning method for controlling the center of the field of view of a visual system to align with an image to be measured based on a positioning mark, comprising the following steps:

[0054] S1. Obtaining the image to be tested about the positioning mark;

[0055] S2. Extract feature points from the image to be tested to obtain feature points to be tested;

[0056] S3. Matching the template image and the image to be tested about the positioning mark according to the template feature points and the feature points to be tested, the template feature points are feature points obtained by pre-processing the template image for feature point extraction;

[0057] S4. When the match is successful, the position of the positioning mark in the image to be tested is obtained based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and a regional image with the same shape as the template image is cut out from the image to be tested with the positioning mark as the center, and the positioning mark in the regional image is identified to obtain the positioning position; when the match fails, a first connected domain is obtained according to the image to be tested, and the first connected domain is the largest white connected domain in the image to be tested whose circumscribed rectangle size meets the requirements, and the position of the center point of the circumscribed rectangle of the first connected domain is the positioning position;

[0058] S5. controlling the movement of the visual field of the visual system according to the deviation between the positioning position and the center position of the visual field of the visual system;

[0059] S6. When the center of the visual system's field of view is not aligned with the positioning mark, repeat steps S1-S5 until the center of the visual system's field of view is aligned with the positioning mark.

[0060] Specifically, the positioning mark is a pattern pre-set on the image to be tested for positioning the image to be tested, and its color is generally white. In addition, the image to be tested is generally used to position a device such as a substrate or a nozzle, so in addition to the positioning mark, there are generally silver-gray lines or black patterns in the image to be tested.

[0061] More specifically, the template image regarding the positioning mark is an image regarding a complete and clear positioning mark.

[0062] More specifically, the template feature points can characterize the position and graphic characteristics of the positioning mark in the template image, and the feature points to be measured can characterize the position and graphic characteristics of the pattern in the image to be measured.

[0063] More specifically, the feature point extraction process can be an existing method that can obtain unique and stable feature points from an image. Preferably, the feature point extraction process method used to obtain the feature points to be measured is the same as the feature point extraction process method used to obtain the template feature points.

[0064] More specifically, the method for matching the template image and the image to be tested may be an existing matching method that can establish a corresponding relationship between the template image and the image to be tested based on the similarity between the template feature points and the feature points to be tested. In addition, it may be determined whether the template feature points and the feature points to be tested have a similarity score higher than a preset threshold to determine whether the matching is successful or not.

[0065] More specifically, the circumscribed rectangle size meeting the requirement means that the long side and the short side are respectively larger than a first long side threshold and a first short side threshold, and the first long side threshold and the first short side threshold are both preset values.

[0066] More specifically, the method for obtaining the first connected domain according to the image to be tested can be to obtain the connected domain in the image to be tested, screen the white connected domain whose long and short sides of the circumscribed rectangle are respectively greater than the first long side threshold and the first short side threshold, and use the screened white connected domain with the largest circumscribed rectangle as the first connected domain; wherein the method for obtaining the connected domain in the image to be tested can be to perform binarization processing on the image to be tested based on threshold segmentation to obtain the connected domain, or other existing methods that can obtain the connected domain; the binarization processing on the image to be tested can be based on simple threshold segmentation, dynamic threshold segmentation or other existing threshold segmentation methods. Preferably, the binarization processing on the image to be tested is performed based on dynamic threshold segmentation.

[0067] More specifically, since the patterns in the two images are considered to be consistent when the similarity of the feature points in the two images is high enough, when the matching based on the feature points to be tested and the template feature points is successful, it can be considered that the pattern corresponding to the feature points to be tested is the positioning mark in the image to be tested. Therefore, in step S4, when the matching is successful, the present application obtains the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and can preliminarily determine the position of the positioning mark in the image to be tested, which is convenient for subsequent precise positioning of the image to be tested based on the positioning mark; a regional image with the same shape as the template image is cut out from the image to be tested with the position of the positioning mark in the image to be tested as the center, which can reduce the area of ​​the image about the positioning mark, thereby quickly eliminating the interference of other irrelevant patterns in the image to be tested on the recognition of the positioning mark, which is convenient for subsequent acquisition of the positioning position for positioning the image to be tested.

[0068] More specifically, a failed match indicates that the positioning mark in the image to be tested may be obscured, and the color of the positioning mark is different from the color of other areas. Therefore, in step S4, when the match fails, the present application only needs to extract the connected domain with white color to eliminate the interference in the image to be tested and obtain a clear and accurate outline of the unobstructed area of ​​the positioning mark, which is convenient for further judgment of the situation where the positioning mark in the image to be tested is obscured.

[0069] More specifically, when the local area of ​​the positioning mark in the image to be tested is not large, there is no deviation or a small deviation between the center of the circumscribed rectangle of the first connected domain and the center of the complete positioning mark. Therefore, it is only necessary to reasonably set the first long side threshold and the first short side threshold, and use the position of the center point of the circumscribed rectangle of the first connected domain as the positioning position, so that positioning can be achieved when the local area of ​​the positioning mark is blocked; in addition, the image to be tested may have a pattern that is white in color but is not a positioning mark. Therefore, the present application only uses the connected domain whose circumscribed rectangle meets the requirements as the first connected domain and obtains the positioning position, which can eliminate the interference caused by the pattern of the same color as the positioning mark.

[0070] More specifically, a matching failure may also mean that the positioning mark has a local area outside the image to be tested. Since there is no deviation or the deviation is small between the positioning position and the center of the complete positioning mark, in this case, the present application obtains the first connected domain, obtains the positioning position, and performs field of view alignment based on the positioning position in steps S4 and S5, which can ensure that the complete positioning mark appears in the field of view, and can cooperate with the repeated positioning process of step S6 to make the result of the next round of matching successful; and the positioning position obtained in the case of a successful match is more accurate than the positioning position obtained in the case of a matching failure. Therefore, when the positioning mark has a local area outside the image to be tested, resulting in a matching failure, the present application can perform preliminary positioning based on the positioning position to make the complete positioning mark located in the image to be tested, thereby making the next round of matching results successful to obtain a more accurate positioning position.

[0071] More specifically, when the long sides and short sides of all connected domains are respectively less than or equal to the first long side threshold and the first short side threshold, it can be known that the positioning mark is completely blocked or the blocked area is large, or the positioning mark has a large local area outside the image to be tested or is completely outside the image to be tested. The current image to be tested is unavailable, and an alarm needs to be issued to prompt the operator to change the image to be tested or perform other processing.

[0072] The image positioning method of the present application can eliminate interference in the image to be tested and obtain a clear and accurate outline of the positioning mark in the image to be tested, thereby making the positioning of the image to be tested based on the positioning mark more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be tested, resulting in a matching failure, so that the complete positioning mark is located in the image to be tested, thereby making the next round of matching results successful to obtain a more accurate positioning position.

[0073] Preferably, the template image is an image that performs ROI region cropping based on the positioning mark.

[0074] More preferably, after the ROI region is cropped based on the positioning mark, the template image is also smoothed based on a Gaussian filter with a Gaussian kernel size of 5X5.

[0075] In this embodiment, the image positioning method of the present application can reduce the noise in the template image, thereby improving the accuracy of the matching result performed in step S3.

[0076] In some preferred embodiments, the feature point extraction process includes edge detection, the template feature points include first marked edge points, the feature points to be measured include first edge points, the first marked edge points are edge points obtained by edge detection of the positioning marks in the template image, and the first edge points are edge points obtained by edge detection of the image to be measured.

[0077] Specifically, edge detection is a method for identifying the edge of a pattern contour in an image, which can be performed based on existing operators such as Sobel and Canny that can implement edge detection.

[0078] More specifically, the first mark edge point is a point on the edge of the positioning mark in the template image, and the first edge point is a point on the edge of the pattern in the image to be measured.

[0079] In this embodiment, the feature point extraction process of the image positioning method of the present application includes edge detection, which can enable the template feature points to better characterize the position and graphic characteristics of the positioning mark in the template image, and can enable the feature points to be measured to better characterize the position and graphic characteristics of the pattern in the template image, thereby making the matching result performed in step S3 more accurate.

[0080] In some preferred embodiments, in step S2, feature point extraction is performed on the image to be tested, and the process of obtaining the feature points to be tested includes:

[0081] S21. Detecting preliminary edge points based on the image to be tested and calculating their gradient amplitudes based on the Sobel operator, where the preliminary edge points are edge points in the image to be tested;

[0082] S22. Performing double threshold detection on the preliminary edge points based on the gradient amplitude of the preliminary edge points to screen out the first strong edge points and the first weak edge points;

[0083] S23. Screen the first strong edge point and the first weak edge point based on the hysteresis boundary tracking algorithm to obtain the first edge point.

[0084] Specifically, the process of performing dual threshold detection on the preliminary edge points based on the gradient amplitude of the preliminary edge points to screen out the first strong edge points and the first weak edge points is to pre-set a first gradient threshold and a second gradient threshold that is less than the first gradient threshold, and to take the preliminary edge points whose gradient amplitude is greater than or equal to the first gradient threshold as the first strong edge points, and take the preliminary edge points whose gradient amplitude is less than the first gradient threshold and greater than or equal to the second gradient threshold as the first weak edge points, and to remove the preliminary edge points whose gradient amplitude is less than the second gradient threshold.

[0085] More specifically, the process of screening the first strong edge points and the first weak edge points based on the hysteresis boundary tracking algorithm is to retain all the first strong edge points and the first weak edge points in the surrounding eight connected areas that have the first strong edge points, and remove the remaining preliminary edge points.

[0086] More specifically, due to the noise and color changes in the image to be tested, some preliminary edge points may not correspond to the edge of the positioning mark in the image to be tested. Therefore, in step S22, the present application performs double threshold detection on the preliminary edge points based on the gradient amplitude of the preliminary edge points to screen out the first strong edge point and the first weak edge point, and can preliminarily screen out the edge point corresponding to the edge of the positioning mark in the image to be tested, that is, the first edge point.

[0087] More specifically, in an image, if the first weak edge point screened out based on the dual threshold detection corresponds to the real edge of the pattern in the image to be tested rather than noise, then the first weak edge point is usually connected to the first strong edge point. Therefore, in step S23, the present application screens the first strong edge point and the first weak edge point based on the hysteresis boundary tracking algorithm to obtain the first edge point, which can accurately retain the first weak edge point corresponding to the edge of the positioning mark in the image to be tested, and exclude the first weak edge point corresponding to the noise of the image to be tested.

[0088] In this embodiment, the image positioning method of the present application can accurately filter out edge points corresponding to the edge of the positioning mark from the image to be tested based on dual threshold detection and hysteresis boundary tracking algorithm, thereby making the matching result performed in step S3 more accurate.

[0089] In some preferred embodiments, the process of acquiring template feature points includes:

[0090] A1. Establish an image pyramid based on the template image. The image pyramid has multiple levels, and each level includes a level image. The level image is a template image with different sizes and resolutions. The higher the level, the smaller the level image size and the lower the resolution.

[0091] A2. Perform edge detection on the positioning mark in the highest level image to obtain the first marked edge point in the highest level image;

[0092] In step S3, the process of matching the template image with the image to be tested according to the template feature points and the feature points to be tested includes:

[0093] S31. Perform hierarchical image matching until the matching fails or the lowest hierarchical image successfully matches the image to be tested. The hierarchical image matching process includes:

[0094] S311. Matching the current level image and the image to be tested according to the first marked edge point and the first edge point in the current level image, wherein the initial current level image is the highest level image;

[0095] S312. When the match is successful, the successfully matched first marked edge point is mapped to the same position in the hierarchical image of the next level, the local area where the first marked edge point is located in the hierarchical image of the next level is used as the new current hierarchical image, and the mapped first marked edge point is used as the first marked edge point in the new current hierarchical image.

[0096] Specifically, since the higher the level in the image pyramid, the smaller the image size and the lower the resolution, the matching process between the higher level images and the image to be tested can be carried out quickly, and the local area where the first marked edge point mapped to the next level is located is usually significantly smaller than the level image of the next level. Therefore, the next matching only needs to be performed on the local area of ​​the image to be tested and the image of the next level. Since the area of ​​the two images to be matched is small, the matching time is greatly reduced. Therefore, in this embodiment, the image positioning method of the present application can quickly match the image to be tested and the template image based on the image pyramid.

[0097] Preferably, in step S311, the process of matching the current level image with the image to be tested according to the first marked edge point and the first edge point in the current level image includes:

[0098] S3111. Divide the current hierarchical image into multiple hierarchical region images, successively obtain the similarity scores of each first marked edge point and the first edge point in each hierarchical region image, judge that the match is successful when there is a first marked edge point whose corresponding similarity score is greater than a first score threshold, and skip the similarity score acquisition process of the remaining first marked edge points and the first edge points in the hierarchical region image where the first marked edge point is located when there is a first marked edge point whose corresponding similarity score is less than a second score threshold.

[0099] Specifically, the second score threshold is smaller than the first score threshold.

[0100] More specifically, under normal circumstances, when there are feature points with too low similarity scores in two images, the two images do not match. Therefore, in this embodiment, the image positioning method of the present application can quickly skip the hierarchical area images whose similarity with the image to be tested is too low, thereby further improving the matching speed between the image to be tested and the template image.

[0101] Preferably, the process of matching the current level image and the image to be tested according to the first marked edge point and the first edge point in the current level image is performed based on a normalized cross-correlation matching algorithm.

[0102] In some preferred embodiments, in step S4, the process of identifying the positioning mark in the regional image and obtaining the positioning position includes:

[0103] S41. Extract the edge of the regional image to obtain a second marked edge point;

[0104] S42. Calculate the positioning position according to the second marked edge point.

[0105] Specifically, edge detection is a method for identifying the edge of a pattern contour in an image, which can be performed based on existing operators such as Sobel and Canny that can implement edge detection.

[0106] More specifically, the method for calculating the positioning position based on the second marker edge point can be based on the coordinates of the second marker edge point based on the positional relationship between the positioning mark and the positioning position acquired in advance, or can be based on other existing methods that can calculate the positioning position based on the second marker edge point.

[0107] More specifically, since the regional image is captured from the image to be tested with the position of the positioning mark in the image to be tested as the center when the match is successful, the positioning mark exists in the regional image. Therefore, the second edge point obtained by edge extraction of the regional image can characterize the edge features of the positioning mark in the regional image.

[0108] In this embodiment, the image positioning method of the present application can easily extract the second marker edge points based on the edge extraction method, so that the positioning position can be obtained quickly and accurately.

[0109] In some preferred embodiments, in step S42, the process of calculating the positioning position according to the second marked edge point includes:

[0110] S421. Perform circle fitting or straight line fitting on the second marked edge point to obtain a fitting contour;

[0111] S422. Calculate the positioning position according to the position of the fitted contour.

[0112] Specifically, the fitting contour is a contour obtained by fitting the second mark edge points based on a circle fitting process or a straight line fitting process.

[0113] More specifically, since the second mark edge point is a point that can characterize the edge features of the positioning mark in the regional image, and the contour of the positioning mark is generally a figure or a circle composed of a plurality of straight lines, in this embodiment, the present application performs circle fitting processing or straight line fitting processing on the second mark edge point, which can further eliminate interference in the regional image and obtain an accurate positioning mark contour; and the present application calculates the positioning position based on the position of the fitted contour, and can quickly and accurately obtain the positioning position.

[0114] In some preferred embodiments, the process of obtaining the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and cutting out a regional image with the same shape as the template image from the image to be tested with the positioning mark as the center and step S41 also includes:

[0115] B1. Perform interference removal on the regional image based on the image processing algorithm.

[0116] Specifically, after capturing the regional image centered on the positioning mark, although the positioning mark in the image to be tested is located within the regional image, interference may still exist within the regional image. Therefore, in this embodiment, the image positioning method of the present application can pre-eliminate the interference within the regional image before performing edge extraction on the regional image, thereby making the positioning position obtained by subsequent calculation more accurate.

[0117] Preferably, in step B1, the process of performing interference removal processing on the regional image based on the image processing algorithm includes:

[0118] B11. Performing threshold segmentation processing on the regional image based on the K-means clustering algorithm, and extracting the connected domain with black color in the regional image to obtain the second connected domain;

[0119] B12. Filter the second connected domain whose long side and short side of the circumscribed rectangle are respectively greater than the second long side threshold and the second short side threshold, and fill it with white.

[0120] Specifically, the second long edge threshold and the second short edge threshold are both preset values.

[0121] More specifically, since the ink droplets generated by the OLED inkjet printing device after working for a long time may contaminate the image to be tested, there may be ink drop stains in the positioning mark, affecting the subsequent edge extraction, and the positioning mark is generally white and the ink drop stains are black, and the K-means clustering algorithm can cluster pixels based on color features, thereby dividing the image into multiple regions based on color features. Therefore, in this embodiment, the image positioning method of the present application can find the ink drop stains of the positioning mark and fill them, which can make the positioning mark more complete and further eliminate interference, so that the positioning position obtained by subsequent calculation is more accurate.

[0122] Second, as Figure 2 As shown, the present application provides an image positioning device for controlling the center of the field of view of a visual system to align with an image to be measured based on a positioning mark, comprising:

[0123] An image acquisition module 1 is used to acquire an image to be tested with respect to the positioning mark;

[0124] Feature point extraction module 2, used to perform feature point extraction processing on the image to be tested to obtain the feature points to be tested;

[0125] A matching module 3 is used to match the template image with the image to be tested based on the template feature points and the feature points to be tested, wherein the template feature points are feature points obtained by performing feature point extraction processing on the template image in advance;

[0126] Positioning module 4, used for obtaining the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points when the matching is successful, and cutting out a regional image with the same shape as the template image from the image to be tested with the positioning mark as the center, identifying the positioning mark in the regional image, and obtaining the positioning position; when the matching fails, obtaining a first connected domain according to the image to be tested, the first connected domain is the largest connected domain of white color in the image to be tested whose circumscribed rectangle size meets the requirements, and taking the position of the center point of the circumscribed rectangle of the first connected domain as the positioning position;

[0127] A field of view movement module 5, used for controlling the field of view movement of the visual system according to the deviation between the positioning position and the center position of the field of view of the visual system;

[0128] The alignment module 6 is used to repeatedly run the image acquisition module 1, the feature point extraction module 2, the matching module 3, the positioning module 4 and the field of view movement module 5 to repeatedly execute the corresponding steps when the center of the field of view of the visual system is not aligned with the positioning mark until the center of the field of view of the visual system is aligned with the positioning mark.

[0129] The image positioning device of the present application can eliminate interference in the image to be tested and obtain a clear and accurate outline of the positioning mark in the image to be tested, so that the positioning of the image to be tested based on the positioning mark can be more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be tested, resulting in a matching failure, so that the complete positioning mark is located in the image to be tested, thereby making the next round of matching results successful to obtain a more accurate positioning position.

[0130] Thirdly, Figure 3As shown, the present application provides an electronic device, including a processor 301 and a memory 302, the processor 301 and the memory 302 are interconnected and communicate with each other through a communication bus 303 and / or other forms of connection mechanisms (not shown), the memory 302 stores a computer program executable by the processor 301, when the electronic device is running, the processor 301 executes the computer program to execute the method in any optional implementation of the above embodiment to achieve the following functions: obtaining an image to be tested with respect to a positioning mark; performing feature point extraction processing on the image to be tested to obtain feature points to be tested; matching the template image with respect to the positioning mark and the image to be tested according to the template feature points and the feature points to be tested, the template feature points being the feature points obtained by performing feature point extraction processing on the template image in advance When the match is successful, the position of the positioning mark in the image to be tested is obtained based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and a regional image with the same shape as the template image is cut out from the image to be tested with it as the center, the positioning mark in the regional image is identified, and the positioning position is obtained; when the match fails, a first connected domain is obtained according to the image to be tested, the first connected domain is the largest connected domain of white color in the image to be tested, the size of the circumscribed rectangle of which meets the requirements, and the position of the center point of the circumscribed rectangle of the first connected domain is taken as the positioning position; the movement of the field of view of the visual system is controlled according to the deviation between the positioning position and the center position of the field of view of the visual system; when the center of the field of view of the visual system is not aligned with the positioning mark, the above steps are repeated until the center of the field of view of the visual system is aligned with the positioning mark.

[0131] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method in any optional implementation of the above embodiments is executed to implement the following functions: obtaining an image to be tested with respect to a positioning mark; performing feature point extraction processing on the image to be tested to obtain feature points to be tested; matching the template image with respect to the positioning mark and the image to be tested according to the template feature points and the feature points to be tested, wherein the template feature points are feature points obtained by performing feature point extraction processing on the template image in advance; when the matching is successful, obtaining a feature point based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points Take the position of the positioning mark in the image to be tested, and use it as the center to cut out a regional image with the same shape as the template image from the image to be tested, identify the positioning mark in the regional image, and obtain the positioning position; when the matching fails, obtain the first connected domain according to the image to be tested, the first connected domain is the largest white connected domain with a circumscribed rectangle size that meets the requirements in the image to be tested, and take the position of the center point of the circumscribed rectangle of the first connected domain as the positioning position; control the movement of the visual system's field of view according to the deviation between the positioning position and the center position of the visual system's field of view; when the center of the visual system's field of view is not aligned with the positioning mark, repeat the above steps until the center of the visual system's field of view is aligned with the positioning mark. Among them, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read-Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0132] In summary, the present application provides an image positioning method, device and related equipment, wherein the image positioning method provided by the present application can eliminate interference in the image to be measured and obtain a clear and accurate outline of the positioning mark in the image to be measured, thereby making the positioning of the image to be measured based on the positioning mark more accurate, and the image positioning method of the present application can achieve positioning when a local area of ​​the positioning mark is blocked, and can prevent excessive deviation in the image positioning based on the positioning mark when the positioning mark is excessively blocked, and the present application can perform preliminary positioning based on the positioning position when the positioning mark has a local area outside the image to be measured, resulting in a matching failure, so that the complete positioning mark is located in the image to be measured, thereby making the next round of matching results become successful to obtain a more accurate positioning position.

[0133] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0134] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0135] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0136] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0137] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image positioning method for controlling the center of the visual field of a visual system to align with an image to be measured based on a positioning mark, characterized in that: The following steps are involved: S1. Obtaining the image to be tested about the positioning mark; S2. Perform feature point extraction on the image to be tested to obtain feature points to be tested, wherein the feature point extraction includes edge detection, and the feature points to be tested include first edge points, which are edge points obtained by edge detection of the image to be tested; S3. Matching the template image about the positioning mark and the image to be tested according to the template feature points and the feature points to be tested, wherein the template feature points are feature points obtained by pre-processing the template image with feature points, and the template feature points include first mark edge points, which are edge points obtained by edge detection of the positioning mark in the template image; S4. When the match is successful, the position of the positioning mark in the image to be tested is obtained based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and a regional image with the same shape as the template image is cut out from the image to be tested with the positioning mark as the center, and the positioning mark in the regional image is identified to obtain the positioning position; when the match fails, a first connected domain is obtained according to the image to be tested, and the first connected domain is the largest white connected domain in the image to be tested whose circumscribed rectangle size meets the requirements, and the position of the center point of the circumscribed rectangle of the first connected domain is the positioning position; S5. Controlling the movement of the visual system's field of view according to the deviation between the positioning position and the center position of the visual system's field of view; S6. When the center of the visual system's field of view is not aligned with the positioning mark, repeat steps S1-S5 until the center of the visual system's field of view is aligned with the positioning mark.

2. The image positioning method according to claim 1, characterized in that: In step S2, the process of extracting feature points from the image to be tested to obtain the feature points to be tested includes: S21. Detecting preliminary edge points based on the image to be tested and calculating the gradient amplitude thereof based on the Sobel operator, wherein the preliminary edge points are edge points in the image to be tested; S22. Performing double threshold detection on the preliminary edge points based on the gradient amplitude of the preliminary edge points to screen out the first strong edge points and the first weak edge points; S23. Screen the first strong edge point and the first weak edge point based on a hysteresis boundary tracking algorithm to obtain a first edge point.

3. The image positioning method according to claim 1, characterized in that: The process of acquiring the template feature points includes: A1. establishing an image pyramid according to the template image, wherein the image pyramid has multiple levels, and each level includes a level image, and the level images are template images of different sizes and resolutions, and the higher the level, the smaller the level image size and the lower the resolution; A2. Perform edge detection on the positioning mark in the highest level image to obtain the first marked edge point in the highest level image; In step S3, the process of matching the template image about the positioning mark and the image to be tested according to the template feature points and the feature points to be tested includes: S31. Perform hierarchical image matching until the matching fails or the lowest hierarchical image successfully matches the image to be tested, the hierarchical image matching process includes: S311. Matching the current level image and the image to be tested according to the first marked edge point and the first edge point in the current level image, wherein the initial current level image is the highest level image; S312. When the match is successful, the successfully matched first marked edge point is mapped to the same position in the hierarchical image of the next level, the local area where the first marked edge point is located in the hierarchical image of the next level is used as the new current hierarchical image, and the mapped first marked edge point is used as the first marked edge point in the new current hierarchical image.

4. The image positioning method according to claim 1, characterized in that: In step S4, the process of identifying the positioning mark in the regional image and obtaining the positioning position includes: S41. Extracting edges of the region image to obtain second marked edge points; S42. Calculate the positioning position according to the second marked edge point.

5. The image positioning method according to claim 4, characterized in that: In step S42, the process of calculating the positioning position according to the second marked edge point includes: S421. Performing circle fitting or straight line fitting on the second marked edge points to obtain a fitting contour; S422. Calculate the positioning position according to the position of the fitting contour.

6. The image positioning method according to claim 4, characterized in that: The process of obtaining the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points, and cutting out a region image with the same shape as the template image from the image to be tested with the positioning mark as the center and step S41 also includes: B1. Perform interference removal processing on the regional image based on an image processing algorithm.

7. An image positioning device, used to control the center of the visual field of a visual system to align with the image to be measured based on a positioning mark, characterized in that: include: An image acquisition module (1) is used to acquire an image to be tested related to the positioning mark; A feature point extraction module (2) is used to perform feature point extraction processing on the image to be tested to obtain feature points to be tested, wherein the feature point extraction processing includes edge detection, and the feature points to be tested include first edge points, which are edge points obtained by performing edge detection on the image to be tested; A matching module (3) is used to match the template image about the positioning mark and the image to be tested according to the template feature points and the feature points to be tested, wherein the template feature points are feature points obtained by pre-processing the template image for feature point extraction, and the template feature points include first mark edge points, which are edge points obtained by edge detection of the positioning mark in the template image; A positioning module (4) is used to obtain the position of the positioning mark in the image to be tested based on the position of the positioning mark in the template image and the successfully matched feature points to be tested and the template feature points when the matching is successful, and to cut out a regional image with the same shape as the template image from the image to be tested with the positioning mark as the center, identify the positioning mark in the regional image, and obtain the positioning position; when the matching fails, obtain a first connected domain based on the image to be tested, the first connected domain being the largest white connected domain in the image to be tested whose circumscribed rectangle size meets the requirements, and take the position of the center point of the circumscribed rectangle of the first connected domain as the positioning position; A field of view movement module (5), used for controlling the field of view movement of the visual system according to the deviation between the positioning position and the center position of the field of view of the visual system; An alignment module (6) is used to repeatedly run the image acquisition module (1), the feature point extraction module (2), the matching module (3), the positioning module (4) and the field of view movement module (5) when the field of view center of the visual system is not aligned with the positioning mark, so as to repeatedly execute corresponding steps until the field of view center of the visual system is aligned with the positioning mark.

8. An electronic device, characterized in that: The invention comprises a processor (301) and a memory (302), wherein the memory (302) stores a computer program executable by the processor (301), and when the processor (301) executes the computer program, the steps in the method according to any one of claims 1 to 6 are performed.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor (301), the steps in the method according to any one of claims 1 to 6 are performed.

Images