Image target matching and positioning method, storage medium and computer

Abstract

The present invention provides an image target matching and positioning method, a storage medium and a computer. The method comprises: converting edge points in a template image into line segments to obtain an edge line segment set, describing each line segment in the obtained edge line segment set to obtain a line segment descriptor; converting edge points in a detection image into line segments to obtain an edge line segment set, describing each line segment in the obtained edge line segment set to obtain a line segment descriptor; matching and screening the line segment descriptors in the template image with the line segment descriptors in the detection image to obtain a pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image; and calculating an affine change matrix between the template image and the detection image based on the pairing relationship between the line segment descriptors in the template image and the detection image to obtain a coordinate mapping relationship between the template image in the detection image.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an image target matching and positioning method, a storage medium and a computer. Background Art

[0002] The edge of an image is the main basis for judging the position, rotation angle, and scaling of the target area in the image. Compared with the entire image, the edge information of an image has less information, more obvious features, and is less affected by external conditions such as lighting. Therefore, contour and edge features are widely used in image feature matching. Traditional point feature-based algorithms such as SIFT, SURF, FAST, and corner points have very limited applications in image matching and positioning due to their real-time nature, sensitivity to image noise, and insufficient matching accuracy.

[0003] Therefore, it is necessary to propose an improved solution to overcome the above problems. Summary of the invention

[0004] The object of the present invention is to provide an image target matching and positioning method, a storage medium and a computer, which can overcome the shortcomings of traditional point feature algorithms and quickly and easily perform matching and positioning based on the edges of images.

[0005] To achieve the purpose of the invention, according to one aspect of the present invention, the present invention provides an image target matching and positioning method, which includes: converting edge points in a template image into line segments to obtain an edge segment set, and describing each line segment in the obtained edge segment set to obtain a line segment descriptor; converting edge points in a detection image into line segments to obtain an edge segment set, and describing each line segment in the obtained edge segment set to obtain a line segment descriptor; matching and screening the line segment descriptors in the template image with the line segment descriptors in the detection image to obtain a pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image; and calculating an affine change matrix between the template image and the detection image based on the pairing relationship between the line segment descriptors in the template image and the detection image to obtain a coordinate mapping relationship of the template image in the detection image.

[0006] According to another aspect of the present invention, the present invention provides a storage medium storing program instructions, wherein the image target matching and positioning method described above is executed when the program is executed.

[0007] According to yet another aspect of the present invention, the present invention provides a computer, comprising a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the image target matching and positioning method described above.

[0008] Compared with the prior art, the present invention can find a specific target area in a detection image quickly and with high precision based on the image edge, and obtain the coordinate offset, rotation angle, scaling and other change factors of the target area. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A schematic diagram of a flow chart of an image target matching and positioning method in one embodiment of the present invention;

[0010] Figure 2 is an example of a template image and a set of edge segments therein in the present invention;

[0011] Figure 3 is an example of a detection image and a set of edge segments therein in the present invention;

[0012] Figure 4 It is the matching relationship drawn in the detection image with the template image;

[0013] Figure 5 It is a schematic diagram of the coordinate position of the template image in the detection image. DETAILED DESCRIPTION

[0014] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0015] The present invention proposes an image target matching and positioning method based on image edge features, which can overcome the shortcomings of traditional point feature algorithms and quickly and simply perform matching and positioning based on the edges of images.

[0016] Figure 1 FIG. 1 is a flow chart of an image target matching and positioning method 100 in one embodiment of the present invention. Figure 1 As shown, the image target matching and positioning method 100 includes the following steps.

[0017] Step 110 , converting edge points in the template image into line segments to obtain an edge line segment set, and describing each line segment in the obtained edge line segment set to obtain a line segment descriptor.

[0018] You can first select a template image that contains specific wheel area distribution features and has obvious and unique contour features, such as product logos, traffic signs, specific parts, etc. For quick processing later, the template image can be grayed out to reduce the amount of information in the template image, which is more conducive to edge information extraction. In addition, the grayed out template image can be preprocessed, and the preprocessing includes one or more of the following processes: bilateral filtering, eliminating image noise, and smoothing the image. Figure 2 As shown, it is an example of a template image.

[0019] In order to convert edge points in the template image into line segments to obtain an edge line segment set, edge calculation may be performed on the template image to obtain a comprehensive edge gradient image of the template image. Specifically, a Sobel operator may be used to perform horizontal convolution calculation on the template image to obtain a horizontal edge gradient image of the template image, a Sobel operator may be used to perform vertical convolution calculation on the template image to obtain a vertical edge gradient image of the template image, and then the horizontal edge gradient image and the vertical edge gradient image of the template image may be added to obtain a comprehensive edge gradient image of the template image.

[0020] Then, the comprehensive edge gradient image of the template image is thresholded. Specifically, pixels in the comprehensive edge gradient image that are lower than a first gradient threshold are set to zero to reduce image noise and weak edges.

[0021] Afterwards, the edge points in the thresholded integrated edge gradient image are segmented to obtain an edge segment set. Specifically, the edge points in the thresholded integrated edge gradient image that are greater than the first gradient threshold are traversed, stored in a linked list data structure, and sorted from large to small by gradient. The first 1% of the pixels in the linked list are used as seed pixels, and the edge pixels in their neighborhood whose expected gradient direction is less than the second gradient threshold are clustered. After repeated iterations, the edge points are segmented, and a minimum length threshold is set to remove shorter segments. After this step, the amount of data in the template image is drastically reduced, and the original pixel information of the template image is converted into a small number of line segment information, while retaining the contour information in the template image as much as possible. The line segments in this article can also be called edge segments. For example Figure 2 As shown, the line segments detected in the template image are illustrated.

[0022] For each line segment in the obtained edge segment set, an ORB (Oriented Brief) description algorithm is used to generate a line segment descriptor with a main direction and a radius length. The main direction of the line segment descriptor is determined by the gradient direction of the edge segment, and the radius length of the line segment descriptor is determined by the line segment length. This ensures that the line segment descriptor of the image area maintains the consistency of angle and scale scaling with the image.

[0023] In a preferred embodiment, the line segment descriptors of the template image can be rasterized to generate a raster feature vector. The rasterization process is to collect line segment descriptors in a small area (grid area) to reduce the amount of rough matching calculation and improve robustness, and generate a raster feature vector. The size of the grid area can be set according to the size of the image and the number of line segments.

[0024] Step 120 , converting edge points in the detected image into line segments to obtain a line segment set, and describing each line segment in the obtained line segment set to obtain a line segment descriptor.

[0025] A detection image is obtained, which may include the image target in the template image. For subsequent rapid processing, the detection image may be grayed to reduce the amount of information in the detection image, which is more conducive to edge information extraction. In addition, the grayed detection image may be preprocessed, and the preprocessing may include one or more of the following processes: bilateral filtering, image noise elimination, and image smoothing. Figure 3 As shown, it is an example of a detection image.

[0026] In order to convert edge points in the detection image into line segments to obtain a set of edge line segments, edge calculation can be performed on the detection image to obtain a comprehensive edge gradient image of the detection image. Specifically, the detection image can be horizontally convolved using a Sobel operator to obtain a horizontal edge gradient image of the detection image, and the detection image can be vertically convolved using a Sobel operator to obtain a vertical edge gradient image of the detection image, and then the horizontal edge gradient image and the vertical edge gradient image of the detection image are added to obtain a comprehensive edge gradient image of the detection image.

[0027] Subsequently, a thresholding operation is performed on the integrated edge gradient image of the detection image. Specifically, pixels in the integrated edge gradient image that are lower than a first gradient threshold are set to zero to reduce image noise and weak edges.

[0028] Afterwards, the edge points in the thresholded integrated edge gradient image are segmented to obtain a set of edge segments. Specifically, the edge points in the thresholded integrated edge gradient image that are greater than the first gradient threshold are traversed, stored in a linked list data structure, and sorted from large to small by gradient. The first 1% of the pixels in the linked list are used as seed pixels, and the edge pixels in their neighborhood whose expected gradient direction is less than the second gradient threshold are clustered. After repeated iterations, the edge points are segmented, and a minimum length threshold is set to remove shorter segments. After this step, even if the contour in the image is rotated and scaled, its gradient direction maintains the same angle difference as the template image, and the length of the segment at a specific position is also guaranteed to be the same as the specific position in the template image. The scaling ratio solves the difficulties of rotation and scaling in traditional template matching. The segments in this article can also be called edge segments. Figure 3 As shown, the line segments detected in the detection image are schematically shown.

[0029] For each line segment in the obtained edge segment set, an ORB (Oriented Brief) description algorithm is used to generate a line segment descriptor with a main direction and a radius length. The main direction of the line segment descriptor is determined by the gradient direction of the edge segment, and the radius length of the line segment descriptor is determined by the line segment length. This ensures that the line segment descriptor of the image area maintains the consistency of angle and scale scaling with the image.

[0030] In a preferred embodiment, the line segment descriptors of the detection image can be rasterized to generate a raster feature vector. The rasterization process is to collect line segment descriptors in a small area (grid area) to reduce the amount of rough matching operations and improve robustness, and generate a raster feature vector. The detection image can be rasterized in multiple sizes according to a specified zoom range.

[0031] Step 130 , matching and screening the line segment descriptors in the template image and the line segment descriptors in the detection image to obtain a pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image.

[0032] In one embodiment, the line segment descriptors in the template image and the line segment descriptors in the detection image can be randomly sampled and consistent (RASANC) screened to obtain a pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image. In this way, although the pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image can be obtained, the amount of calculation required is large, and the accuracy can be further improved.

[0033] In a preferred embodiment, at this time, the template image and the detection image are both gridded to obtain grid feature vectors. The step 130 specifically includes the following steps.

[0034] Step 131, traverse and calculate the Euclidean distance between the grid feature vector in the template image and the grid feature vector in the detection image, find one or more grid feature vectors with the smallest Euclidean distance to each grid feature vector of the template image in the detection image in turn, and set the grid corresponding to the grid feature vector found in the detection image as the candidate grid for matching and pairing with the grid corresponding to each grid feature vector of the template image; it should be noted that the one or more grid feature vectors with the smallest Euclidean distance to each grid feature vector of the template image include not only the grid feature vector with the smallest Euclidean distance, but may also include one or several next smallest grid feature vectors, and the specific selection of the smallest several grid feature vectors can be determined according to the set rules.

[0035] Step 132, sequentially traverse the line segment descriptors in each grid of the template image and the line segment descriptors in the candidate grids paired with each grid in the detection image, and screen the Euclidean distances between these descriptors to obtain a preliminary pairing relationship between the line segment descriptors in the template image and the descriptors in the detection image;

[0036] Step 133 , performing random sample consensus (RASANC) screening on the line segment descriptors in the template image and the line segment descriptors in the detection image that have a preliminary pairing relationship, to obtain a final pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image.

[0037] Compared with the screening matching scheme without rasterization, this embodiment can improve the matching accuracy, speed up the matching speed, and save the amount of calculation.

[0038] Figure 4 It is the matching relationship drawn in the detection image with the template image.

[0039] Step 140, based on the pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image, the affine change matrix between the template image and the detection image is calculated by iterative optimization using the least square method to obtain the coordinate mapping relationship of the template image in the detection image. Based on the coordinate mapping relationship, information such as position offset, angle rotation, and scale scaling of the template image in the detection image can be obtained. Figure 5 It is a schematic diagram of the coordinate position of the template image in the detection image.

[0040] In this way, the present invention can find a specific target area in a detection image quickly and with high accuracy based on the image edge, and obtain the coordinate offset, rotation angle, scaling and other change factors of the target area.

[0041] According to another aspect of the present invention, the present invention provides a storage medium storing program instructions, which are executed to execute the image target matching and positioning method described above. For simplicity, the specific content of the image target matching and positioning method is not repeated here.

[0042] According to another aspect of the present invention, the present invention provides a computer, comprising a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the image target matching and positioning method described above. For simplicity, the specific content of the image target matching and positioning method is not repeated here.

[0043] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0044] In this document, the directional words such as front, back, top, and bottom are defined by the positions of the components in the drawings and the positions of the components relative to each other, and are only for the sake of clarity and convenience in expressing the technical solution. It should be understood that the use of the directional words should not limit the scope of protection claimed in this application.

[0045] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0046] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0047] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0048] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0049] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. An image target matching and positioning method, characterized in that: It includes: The edge points in the template image are segmented to obtain an edge segment set, and each segment in the obtained edge segment set is described to obtain a segment descriptor; Segmenting edge points in the detected image to obtain an edge segment set, describing each segment in the obtained edge segment set to obtain a segment descriptor, wherein the segment descriptor has a main direction and a radius length, wherein the main direction of the segment descriptor is determined by the gradient direction of the edge segment, and the radius length of the segment descriptor is determined by the segment length; Matching and screening the line segment descriptors in the template image with the line segment descriptors in the detection image to obtain a pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image; and Based on the pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image, an affine change matrix between the template image and the detection image is calculated to obtain a coordinate mapping relationship between the template image and the detection image. It also includes: Rasterizing the line segment descriptor of the template image to generate a raster feature vector; Rasterize the line segment descriptors in the detection image to generate a raster feature vector; The matching and screening of the line segment descriptors in the template image and the line segment descriptors in the detection image to obtain the pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image includes: Traversing and calculating the Euclidean distance between the grid feature vector in the template image and the grid feature vector in the detection image, sequentially finding one or more grid feature vectors in the detection image that have the smallest Euclidean distance with each grid feature vector of the template image, and setting the grid corresponding to the grid feature vector found in the detection image as a candidate grid for matching and pairing with the grid corresponding to each grid feature vector of the template image; Sequentially traverse the line segment descriptors in each grid of the template image and the line segment descriptors in the candidate grids paired with each grid in the detection image, and screen the Euclidean distances between these descriptors to obtain a preliminary pairing relationship between the line segment descriptors in the template image and the descriptors in the detection image; The line segment descriptors in the template image and the line segment descriptors in the detection image that have a preliminary pairing relationship are randomly sampled and screened to obtain a final pairing relationship between the line segment descriptors in the template image and the line segment descriptors in the detection image.

2. The image target matching and positioning method according to claim 1, characterized in that: The step of converting edge points in the template image into line segments to obtain an edge line segment set comprises: Performing edge calculation on the template image to obtain a comprehensive edge gradient image of the template image; Performing a thresholding operation on the integrated edge gradient image of the template image; and The edge points in the thresholded integrated edge gradient image are segmented to obtain an edge segment set; The step of converting edge points in the detected image into line segments to obtain an edge line segment set comprises: Performing edge calculation on the detection image to obtain a comprehensive edge gradient image of the detection image; Performing a thresholding operation on the integrated edge gradient image of the detection image; and The edge points in the thresholded integrated edge gradient image are segmented to obtain an edge segment set.

3. The image target matching and positioning method according to claim 2, characterized in that: The step of performing edge calculation on the template image to obtain a comprehensive edge gradient image of the template image includes: Performing horizontal convolution calculation on the template image using a Sobel operator to obtain a horizontal edge gradient image of the template image; Performing vertical convolution calculation on the template image using a Sobel operator to obtain a vertical edge gradient image of the template image; and Adding the horizontal edge gradient image and the vertical edge gradient image of the template image to obtain a comprehensive edge gradient image of the template image; The step of performing edge calculation on the detection image to obtain a comprehensive edge gradient image of the detection image comprises: Performing horizontal convolution calculation on the detection image using a Sobel operator to obtain a horizontal edge gradient image of the detection image; Performing vertical convolution calculation on the detection image using a Sobel operator to obtain a vertical edge gradient image of the detection image; and The horizontal edge gradient image and the vertical edge gradient image of the detection image are added together to obtain a comprehensive edge gradient image of the detection image.

4. The image target matching and positioning method according to claim 2, characterized in that: Before edge computing is performed on the template image, the method further includes: Grayscale processing and preprocessing are performed on the template image, wherein the preprocessing includes one or more of the following processing: bilateral filtering, image noise elimination and image smoothing; Before edge computing is performed on the detected image, it also includes: The image to be detected is subjected to grayscale processing and preprocessing, wherein the preprocessing includes one or more of the following processing: bilateral filtering, image noise elimination and image smoothing.

5. The image target matching and positioning method according to claim 2, characterized in that: The thresholding operation of the integrated edge gradient image of the template image includes: setting pixels below a first gradient threshold to zero.

6. The image target matching and positioning method according to claim 2, characterized in that: The step of converting edge points in the thresholded integrated edge gradient image into line segments to obtain an edge line segment set comprises: The edge points in the thresholded integrated edge gradient image that are greater than the first gradient threshold are traversed, stored in a linked list data structure, and sorted from large to small by gradient. The first 1% of pixels in the linked list are used as seed pixels, and the edge pixel points in their neighborhood whose expected gradient direction is less than the second gradient threshold are clustered. After repeated iterations, the edge points are segmented, and a minimum length threshold is set to remove shorter segments.

7. The image target matching and positioning method according to claim 1, characterized in that: The line segment descriptor is generated for each line segment in the obtained edge line segment set using the ORB description algorithm.

8. A storage medium, characterized in that: It stores program instructions, and when the program is executed, it is run to execute the image target matching and positioning method as described in any one of claims 1-7.

9. A computer, characterized in that: It includes a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the image target matching and positioning method as described in any one of claims 1-7.

Images