Image label matching method and system fusing ovr-svm and psnr similarity

Abstract

The application discloses a label image matching method and system combining OVR-SVM and PSNR similarity, and belongs to the technical field of label image matching. In the template matching, the application first reduces the image, and after the matching is completed, detection is performed on the original image again, the label is accurately positioned, and the real-time performance of the matching process is improved. Before the binary image is matched, the perspective matrix of the label matrix and its circumscribed matrix is calculated, and the original image is subjected to perspective transformation, so that the matching problem of the rotating label can be better handled. After the label is uniformly transversely and flipped, the template matching is performed by combining the support vector machine classification and the peak signal-to-noise ratio similarity, so that the accuracy and recall rate of the label matching in the real scene are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image matching, and particularly relates to an image label matching method and system fusing OVR-SVM and PSNR similarity. BACKGROUND

[0002] Image template matching refers to finding a similar part of a template image on a test image through comparison between the template image and the test image, and then calculating the similarity of the template image and the target in the test image to achieve the image template matching.

[0003] In the prior art, the similarity measurement between different samples is estimated by calculating the "distance" of the samples, and the distance generally includes Euclidean distance, Manhattan distance, Chebyshev distance, Mahalanobis distance, cosine of angle, Hamming distance, etc. The template matching based on the gray value is the most classic template matching algorithm, which calculates the sum of absolute values (SAD) or the sum of square differences (SSD) of the pixel gray difference between the template image and the detection image. Another commonly used template matching method is to use normalized cross-correlation information to measure the matching probability, which has good illumination invariance. In addition, the shape-based template matching is also a commonly used matching algorithm, which takes the edge gradient correlation of an object as the matching standard, for example, the algorithm for detecting the label in the aprilTag.

[0004] The template matching based on the pixel gray value is greatly affected by the gray value, and is only suitable for stable illumination, and only a few simple image matching is suitable. The correlation-based matching cannot solve the rotation and scale change, and the shape-based template matching has high matching accuracy, but the processing of the image is complex, resulting in long calculation time, which cannot meet the real-time requirement in actual situations. SUMMARY

[0005] The application provides an image label matching method and system fusing OVR-SVM and PSNR similarity to solve the technical problems in the background art.

[0006] The application adopts the following technical scheme to realize the image label matching method fusing OVR-SVM and PSNR similarity, which at least includes the following steps:

[0007] A predetermined number of labels are generated into a template label in a specified form to obtain a label information list, and the label information list at least includes an index number of the template label and a corresponding template label vertex set.

[0008] An input image is processed in real time, the input image with the label is recognized, and the image label matching mode is entered.

[0009] Preprocessing the input image to obtain a reduced version of the binary image, determining a polygon region in the binary image, obtaining each vertex of the polygon region, and generating a vertex list about the binary image;

[0010] Traversing the vertex list to extract an external vertex set, obtaining an external rectangle about the polygon region through the external vertex set, and calculating a perspective transformation matrix based on the external vertex set and the external rectangle; transforming the binary image using the perspective transformation matrix to obtain a transformed image, and cutting out the overlapping part between the binary image and the transformed image to obtain a rectangular region;

[0011] According to the specified form, obtaining the label ID and the rectangular vertex set of the rectangular region, corresponding the vertex coordinates in the rectangular vertex set to the input image to obtain a corresponding actual vertex coordinate set; generating a label region about the input image based on the actual vertex coordinate set, and re-obtaining a test label vertex set for the label region;

[0012] Based on the label ID and the test label vertex set, completing matching in the label information list.

[0013] In further embodiments, the template label is generated as follows:

[0014] Rectangularizing each label to obtain a corresponding rectangular label, obtaining an index number of the rectangular label and a template label vertex set;

[0015] The elements in the template label vertex set include the origin, the long edge end point, the short edge end point, and the diagonal point.

[0016] In further embodiments, it further includes:

[0017] Using an SVM learning algorithm to classify and train the template label, and training an SVM training model;

[0018] When training the SVM training model, a predetermined number of labels are used as positive samples, and a non-predetermined number of labels are used as negative samples.

[0019] In further embodiments, the preprocessing of the input image includes the following procedures:

[0020] Using a contour extraction and quadrilateral fitting method to determine the rectangular region of the label, and based on the size of the label, the label pixel area is screened to obtain a region of interest, and based on the region of interest, the label information is obtained;

[0021] It further includes: reducing the input image according to a predetermined ratio, and using an adaptive threshold processing method to binarize the reduced input image to obtain a binary image.

[0022] In further embodiments, the specific process of transforming the binary image to obtain a transformed image by using a perspective transformation matrix is as follows:

[0023] Define the coordinates of a point in the binary image as (x, y) , then the coordinates of the corresponding point in the transformed image after transformation are (x', y') ; the given perspective transformation matrix is ; wherein k is the serial number of the point in the corresponding image, k is an integer greater than or equal to 2; 、 、 、 、 、 、 、 and are matrix parameters of the perspective transformation matrix;

[0024] Then, (x', y') = (x, y) ) .

[0025] In further embodiments, the label ID of the rectangular region and the acquisition method of the rectangular vertex set are as follows:

[0026] The rectangular region has a horizontal label or a vertical label, the rectangular region with a vertical label is flipped to be a horizontal label, and the rectangular region corresponding to the horizontal label is horizontally flipped, so that each horizontal label corresponds to two horizontal and reversed rectangular regions.

[0027] The SVM learning algorithm is used for classification to match two index numbers and the corresponding template label in the label information list; the peak signal-to-noise ratio similarity of the positive rectangular region and the matched template label is calculated, and the index number corresponding to the positive rectangular region with high similarity is taken as the label ID of the rectangular region, and the template label vertex set adapted to the corresponding index number is the rectangular vertex set.

[0028] In further embodiments, the size of the binary image is defined as 1 / n of the size of the input image, and the values in the actual vertex coordinate set are n times the values in the positive rectangular vertex set.

[0029] In further embodiments, the polygon region and the label region are obtained by contour extraction and polygon fitting.

[0030] In further embodiments, the peak signal-to-noise ratio similarity is calculated by the following formula:

[0031] ;

[0032] wherein, ;

[0033] In the formula, MSE represents the square error of the image pixel value, i, j represents the pixel position; I, K represents the corresponding position pixel value of the image; m, n represents the width and height of the image.

[0034] The image label matching system fusing OVR-SVM and PSNR similarity includes:

[0035] The first module is configured to generate a template label in a specified form according to a predetermined number of labels, so as to obtain a label information list; the label information list at least includes an index number and a template label vertex set corresponding to the template label;

[0036] The second module is configured to perform real-time processing on an input image, identify the input image with a label, and enter an image label matching mode:

[0037] The third module is configured to pre-process the input image to obtain a reduced version of a binary image, determine a polygon region in the binary image, obtain each vertex of the polygon region, and generate a vertex list about the binary image;

[0038] The fourth module is configured to traverse the vertex list to extract an external vertex set, obtain an external rectangle about the polygon region through the external vertex set, calculate a perspective transformation matrix based on the external vertex set and the external rectangle, transform the binary image by using the perspective transformation matrix to obtain a transformed image, and cut out a rectangular region from the overlapping part between the binary image and the transformed image;

[0039] The fifth module is configured to obtain a label ID and a rectangular vertex set of the rectangular region in the specified form, correspondingly enlarge the vertex coordinates in the rectangular vertex set to the input image to obtain a corresponding actual vertex coordinate set, generate a label region about the input image based on the actual vertex coordinate set, and re-obtain a test label vertex set of the label region;

[0040] The sixth module is configured to complete matching in the label information list based on the label ID and the test label vertex set.

[0041] The present application has the beneficial effects that: in the template matching, the image is first reduced, and after the matching is completed, the original image is detected again to accurately position the label, and the real-time performance of the matching process is improved.

[0042] Before matching the binary image, the perspective matrix of the label matrix and the external matrix thereof is calculated, and the original image is perspective transformed, so that the matching problem of the rotated label can be better handled.

[0043] After the labels are aligned horizontally and flipped, the template matching is performed by combining the support vector machine classification and the peak signal-to-noise ratio similarity, so as to improve the accuracy and recall rate of the label matching in a real scene. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 A schematic diagram of a specified form in Example 1.

[0045] Figure 2 A flowchart of label template matching in Example 1.

[0046] Figure 3 A training sample graph in Example 1.

[0047] Figure 4 A label template matching schematic diagram in the prior art. DETAILED DESCRIPTION

[0048] The SVM algorithm is originally designed for binary classification problems, and when dealing with multi-class problems, a suitable multi-class classifier needs to be constructed. Generally, it is divided into OVR-SVM and OVO-SVM. OVR-SVM trains a sample of a certain class as a class, and the remaining samples are classified as another class. In this way, k SVMs are constructed for k samples. When classifying, the unknown sample is classified as the class with the largest classification function value. OVO-SVM designs an SVM between any two classes of samples, so k(k-1) / 2 SVMs are needed for k classes of samples. When classifying an unknown sample, the class with the most votes is the class of the unknown sample. EMBODIMENT

[0049] To this end, the embodiment provides an image label matching method combining OVR-SVM and PSNR similarity, which at least includes the following steps:

[0050] Step one, a predetermined number of labels are generated into template labels according to a specified form, and a label information list is obtained. The label information list at least includes an index number of the template label and a corresponding template label vertex set.

[0051] Step two, the input image is processed in real time to identify the input image with labels, and enter the image label matching mode:

[0052] The input image is preprocessed to obtain a reduced binary image, the polygon region in the binary image is determined, each vertex of the polygon region is obtained, and a vertex list about the binary image is generated.

[0053] Step three, traversing the vertex list to extract the circumscribed vertex set, obtaining the circumscribed rectangle of the polygon region through the circumscribed vertex set, calculating the perspective transformation matrix based on the circumscribed vertex set and the circumscribed rectangle; using the perspective transformation matrix to transform the binary image to obtain a transformed image, and cutting out the overlapping part between the binary image and the transformed image to obtain a rectangular region;

[0054] Step four, obtaining the label ID and the rectangular vertex set of the rectangular region according to the specified form, corresponding the vertex coordinates in the rectangular vertex set to the input image to obtain the corresponding actual vertex coordinate set, and generating the label region of the input image based on the actual vertex coordinate set, and re-obtaining the test label vertex set of the label region;

[0055] Step five, matching the label ID and the test label vertex set in the label information list.

[0056] In an embodiment, the generation of the template label in step one is as follows: obtaining the corresponding rectangular label by rectangularizing each label, obtaining the index number of the rectangular label and the template label vertex set; the elements in the template label vertex set include the origin, the long edge end point, the short edge end point and the diagonal point. In other words, the specified form is the rectangularized label, and each label has four vertices, which are the origin, the long edge end point, the short edge end point and the diagonal point with fixed relative positions. First, one of the four vertices is determined as the origin, then the vertex on the same diagonal line as the origin is the diagonal point, the vertex on the short edge is the short edge end point, and the vertex on the long edge is the long edge end point.

[0057] In another embodiment, it further includes using SVM to perform classification training for the template label, and the positive and negative samples are as shown in Figure 3 The categories of the positive samples are 1-18, and the category of the negative sample is -1, and the SVM model is saved after training.

[0058] In this embodiment, the number of labels is 18, i.e., as shown in Figure 1 SVM is used for classification training for the template label, and the positive and negative samples are as shown in Figure 3 The categories of the positive samples are 1-18, and the category of the negative sample is -1, and the SVM model is saved after training. The system loads the trained SVM model file and 18 template images when starting.

[0059] In a further embodiment, the preprocessing of the input image in step two includes the following processes: determining the rectangular region of the label by using the contour extraction and quadrilateral fitting method, screening the label pixel area corresponding to the size of the label to obtain the region of interest, and obtaining the label information based on the region of interest;

[0060] Further comprising: reducing the input image according to a predetermined scale, and using an adaptive threshold processing method to binarize the reduced input image to obtain a binary image. In this embodiment, the threshold of each pixel point in the adaptive threshold is different, and the threshold is determined by the weighted average of the image pixel points in its domain. In this way, the binarization threshold of the image area with higher brightness will be generally higher, while the binarization threshold of the image area with lower brightness will be correspondingly smaller. Therefore, the image is adaptively binarized by using the properties of the label.

[0061] In further embodiments, the specific process of transforming the binary image to obtain a transformed image using a perspective transformation matrix is as follows:

[0062] Define the coordinates of a point in the binary image as (x, y) , then the coordinates of the corresponding point in the transformed image after transformation are (x', y') ; the given perspective transformation matrix is ; wherein k is the serial number of the point in the corresponding image, k is an integer greater than or equal to 2; ; , , , , , , , and are matrix parameters of the perspective transformation matrix;

[0063] Then, (x', y') = (x, y) ) = (x, y) ) .

[0064] Based on the above formula, ;

[0065] ;

[0066] After sorting, we get: ;

[0067] ;

[0068] Based on a pair of matching points, two constraints are constructed, so the perspective transformation matrix with 8 degrees of freedom is obtained by the following formula. For further example, when k = 4, i.e. 4 coordinates need to be transformed, the transformation process is as follows:

[0069] .

[0070] Correspondingly, the label ID of the rectangular region and the acquisition method of the rectangular vertex set are as follows:

[0071] The rectangular region has a horizontal label or a vertical label, the rectangular region with a vertical label is flipped to be a horizontal label, the rectangular region corresponding to the horizontal label is horizontally flipped, so that each horizontal label corresponds to two horizontal and reversed rectangular regions;

[0072] Two index numbers and corresponding template labels are matched in the label information list using the SVM learning algorithm, and the index number of the positive rectangular region corresponding to the high similarity is taken as the label ID of the rectangular region by calculating the peak signal-to-noise ratio similarity of the positive rectangular region and the matched template label.

[0073] For example, the rectangular region inside is a rectangular label image obtained through the previous processing, and the rectangular region after the perspective transformation processing can be a vertical rectangle (the width is greater than the length) or a horizontal rectangle (the width is less than the length). However, the svm classification and the psnr similarity calculation need to be consistent with the size of the template, and the template used is horizontal (the width is less than the length), so the extracted rectangular region needs to be converted to horizontal first. However, the horizontal rectangular region has two rotation directions, which can be understood as the vertical rectangular region being rotated to the horizontal direction, which can be rotated to the left or to the right, so the psnr method needs to be used to screen out the correct match.

[0074] The peak signal-to-noise ratio similarity is calculated by the following formula:

[0075] ;

[0076] Wherein, ;

[0077] In the formula, MSE represents the square error of the image pixel value, i, j represents the pixel position; I, K represents the corresponding pixel value of the image; m, n represents the width and height of the image.

[0078] It should be noted that in the label classification, the support vector machine can also be replaced by a general traditional classifier, such as KNN, Naive Bayes classification, random forest, Adaboost, etc. In the PSNR similarity screening, cosine distance, Hamming distance, MSE, SSIM, etc. can also be used for measurement.

[0079] It should be noted that in the present embodiment, the elements in the circumscribed vertex set, the positive rectangular vertex set, the actual vertex coordinate set and the test label vertex set are the origin, the long edge end point, the short edge end point and the diagonal point, but the numerical values are different. The accuracy is improved by changing the numerical values.

[0080] In summary, the application uses the properties of the label to adaptively binarize the image, effectively reducing the influence of light on the pixel gray value; then the extracted label contour and the template are used for size scaling and perspective transformation, reducing the influence of rotation and scale; finally, all labels are uniformly processed, and support vector machines and PSNR similarity metrics are used for classification and screening to obtain more robust classification results. Embodiments

[0081] The embodiment discloses an image label matching system combining OVR-SVM and PSNR similarity, which is used to implement the image label matching method in embodiment 1, and comprises:

[0082] The first module is configured to generate a template label in a specified form according to a predetermined number of labels, and obtain a label information list; the label information list at least includes an index number and a template label vertex set corresponding to the template label;

[0083] The second module is configured to process the input image in real time, identify the input image with the label, and enter the image label matching mode:

[0084] The third module is configured to pre-process the input image to obtain a reduced version of the binary image, determine a polygon region in the binary image, obtain each vertex of the polygon region, and generate a vertex list about the binary image;

[0085] The fourth module is configured to traverse the vertex list to extract an external vertex set, obtain an external rectangle about the polygon region through the external vertex set, calculate a perspective transformation matrix based on the external vertex set and the external rectangle, transform the binary image through the perspective transformation matrix to obtain a transformed image, and cut out a rectangular region from the overlapping part between the binary image and the transformed image;

[0086] The fifth module is configured to obtain a label ID and a rectangular vertex set of the rectangular region in the specified form, correspondingly enlarge the vertex coordinates in the rectangular vertex set to the input image to obtain a corresponding actual vertex coordinate set, generate a label region about the input image based on the actual vertex coordinate set, and re-obtain a test label vertex set of the label region;

[0087] The sixth module is configured to complete the matching in the label information list based on the label ID and the test label vertex set.

Claims

1. An image label matching method fusing OVR-SVM and PSNR similarity, characterized in that, At least comprising the following steps: A predetermined number of labels are generated into a template label according to a specified form to obtain a label information list; The label information list at least includes: index number of the template label and corresponding template label vertex set; The input image is processed in real time to identify the input image with the label and enter the image label matching mode: The input image is preprocessed to obtain a reduced version of the binary image, the polygon region in the binary image is determined, each vertex of the polygon region is obtained, and the vertex list about the binary image is generated; The vertex list is traversed to extract the circumscribed vertex set, the circumscribed rectangle about the polygon region is obtained through the circumscribed vertex set, and the perspective transformation matrix is calculated based on the circumscribed vertex set and the circumscribed rectangle; the binary image is transformed by using the perspective transformation matrix to obtain a transformed image, and the overlapping part between the binary image and the transformed image is cut out to obtain a rectangular region; The rectangular region has a horizontal label or a vertical label, the rectangular region with a vertical label is flipped to be uniform as a horizontal label, and the rectangular region corresponding to the horizontal label is horizontally flipped, so that each horizontal label corresponds to two horizontal and reverse rectangular regions; The SVM learning algorithm is used for classification, and two index numbers and corresponding template labels are matched in the label information list; the similarity of the peak signal-to-noise ratio between the rectangular region and the matched template label is calculated, and the index number corresponding to the rectangular region with high similarity is taken as the label ID of the rectangular region; the template label vertex set corresponding to the index number is adapted to the rectangular vertex set; Based on the label ID and the template label vertex set, the matching in the label information list is completed. 2.The image tag matching method of fusing OVR-SVM and PSNR similarity according to claim 1, characterized in that, The generation mode of the template label is as follows: Each label is rectangularized to obtain a corresponding rectangular label, and the index number of the rectangular label and the template label vertex set are obtained; The elements in the template label vertex set include the origin, the long edge end point, the short edge end point and the diagonal point. 3.The image tag matching method of fusing OVR-SVM and PSNR similarity according to claim 1, characterized in that, Further comprising: The SVM learning algorithm is used for classification training of the template label, and an SVM training model is trained; During SVM training model, a predetermined number of labels are used as positive samples, and a non-predetermined number of labels are used as negative samples.

4. The image tag matching method of claim 1, wherein, The preprocessing of the input image includes the following processes: The contour extraction and quadrilateral fitting method is used to determine the rectangular region of the label, the label pixel area corresponding to the size of the label is screened to obtain the region of interest, and the label information is obtained based on the region of interest; Further comprising: the input image is reduced according to a predetermined ratio, and the reduced input image is binarized by using an adaptive threshold processing method to obtain a binary image.

5. The image tag matching method of claim 1, wherein, The specific process of transforming the binary image into a transformed image by using the perspective transformation matrix is as follows: The coordinates of a point in a binary image are defined as (x, y) , and the coordinates of the corresponding point in the transformed image are (x', y') ; the given perspective transformation matrix is ; wherein k is the serial number of the point in the corresponding image, and k is an integer greater than or equal to 2; , , , , , , , , and are matrix parameters of the perspective transformation matrix. but,( ) = ( ) .

6. The image tag matching method of claim 1, wherein, The size of the binary image is defined as 1 / n of the size of the input image, and the value in the actual vertex coordinate set is n times the value in the positive rectangular vertex set.

7. The image tag matching method of claim 1, wherein, The polygon region and the label region are obtained by contour extraction and polygon fitting.

8. An image tag matching system fusing OVR-SVM and PSNR similarity, characterized in that, Comprising: A first module is configured to generate a template label from a predetermined number of labels according to a specified form to obtain a label information list; The label information list at least includes: index number of corresponding template label and template label vertex set; The second module is configured to process the input image in real time, identify the input image with label, and enter the image label matching mode; The third module is configured to pre-process the input image to obtain a reduced binary image, determine a polygon region in the binary image, obtain each vertex of the polygon region, and generate a vertex list about the binary image; The fourth module is configured to traverse the vertex list to extract an external vertex set, obtain an external rectangle about the polygon region through the external vertex set, calculate a perspective transformation matrix based on the external vertex set and the external rectangle, transform the binary image through the perspective transformation matrix to obtain a transformed image, and cut out an overlapping part between the binary image and the transformed image to obtain a rectangular region; The fifth module is configured to obtain a label ID and a normal rectangle vertex set of the rectangular region according to the specified form, correspondingly enlarge the vertex coordinates in the normal rectangle vertex set to the input image to obtain a corresponding actual vertex coordinate set, generate a label region about the input image based on the actual vertex coordinate set, and re-obtain a template label vertex set of the label region; The rectangular region has a horizontal label or a vertical label, the rectangular region with the vertical label is flipped to be uniform as the horizontal label, and the rectangular region corresponding to the horizontal label is horizontally flipped, so that each horizontal label corresponds to two horizontal and reverse rectangular regions; The SVM learning algorithm is used for classification, two index numbers and corresponding template labels are matched in the label information list, the peak signal-to-noise ratio similarity of the rectangular region and the matched template label is calculated, the index number corresponding to the rectangular region with high similarity is taken as the label ID of the rectangular region, and the template label vertex set corresponding to the index number is adapted as the rectangular vertex set; The sixth module is configured to complete matching in the label information list based on the label ID and the template label vertex set.

Images