Distortion measurement method and system in image matching and image matching method and device
By calculating the homography matrix and setting the distortion interval, the degree of image distortion is evaluated, and the recognition failure caused by the inspection image is solved, and the accuracy and reliability of image matching are improved.
Patent Information
- Application Number
- CN202410879422.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-11-08
AI Technical Summary
The distortion of the inspection image due to camera viewing angle offset or environmental changes during the substation inspection process leads to deviation of the homography matrix calculation or too few feature points, which in turn causes the target matching and positioning failure, affecting the image recognition effect.
By calculating the homography matrix of the image to be matched and the template image, setting the distortion interval of each parameter, determining whether the image has distortion, and evaluating the degree of distortion based on the distortion metric value to avoid target positioning matching failure caused by distortion.
Effectively judge image distortion, adjust the camera in time or remade template images to avoid failure in patrol image recognition, and improve the accuracy and reliability of image matching recognition.
Smart Images

Figure CN118918451A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image recognition, and in particular relates to a distortion measurement method and system in image matching, and an image matching method and device. Background Art
[0002] Substation inspection image target matching and positioning algorithm is widely used in most intelligent inspection image matching and recognition scenarios such as meters, GIS switches, and switch pressure plate indicator lights due to its good portability and operability. For example, the meter inspection image matching and recognition process is roughly divided into 6 steps: Step 1: Model the inspection points to obtain the dial model (template) image and record key information; Step 2: Extract features from the captured image to be matched and the model image at the same time to obtain two sets of feature point sets; Step 3: Match the two sets of feature point sets to calculate the homography matrix; Step 4: Calculate the coordinates of the four vertices of the model image under the above homography transformation, and complete the target matching and positioning; Step 5: Extract the dial area in the image to be matched according to the size of the model image, and complete the dial area matching and extraction in the inspection image; Step 6: Use the key information recorded in the model image to realize meter recognition. The most important step in the existing image matching method is the matching and positioning of the target, but the inspection images collected at the inspection points are not fixed and may be offset in perspective or even severely distorted due to the camera and various unknown reasons. For example, the camera used to collect inspection images at the inspection point may be offset due to the environment or a long installation time, causing the captured inspection image to be distorted, thereby resulting in deviations in the calculation of the homography matrix, or too few feature points leading to failure in the calculation of the homography matrix, resulting in failure in target matching and positioning, and then failure in inspection point recognition. Summary of the invention
[0003] The purpose of the present invention is to provide a distortion measurement method and system in image matching, an image matching method and device, so as to solve the problem of inspection point recognition failure caused by image distortion in inspection image recognition.
[0004] In order to solve the above technical problems, the present invention provides a distortion measurement method in image matching, comprising:
[0005] Calculate the homography matrix according to the image to be matched and the template image;
[0006] Whether the image to be matched is distorted is determined according to the linear parameters and projection parameters in the homography matrix. When the parameters of the homography matrix are in the corresponding distortion interval, it is determined that the image to be matched is distorted.
[0007] Furthermore, the distortion interval is set according to the transformation type: for projection transformation, the distortion interval is set to: |m 31 |>first threshold or |m32 |>First threshold; for flip transformation, the distortion interval is set to: m 11 < the second threshold or m 22 <Second threshold; for scaling transformation, the distortion interval is set to: m 11 <Third threshold, m 11 >The fourth threshold, m 22 <The third threshold or m 22 >Fourth threshold; for compression-stretch transformation, the distortion interval is set to: |m 11 -m 22 |>Fourth threshold; for affine transformation, the distortion interval is set to: |m 12 |>Third threshold or |m 21 |>Third threshold; where m 11 、m 12 、m 21 and m 22 is the linear parameter of the homography matrix, m 31 、m 32 are projection parameters of the homography matrix; when the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted; the value interval of the first threshold is [0.0005, 0.001], the second threshold is 0, the value interval of the third threshold is [0.4, 0.6], and the value interval of the fourth threshold is [1.5, 2].
[0008] Furthermore, the distortion intervals corresponding to the parameters of the homography matrix include: 31 |>First threshold, |m 32 |>First threshold, m 11 <Third threshold, m 11 >The fourth threshold, m 22 <Third threshold, m 22 >The fourth threshold, |m 12 |>Third threshold and |m 21 |>Third threshold; where m 11 、m 12 、m 21 and m 22 is the linear parameter of the homography matrix, m 31 、m 32 are projection parameters of the homography matrix; when the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted; the value interval of the first threshold is [0.0005, 0.001], the value interval of the third threshold is [0.4, 0.6], and the value interval of the fourth threshold is [1.5, 2].
[0009] Furthermore, the more parameters of the homography matrix are in the distortion interval, the more serious the distortion degree is evaluated to be.
[0010] Furthermore, the distortion measurement value is calculated according to the distortion measurement formula, and the distortion measurement formula is:
[0011] d′(M)=f(m 31 |×10 3 )+f(m 32 |×10 3 )+f(m 12 |×2)+
[0012] f(m 21 |×2)+f(m 11 ×4 / 3-5 / 3)+f(m 22 ×4 / 3-5 / 3);
[0013]
[0014] Among them, d′(M) is the distortion measure corresponding to the homography matrix, f(x) is the distortion function, and x is the parameter of the homography matrix.
[0015] Furthermore, the distortion degree evaluation is determined according to the magnitude of the distortion metric value. The larger the distortion metric value is, the more severe the corresponding distortion degree evaluation is.
[0016] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention, which analyzes the parameters in the homography matrix, sets the distortion interval corresponding to each parameter, and defines that the image to be matched is distorted when the parameters are in the corresponding distortion interval; in actual use, after calculating the homography matrix according to the image to be matched and the template image, it is possible to directly judge whether the image to be matched is distorted based on whether the parameters of the homography matrix are in the corresponding distortion interval, so that the staff can obtain the inspection image quality of the inspection point in time, and when distortion occurs, the camera can be adjusted in time or the template image can be re-produced to avoid the failure of target positioning and matching caused by the distortion of the inspection image, which in turn leads to the failure of inspection point recognition.
[0017] In order to solve the above technical problem, the present invention also provides a distortion measurement system in image matching, including a processor, and the processor implements the distortion measurement method in image matching introduced above.
[0018] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention, which analyzes the parameters in the homography matrix, sets the distortion interval corresponding to each parameter, and defines that the image to be matched is distorted when the parameter is in the corresponding distortion interval; in actual use, after calculating the homography matrix according to the image to be matched and the template image, the system can directly judge whether the image to be matched is distorted according to whether the parameters of the homography matrix are in the corresponding distortion interval, so that the staff can obtain the inspection image quality of the inspection point in time, and when distortion occurs, they can adjust the camera or re-produce the template image in time to avoid the failure of target positioning and matching caused by the distortion of the inspection image, and then lead to the failure of inspection point recognition.
[0019] In order to solve the above-mentioned technical problems, the present invention also provides an image matching method, which uses the distortion measurement method in image matching introduced above to determine whether the image to be matched is distorted. When no distortion occurs, matching and identification are performed according to the homography matrix; when distortion occurs, it is prompted that the image to be matched is distorted, and subsequent matching and identification are no longer performed.
[0020] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention, which first uses a distortion measurement method in image matching to determine whether the image to be matched is distorted. When distortion occurs, it is prompted that the image to be matched is distorted, and subsequent matching and recognition are no longer performed. This can avoid the failure of target positioning and matching caused by the distortion of the inspection image, which in turn leads to the failure of inspection point recognition, thereby improving the quality of inspection image recognition.
[0021] To solve the above technical problem, the present invention further provides an image matching device, including a controller, wherein the controller is used to implement the above-mentioned image matching method.
[0022] The beneficial effects of the above technical solution are as follows: the present invention is an improved invention, in which the controller first uses a distortion measurement method in image matching to determine whether the inspection image taken by the camera to be identified is distorted. When distortion occurs, it prompts that the image to be matched is distorted, and no subsequent matching and identification is performed. This can avoid the failure of target positioning and matching caused by the distortion of the inspection image, which in turn leads to the failure of inspection point identification, thereby improving the quality of inspection image recognition. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 is a distortion measurement flow chart of an embodiment of a distortion measurement method in image matching of the present invention;
[0024] Figure 2 It is a distortion interval diagram of homography matrix parameters in an embodiment of the distortion measurement method in image matching of the present invention;
[0025] Figure 3is another distortion interval diagram of homography matrix parameters in an embodiment of the distortion measurement method in image matching of the present invention;
[0026] Figure 4 is the modeling reference diagram of Verification Example 3 of the present invention;
[0027] Figure 5 is the dial reference diagram of Verification Example 3 of the present invention;
[0028] Figure 6 is the inspection diagram to be identified in Verification Example 3 of the present invention;
[0029] Figure 7 It is a dial area map obtained by matching and identifying Verification Example 3 of the present invention. DETAILED DESCRIPTION
[0030] In order to make the objectives, technical solutions and advantages of the present invention more clear, the specific implementation modes of the present invention are further described below in conjunction with the accompanying drawings.
[0031] The present invention analyzes the parameters in the homography matrix obtained by target matching, sets the distortion interval corresponding to each parameter, and defines the distortion of the image to be matched when the parameter is in the corresponding distortion interval; in actual use, after calculating the homography matrix according to the image to be matched and the template image, it can directly judge whether the image to be matched is distorted according to whether the parameters of the homography matrix are in the corresponding distortion interval, thereby evaluating whether the inspection image meets the algorithm recognition requirements, and giving the image quality evaluation result for the inspection image that does not meet the requirements. This allows the staff to obtain the inspection image quality of the inspection point in time, and when distortion occurs, they can adjust the camera or re-make the template image in time to avoid the inspection image distortion causing the target positioning and matching positioning failure, and then leading to the inspection point recognition failure.
[0032] Distortion measurement method embodiment in image matching
[0033] Aiming at the key link of substation inspection image target matching and positioning, the present invention provides a distortion measurement method in image matching. The method first proposes a distortion measurement definition based on the homography matrix obtained by the target matching algorithm; then gives a distortion measurement calculation method; finally, the inspection image quality evaluation result is given according to the distortion measurement value. The implementation principle is as follows: Figure 1 As shown, the detailed description is given below.
[0034] 1. Calculate the homography matrix based on the image to be matched and the template image.
[0035] Various existing inspection target matching algorithms will calculate the homography matrix. The matching algorithm performs homography transformation on the inspection image according to the homography matrix to achieve target matching. Therefore, the present invention first calculates the homography matrix of the image to be matched and the template image. Specifically, the inspection points are modeled to obtain the modeling reference map of the inspection points. The dial model image is extracted based on the modeling reference map and key information is recorded; the captured image to be matched and the model image are simultaneously subjected to feature extraction to obtain two sets of feature point sets; the two sets of feature point sets are matched using the target matching algorithm to calculate the homography matrix. The homography transformation is also called the projection transformation. The projection transformation has 2 more degrees of freedom than the affine transformation and has 8 degrees of freedom. During the projection transformation, the 3 points that are kept collinear in the original image are still collinear after the transformation. The homography matrix is used to describe the transformation relationship between corresponding points on two planes:
[0036]
[0037] Among them, x′1 is the x-axis coordinate after transformation, x′2 is the y-axis coordinate after transformation, x1 is the x-axis coordinate before transformation, x2 is the y-axis coordinate before transformation, m 11 、m 12 、m 21 and m 22 is the linear parameter of the homography matrix, m 13 、m 23 is the translation parameter of the homography matrix, m 31 、m 32 is the projection parameter of the homography matrix, and m is normalized in the homogeneous coordinate system. 33 Usually it is 1.
[0038] 2. Set the distortion measurement interval.
[0039] Considering the impact of severe image distortion on recognition results, the present invention proposes a distortion measurement definition. Although generally speaking, stretching, flipping, scaling and affine transformations are non-deformative, and only projection transformations change the shape of objects, based on a large amount of engineering experience, the present invention still considers situations such as excessive rotation angles, excessive scaling and stretching scales, excessive affine deformations, and flipping when measuring the distortion of the inspection target area. The distortion interval is set according to the transformation type, and the image to be matched is distorted when the definition parameters are in the corresponding distortion interval. It is specifically divided into the following five situations:
[0040] Case 1: For projection transformation, the distortion interval is set to: |m 31 |>first threshold or |m 32 |>The first threshold, the value interval of the first threshold is [0.0005, 0.001];
[0041] Case 2: For flip transformation, the distortion interval is set to: m11 < first threshold or m 22 < first threshold, the second threshold is 0;
[0042] Case 3: For scaling transformation, the distortion interval is set to: m 11 <The third threshold or m 11 > the fourth threshold or m 22 <The third threshold or m 22 >Fourth threshold, the value interval of the third threshold is [0.4, 0.6], and the value interval of the fourth threshold is [1.5, 2];
[0043] Case 4: For compression-stretch transformation, the distortion interval is set to: |m 11 -m 22 |>Fourth threshold;
[0044] Case 5: For affine transformation, the distortion interval is set to: |m 12 |>Third threshold or |m 21 |>Third threshold.
[0045] When the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted.
[0046] In a specific embodiment, the first threshold is 0.001, the second threshold is 0, the third threshold is 0.5, the fourth threshold is 0, and the distortion interval is as follows: Figure 2 As shown:
[0047] Case 1: For projection transformation, the distortion interval is set to: |m 31 |>10 -3 or |m 32 |>10 -3 ;
[0048] Case 2: For flip transformation, the distortion interval is set to: m 11 <0 or m 22 <0;
[0049] Case 3: For scaling transformation, the distortion interval is set to: m 11 <0.5 or m 11 >2 or m 22 <0.5 or m 22 >2;
[0050] Case 4: For compression-stretch transformation, the distortion interval is set to: |m 11 -m 22 |>2;
[0051] Case 5: For affine transformation, the distortion interval is set to: |m 12 |>0.5 or |m 21 |>0.5;
[0052] pass Figure 2 From the analysis of the interval graph, we can see that m 11 <0, then it must be m 11 <Third threshold, similarly m 22 <0, then it must be m 22 < the third threshold, when m 11 、m 22 When both are not in the distortion range of case 3, |m 11 -m 22 |≤ the fourth threshold, that is, in case 4, m is judged 11 、m 22 This is also normal. Therefore, in another embodiment of the present invention, the distortion interval is set only for situations 1, 3, and 5. The distortion interval corresponding to each parameter is:
[0053] For parameter m 31 、m 32 , the distortion interval is set to: |m 31 |>First threshold, |m 32 |>first threshold;
[0054] For parameter m 11 、m 22 , the distortion interval is set to: m 11 <The third threshold or m 11 >The fourth threshold, m 22 <The third threshold or m 22 > fourth threshold;
[0055] For parameter m 12 、m 21 , the distortion interval is set to: |m 12 |>Third threshold, |m 21 |>Third threshold.
[0056] According to the threshold setting in the above specific embodiment, the corresponding distortion interval is as follows: Figure 3 As shown, including:
[0057] For parameter m 31 、m 32 , the distortion interval is set to: |m 31 |>10 -3 、|m 32 |>10 -3 ;
[0058] For parameter m 11 、m 22 , the distortion interval is set to: m 11 <0.5 or m 11 >2, m 22<0.5 or m 22 >2;
[0059] For parameter m 12 、m 21 , the distortion interval is set to: |m 12 |>0.5,|m 21 |>0.5.
[0060] When the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted.
[0061] The more parameters of the homography matrix are in the distortion interval, the more serious the distortion is evaluated.
[0062] 3. Calculate the distortion measurement value according to the distortion measurement formula.
[0063] According to the distortion interval corresponding to each parameter, the corresponding distortion function is set, and then the distortion functions are added to obtain the distortion metric. As a preferred implementation, in order to facilitate the output of the distortion metric evaluation result, when the parameter x of the homography matrix is not in the corresponding distortion interval, the distortion function f(x) value corresponding to the distortion interval is 0, and when the parameter x of the homography matrix is in the corresponding distortion interval, the distortion function f(x) value corresponding to the distortion interval is 1. In order to further improve the calculation efficiency, each parameter interval is normalized. For the image to be matched image and the template image normal, as well as the homography matrix M between the two, for the specific setting of the above threshold, the optimized distortion metric formula is as follows:
[0064]
[0065] Among them, d′(M) is the distortion metric value corresponding to the homography matrix, d′ is the distortion metric, and f(x) represents the distortion function.
[0066] 4. Determine the distortion evaluation according to the calculated distortion metric value. The larger the distortion metric value, the more severe the corresponding distortion evaluation. In this embodiment, the distortion metric is set to 7 levels from 0 to 6, and the inspection image quality evaluation result is given according to the distortion metric value. When the distortion metric value is 0, the distortion evaluation is no; when the distortion metric value is 1-2, the distortion evaluation is general; when the distortion metric value is 3-4, the distortion evaluation is severe; when the distortion metric value is 5-6, the distortion evaluation is very severe. The specific level classification is shown in Table 1:
[0067] Table 1
[0068]
[0069]
[0070] The image quality evaluation results are sent to the staff, who can then obtain the inspection image quality of the inspection points in a timely manner. When distortion occurs or the distortion is severe, the inspection image recognition system is controlled to stop subsequent matching and recognition. The staff can also adjust the camera or re-make the template image in time to avoid distortion of the inspection image causing target positioning and matching failure, which in turn leads to inspection point recognition failure, thereby improving the quality of inspection image matching and recognition.
[0071] Distortion measurement system embodiment in image matching
[0072] The present invention provides a distortion measurement system in image matching, including a processor, and the processor is used to implement a distortion measurement method in image matching. The method is the distortion measurement method in image matching described in the above-mentioned distortion measurement method embodiment in image matching, which will not be described in detail here.
[0073] Image matching method embodiment
[0074] The present invention provides an image matching method, which uses a distortion measurement method in image matching to determine whether a to-be-matched image is distorted. When no distortion occurs, matching and identification are performed according to a homography matrix; when distortion occurs, it is prompted that the to-be-matched image is distorted, and subsequent matching and identification are no longer performed. This method is the distortion measurement method in image matching described in the embodiment of the distortion measurement method in image matching, and will not be described in detail here.
[0075] Image matching device embodiment
[0076] The present invention provides an image matching device, including a controller, wherein the controller is used to implement an image matching method, which is the image matching method described in the above-mentioned image matching method embodiment and will not be described in detail here.
[0077] The distortion measurement method in image matching of the present invention is tested through three verification examples.
[0078] Verification Example 1
[0079] The dial template image is obtained from the modeling reference image, and the photographed inspection image to be identified is matched. The actual meter reading of the obtained dial area is 0.446, and the meter reading obtained by inspection image recognition is also 0.446, indicating that the image is not distorted in this scene. The homography matrix M calculated in this scene is:
[0080]
[0081] Substituting the optimized distortion measurement formula into the calculated distortion measurement value is 0, indicating that the image is not distorted in this scene.
[0082] Verification Example 2
[0083] The dial template image is obtained from the modeling reference image, and the inspection image to be identified is matched with the dial template image to obtain the identified dial area. The actual meter reading is 0.442, and the meter reading obtained by inspection image recognition is also 0.442, indicating that the image is not distorted in this scene. The homography matrix M calculated in this scene is:
[0084]
[0085] Substituting the optimized distortion measurement formula into the calculated distortion measurement value is 0, indicating that the image is not distorted in this scene.
[0086] Verification Example 3
[0087] like Figure 4 and Figure 5 As shown, the dial template image is obtained from the modeling reference image, such as Figure 6 and Figure 7 As shown in the figure, the inspection image to be identified is matched with the dial template image to obtain the identified dial area. The actual meter reading is 4.973, and the meter reading obtained by inspection image recognition is 1.442, indicating that the image is distorted in this scene. The homography matrix M calculated in this scene is:
[0088]
[0089] Substituting the optimized distortion measurement formula, the calculated distortion measurement value is 5, indicating that the image distortion in this scene is very serious and is no longer suitable for subsequent target matching and recognition, affecting the accuracy of meter recognition.
Claims
1. A distortion measurement method in image matching, characterized in that: include: Calculate the homography matrix according to the image to be matched and the template image; Whether the image to be matched is distorted is determined according to the linear parameters and projection parameters in the homography matrix. When the parameters of the homography matrix are in the corresponding distortion interval, it is determined that the image to be matched is distorted.
2. The distortion measurement method in image matching according to claim 1, characterized in that: The distortion interval is set according to the transformation type: for projection transformation, the distortion interval is set to: |m 31 |>first threshold or |m 32 |>First threshold; for flip transformation, the distortion interval is set to: m 11 < the second threshold or m 22 <Second threshold; for scaling transformation, the distortion interval is set to: m 11 <Third threshold, m 11 >The fourth threshold, m 22 <The third threshold or m 22 >Fourth threshold; for compression-stretch transformation, the distortion interval is set to: |m 11 -m 22 |>Fourth threshold; for affine transformation, the distortion interval is set to: |m 12 |>Third threshold or |m 21 |>Third threshold; where m 11 、m 12 、m 21 and m 22 is the linear parameter of the homography matrix, m 31 、m 32 are projection parameters of the homography matrix; when the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted; the value interval of the first threshold is [0.0005, 0.001], the second threshold is 0, the value interval of the third threshold is [0.4, 0.6], and the value interval of the fourth threshold is [1.5, 2].
3. The distortion measurement method in image matching according to claim 1, characterized in that: The distortion intervals corresponding to the parameters of the homography matrix include: 31 |>First threshold, |m 32 |>First threshold, m 11 <Third threshold, m 11 >The fourth threshold, m 22 <Third threshold, m 22 >The fourth threshold, |m 12 |>Third threshold and |m 21 |>Third threshold; where m 11 、m 12 、m 21 and m 22 is the linear parameter of the homography matrix, m 31 、m 32 are projection parameters of the homography matrix; when the parameters of the homography matrix are in at least one distortion interval, it is determined that the image to be matched is distorted; the value interval of the first threshold is [0.0005, 0.001], the value interval of the third threshold is [0.4, 0.6], and the value interval of the fourth threshold is [1.5, 2].
4. The distortion measurement method in image matching according to claim 3, characterized in that: The more parameters of the homography matrix are in the distortion interval, the more serious the distortion is evaluated.
5. The distortion measurement method in image matching according to claim 3, characterized in that: The distortion measurement value is calculated according to the distortion measurement formula, and the distortion measurement formula is: d′(M)=f(m 31 |×10 3 )+f(m 32 |×10 3 )+f(m 12 |×2)+ f(m 21 |×2)+f(m 11 ×4 / 3-5 / 3)+f(m 22 ×4 / 3-5 / 3); Among them, d′(M) is the distortion measure corresponding to the homography matrix, f(x) is the distortion function, and x is the parameter of the homography matrix.
6. The distortion measurement method in image matching according to claim 5, characterized in that: The distortion degree evaluation is determined according to the size of the distortion metric value. The larger the distortion metric value, the more severe the corresponding distortion degree evaluation.
7. A distortion measurement system in image matching, characterized in that: The method comprises a processor, wherein the processor implements the distortion measurement method in image matching as described in any one of claims 1 to 6.
8. An image matching method, characterized in that: A distortion measurement method in image matching as described in any one of claims 1 to 6 is used to determine whether the image to be matched is distorted. When no distortion occurs, matching and identification are performed according to the homography matrix. When distortion occurs, it is prompted that the image to be matched is distorted, and subsequent matching and identification are no longer performed.
9. An image matching device, comprising a controller, characterized in that: The controller is used to implement the image matching method described in claim 8.
