Method for eliminating image distortion of pointer-type instrument based on calibration plate

Abstract

The present invention discloses a method for eliminating image distortion of a pointer-type instrument based on a calibration plate, which at least includes the following steps: S1, designing a pointer-type instrument with a micro-calibration plate: attaching the micro-calibration plate to a blank area of ​​an instrument panel so that the micro-calibration plate and the instrument panel are in the same plane; S2, eliminating distortion of an instrument distortion image: photographing to obtain an instrument distortion image, calculating a distortion transformation function through constraint corresponding points of image transformation, and transforming the instrument distortion image to obtain an instrument correction image; S3, determining a pointer deflection angle: segmenting a pointer binary image in the instrument correction image, fitting a pointer center line, and calculating a deflection angle of the pointer center line. The method can eliminate distortion of an instrument image to a certain extent and improve the accuracy of reading recognition of a pointer-type instrument.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, and in particular to a method for eliminating image distortion of a pointer-type instrument based on a calibration plate. Background Art

[0002] Pointer meters are simple in structure and low in cost. They are commonly used in the power, chemical, petroleum and other industries. There are many types of them, such as thermometers, ammeters, voltmeters and pressure gauges. Manual meter reading is prone to visual deviations or human errors, and the detection efficiency is low, making it difficult to adapt to high-frequency and high-precision meter reading requirements. In order to solve this problem and realize the automation and intelligence of reading recognition, some image-based automatic recognition algorithms for pointer meter readings have been proposed and widely used. These methods install a camera in front of the meter, transmit the real-time collected meter image back to the background server, and use machine vision image processing and other means to process the acquired image to automatically identify the meter reading.

[0003] When the camera is collecting instrument images, the imaging plane of the camera cannot be completely parallel to the plane of the instrument panel. The distortion of the lens itself, or the camera shooting angle offset from the front of the instrument panel due to factors such as equipment migration and deployment, external force vibration, etc., will cause distortion of the instrument image. In the process of automatic recognition of instrument readings, even tiny distortions cannot adapt to high-precision and high-sensitivity application scenarios. Therefore, before performing instrument reading recognition, the instrument image needs to be distorted to accurately obtain the pointer deflection angle and improve the accuracy of instrument reading recognition. Summary of the invention

[0004] In order to overcome the deficiencies of the above technologies, the present invention provides a method for eliminating image distortion of a pointer-type instrument based on a calibration plate.

[0005] The technical solution adopted by the present invention to overcome the technical problems is:

[0006] A method for eliminating image distortion of a pointer-type instrument based on a calibration plate comprises at least the following steps:

[0007] S1. Design a pointer instrument with a micro-calibration plate: attach the micro-calibration plate to the blank area of ​​the instrument panel so that the micro-calibration plate and the instrument panel are in the same plane;

[0008] S2. Eliminate distortion of the instrument distortion image: photograph to obtain the instrument distortion image, calculate the distortion transformation function through the constraint corresponding points of the image transformation, and transform the instrument distortion image to obtain the instrument correction image;

[0009] S3. Determine the deflection angle of the pointer: segment the pointer binary image in the instrument correction image, fit the pointer center line, and calculate the deflection angle of the pointer center line.

[0010] Furthermore, in step S1, designing an instrument with a micro-calibration plate specifically includes:

[0011] S11, assuming that the substrate of the micro-calibration plate includes s×t characteristic elements and are arranged in an array on the substrate in an equidistant manner, wherein s represents a row, t represents a column, s≥2, t≥2, and the characteristic elements are square spots or circular spots of the same size;

[0012] S12, attaching the micro-calibration plate to a blank area of ​​the instrument panel so that the micro-calibration plate and the instrument panel are in the same plane, thereby obtaining a pointer-type instrument with the micro-calibration plate.

[0013] Furthermore, in step S12, the size of the micro calibration plate is 1 / 4 to 3 / 4 of the blank area of ​​the instrument panel.

[0014] Furthermore, in step S2, eliminating the distortion of the instrument distortion image specifically includes:

[0015] S21, for the pointer instrument with the micro-calibration plate obtained in step S1, photograph the instrument panel including the micro-calibration plate with forward vision, and ensure that the camera is parallel to the instrument panel during the photographing, to obtain an instrument distortion image;

[0016] S22, using an image processing corner point detection algorithm to extract the measured coordinates of the feature points of the micro-calibration plate in the instrument distortion image;

[0017] S23, according to the image resolution requirements of the pointer instrument, setting the ideal coordinates corresponding to each feature point of the micro-calibration plate;

[0018] S24, using the measured coordinates of the feature points and the ideal coordinates to form the constraint corresponding points of the image transformation, and calculating the distortion transformation function through n groups of constraint corresponding points, where n≥4;

[0019] S25. Correct the pixel spatial position of the instrument distortion image using the distortion transformation function to eliminate the distortion of the instrument distortion image and obtain the instrument correction image.

[0020] Further, in step S22, the characteristic point of the micro-calibration plate is the center of the circular spot or the corner point of the square spot.

[0021] Further, step S24 specifically includes:

[0022] The distortion transformation function to be estimated is a model based on bilinear approximation, which is given by the following two equations:

[0023] x'=k1x+k2y+k3xy+k4=[x,y,xy,1]×[k1,k2,k3,k4] T

[0024] y'=k5x+k6y+k7xy+k8=[x,y,xy,1]×[k5,k6,k7,k8] T

[0025] (x, y) and (x', y') constitute a set of constraint corresponding points, representing the measured coordinates and the ideal coordinates respectively. All the unknown coefficients k1, k2, k3, k4, k5, k6, k7, k8 of the above equation are solved through n sets of constraint corresponding points;

[0026] Assume that there are n sets of constraint corresponding points, (x1, y1), (x2, y2), ..., (x n ,y n ) and (x'1,y'1), (x'2,y'2),..., (x' n ,y' n ), where (x i ,y i ) and (x' i ,y' i ) represent the measured coordinates and ideal coordinates of the corresponding points of the i-th group of constraints, respectively, and we get:

[0027]

[0028]

[0029] Let X' = [x'1, x'2, ..., x' n ] T ,Y'=[y'1,y'2,...,y' n ] T , K1=[k1,k2,k3,k4] T , K2=[k5,k6,k7,k8] T , then the above X' and Y' are expressed as:

[0030] X'=A×K1

[0031] Y'=A×K2

[0032] So we can solve K1 and K2 as:

[0033] K1=(A T A) -1 A T X'

[0034] K2=(A T A) -1 A T Y'.

[0035] Further, step S25 specifically includes:

[0036] According to the coefficients k1, k2, k3, k4, k5, k6, k7, and k8 solved in step S24, for all pixels f(x, y) in the instrument distortion map, a spatial transformation is performed through the distortion transformation function to obtain g(x', y'), where f represents the instrument distortion map and g represents the instrument correction map.

[0037] Further, in step S3, determining the pointer deflection angle specifically includes:

[0038] S31, segmenting the instrument correction image using an image segmentation method to obtain a pointer binary image;

[0039] S32, fitting the pointer center line using the least squares method based on the pointer binary image;

[0040] S33. Calculate the deflection angle of the pointer according to the fitted pointer center line.

[0041] Furthermore, in step S31, the image segmentation method includes but is not limited to threshold segmentation and region segmentation.

[0042] Further, step S33 specifically includes:

[0043] The pointer deflection angle θ is calculated based on the pointer centerline obtained in step S32:

[0044]

[0045] Where k is the slope of the straight line where the center line of the pointer lies.

[0046] Beneficial effects of the present invention:

[0047] The present invention corrects the image by attaching a micro-calibration plate on the surface of the instrument panel, selects the corresponding points of the image transformation constraint and uses its coordinate values ​​to calculate the distortion transformation function, and then corrects the instrument distortion image obtained by shooting, reducing the influence of factors such as camera distortion and camera shooting angle deviation on the image, thereby improving the reading recognition accuracy of the pointer instrument. The method can be easily migrated and deployed in different industrial fields to meet practical application needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 Schematic diagram of a pointer-type instrument with a micro-calibration plate in an embodiment of the present invention.

[0049] Figure 2 Schematic diagram of constraining corresponding points in an embodiment of the present invention.

[0050] Figure 3 It is a flow chart of a method for eliminating image distortion of a pointer-type instrument based on a calibration plate in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to facilitate those skilled in the art to better understand the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. The following is only exemplary and does not limit the protection scope of the present invention.

[0052] This embodiment discloses a method for eliminating image distortion of a pointer-type instrument based on a calibration plate. Figure 3 As shown, the process includes three steps: designing an instrument with a micro-calibration plate, eliminating the distortion of the instrument distortion diagram, and determining the pointer deflection angle.

[0053] 1. The process of designing a pointer instrument with a micro-calibration plate is as follows: attach the micro-calibration plate to a blank area of ​​the instrument panel so that the micro-calibration plate and the instrument panel are in the same plane.

[0054] Specifically, this embodiment discloses a process of designing an instrument with a micro-calibration plate, which specifically includes:

[0055] S11, the substrate of the micro-calibration plate includes s×t characteristic elements and is arranged in an array on the substrate in an equidistant manner, wherein s represents a row, t represents a column, s≥2, t≥2, and the characteristic elements are square spots or circular spots of the same size, such as Figure 1 As shown, Figure 1 The characteristic elements of the micro-calibration plate in the left picture are black square spots of the same size. Figure 1 The characteristic elements of the micro-calibration plate in the right figure are black circular spots of the same size. In this embodiment, both the circular spots and the square spots are set to black in order to better distinguish them from the white blank area of ​​the instrument panel. They can also be set to other colors, not limited to black, as long as they can form a clear contrast with the blank area of ​​the instrument panel.

[0056] S12, specify the blank area of ​​the instrument panel of the instrument to be measured, attach the micro calibration plate to the instrument panel, make the micro calibration plate and the instrument panel of the pointer instrument in the same plane, and produce a pointer instrument with a micro calibration plate. The size of the micro calibration plate can generally be designed to be 1 / 4 to 3 / 4 of the blank area of ​​the instrument panel. In the embodiment of the present invention, the size of the selected micro calibration plate is approximately 1 / 4 of the blank area of ​​the instrument panel, such as Figure 1 shown.

[0057] 2. The process of eliminating the distortion of the instrument distortion map is as follows: photographing to obtain the instrument distortion map, calculating the distortion transformation function through the constraint corresponding points of the image transformation, and transforming the instrument distortion map to obtain the instrument correction map.

[0058] Specifically, this embodiment discloses a process for eliminating distortion of an instrument distortion image, which specifically includes:

[0059] S21. For the pointer instrument with the micro-calibration plate obtained in step S1, the instrument panel including the micro-calibration plate is photographed with forward vision to obtain an instrument distortion image, wherein the camera adopts an industrial camera, which can withstand high and low temperatures, has good waterproof and dustproof properties, and can obtain high-quality images.

[0060] S22. Use an image processing corner point detection algorithm to extract the measured coordinates of the feature points of the micro-calibration plate in the instrument distortion image. The feature points of the micro-calibration plate described in this embodiment are the corner points of the square spots. The Harris corner point detection algorithm or the Shi-Tomasi corner point detection algorithm can be used to obtain the measured coordinates of the feature points.

[0061] S23, according to the image resolution requirements of the pointer instrument and the number of rows s and columns t of the characteristic elements of the micro-calibration plate, set the ideal coordinates corresponding to each characteristic point of the micro-calibration plate. Figure 1 For the micro-calibration plate shown in the left figure, assume that the ideal coordinates of the feature point in the upper left corner of the micro-calibration plate are (X0, Y0), the ideal coordinates of the feature point in the upper right corner are (X1, Y0), the ideal coordinates of the feature point in the lower left corner are (X0, Y1), and the ideal coordinates of the feature point in the lower right corner are (X1, Y1), then:

[0062]

[0063]

[0064]

[0065] Where W×H is the image resolution. The distances between adjacent feature points in the micro-calibration plate are equal, and the ideal coordinates of each feature point are calculated proportionally.

[0066] S24, the measured coordinates of the feature points and the ideal coordinates constitute the constrained corresponding points of the image transformation, such as Figure 2 As shown, the left figure is a schematic diagram of the measured coordinates, and the right figure is a schematic diagram of the ideal coordinates. The distortion transformation function is calculated through n sets of constraint corresponding points, where n ≥ 4. Figure 2 Points 1, 2, 3 and 4 on the four corners of the micro-calibration plate are shown in FIG. 4 , which represent four sets of constrained corresponding points. Specifically, the distortion transformation function to be estimated is a model based on bilinear approximation, which is given by the following two equations:

[0067] x'=k1x+k2y+k3xy+k4=[x,y,xy,1]×[k1,k2,k3,k4] T

[0068] y'=k5x+k6y+k7xy+k8=[x,y,xy,1]×[k5,k6,k7,k8] T

[0069] (x, y) and (x', y') form a set of constraint corresponding points, representing the measured coordinates and the ideal coordinates respectively. All the unknown coefficients k1, k2, k3, k4, k5, k6, k7, k8 of the above equation are solved through n sets of constraint corresponding points.

[0070] Assume that there are n sets of constraint corresponding points, (x1, y1), (x2, y2), ..., (x n ,y n ) and (x'1,y'1), (x'2,y'2),..., (x' n ,y' n ), where (x i ,y i ) and (x' i ,y' i ) represent the measured coordinates and ideal coordinates of the corresponding points of the i-th group of constraints, respectively, and we get:

[0071]

[0072]

[0073] Let X' = [x'1, x'2, ..., x' n ] T ,Y'=[y'1,y'2,...,y' n ] T , K1=[k1,k2,k3,k4] T , K2=[k5,k6,k7,k8] T , then the above X' and Y' are expressed as:

[0074] X'=A×K1

[0075] Y'=A×K2

[0076] So we can solve K1 and K2 as:

[0077] K1=(A T A) -1 A T X'

[0078] K2=(A T A) -1 A T Y'.

[0079] S25, using the distortion transformation function to correct the pixel spatial position of the instrument distortion image, eliminating the distortion of the instrument distortion image, and obtaining the instrument correction image, specifically including the following:

[0080] According to the coefficients k1, k2, k3, k4, k5, k6, k7, and k8 solved in step S24, for all pixels f(x, y) in the instrument distortion map, a spatial transformation is performed through the distortion transformation function to obtain g(x', y'), where f represents the instrument distortion map and g represents the instrument correction map.

[0081] 3. Determine the deflection angle of the pointer: segment the binary image of the pointer in the instrument correction image, fit the center line of the pointer, and calculate the deflection angle of the center line of the pointer.

[0082] Specifically, this embodiment discloses a process for calculating the deflection angle of an instrument pointer, which specifically includes:

[0083] S31. Use an image segmentation method to segment the instrument correction image to obtain a pointer binary image. The image segmentation method includes threshold segmentation, region segmentation and other segmentation methods, all of which are existing methods. This embodiment can adopt any one of them.

[0084] S32. Based on the pointer binary image, the pointer center line is fitted using the least squares method:

[0085] Assume that the fitted straight line is expressed as:

[0086] y(x)=kx+b

[0087] Then n fitting points (x i ,y i )(i=1,2,3,...,n), and the residual sum of squares of the dependent variable value on the fitting line is expressed as:

[0088]

[0089] When 2 When it is the smallest, the corresponding coefficient values ​​of k and b are the desired values. The first-order partial derivatives of k and b are obtained, and the formula is as follows:

[0090]

[0091]

[0092] The connected domain pixel points of the pointer binary image obtained by segmentation in S31 are used as fitting points to obtain the k and b values, and obtain the best fitting straight line of the pointer, that is, the center line of the pointer.

[0093] S33, according to the fitted pointer center line, the deflection angle of the pointer is calculated, and the specific calculation process is as follows:

[0094] The pointer deflection angle θ is calculated based on the pointer centerline obtained in step S32:

[0095]

[0096] Where k is the slope of the straight line where the center line of the pointer lies.

[0097] At this point, the distortion elimination of the pointer instrument image is completed through the above steps S1-S3, so that the pointer deflection angle is accurately obtained, thereby improving the instrument reading recognition accuracy.

[0098] The above only describes the basic principles and preferred embodiments of the present invention. Those skilled in the art may make many changes and improvements based on the above description, and these changes and improvements should fall within the protection scope of the present invention.

Claims

1. A method for eliminating image distortion of a pointer instrument based on a calibration plate, characterized in that: At least the steps include: S1. Design a pointer instrument with a micro-calibration plate: attach the micro-calibration plate to the blank area of ​​the instrument panel so that the micro-calibration plate and the instrument panel are in the same plane; S2. Eliminating the distortion of the instrument distortion image: photographing to obtain the instrument distortion image, calculating the distortion transformation function through the constraint corresponding points of the image transformation, and transforming the instrument distortion image to obtain the instrument correction image; eliminating the distortion of the instrument distortion image specifically includes: S21, for the pointer instrument with the micro-calibration plate obtained in step S1, photograph the instrument panel including the micro-calibration plate with forward vision, and ensure that the camera is parallel to the instrument panel during the photographing, to obtain an instrument distortion image; S22, using an image processing corner point detection algorithm to extract the measured coordinates of the feature points of the micro-calibration plate in the instrument distortion image; S23, according to the image resolution requirements of the pointer instrument, setting the ideal coordinates corresponding to each feature point of the micro-calibration plate; S24, using the measured coordinates of the feature points and the ideal coordinates to form the constraint corresponding points of the image transformation, and calculating the distortion transformation function through n groups of constraint corresponding points, wherein n≥4; step S24 specifically includes: The distortion transformation function to be estimated is a model based on bilinear approximation, which is given by the following two equations: x'=k1x+k2y+k3xy+k4=[x,y,xy,1]×[k1,k2,k3,k4] T y'=k5x+k6y+k7xy+k8=[x,y,xy,1]×[k5,k6,k7,k8] T (x, y) and (x', y') constitute a set of constraint corresponding points, representing the measured coordinates and the ideal coordinates respectively. All the unknown coefficients k1, k2, k3, k4, k5, k6, k7, k8 of the above equation are solved through n sets of constraint corresponding points; Assume that there are n sets of constraint corresponding points, (x1, y1), (x2, y2), ..., (x n ,y n ) and (x'1,y'1), (x'2,y'2),..., (x' n ,y' n ), where (x i ,y i ) and (x' i ,y' i ) represent the measured coordinates and ideal coordinates of the corresponding points of the i-th group of constraints, respectively, and we get: Let X' = [x'1, x'2, ..., x' n ] T ,Y'=[y'1,y'2,...,y' n ] T , K1=[k1,k2,k3,k4] T , K2=[k5,k6,k7,k8] T , then the above X' and Y' are expressed as: X'=A×K1 Y'=A×K2 So we can solve K1 and K2 as: K1=(A T A) -1 A T X' K2=(A T A) -1 A T Y' S25, using the distortion transformation function to correct the pixel spatial position of the instrument distortion image, eliminating the distortion of the instrument distortion image, and obtaining the instrument correction image; S3. Determine the deflection angle of the pointer: segment the pointer binary image in the instrument correction image, fit the pointer center line, and calculate the deflection angle of the pointer center line.

2. The method for eliminating image distortion of a pointer instrument based on a calibration plate according to claim 1, characterized in that: In step S1, designing an instrument with a micro-calibration plate specifically includes: S11, assuming that the substrate of the micro-calibration plate includes s×t characteristic elements and are arranged in an array on the substrate in an equidistant manner, wherein s represents a row, t represents a column, s≥2, t≥2, and the characteristic elements are square spots or circular spots of the same size; S12, attaching the micro-calibration plate to a blank area of ​​the instrument panel so that the micro-calibration plate and the instrument panel are in the same plane, thereby obtaining a pointer-type instrument with the micro-calibration plate.

3. The method for eliminating image distortion of a pointer-type instrument based on a calibration plate according to claim 2, characterized in that: In step S12, the size of the micro calibration plate is 1 / 4 to 3 / 4 of the blank area of ​​the instrument panel.

4. The method for eliminating image distortion of a pointer instrument based on a calibration plate according to claim 2 or 3, characterized in that: In step S22, the characteristic points of the micro-calibration plate are the corner points of the square spots or the centers of the circular spots.

5. The method for eliminating image distortion of a pointer-type instrument based on a calibration plate according to claim 2 or 3, characterized in that: Step S25 specifically includes: According to the coefficients k1, k2, k3, k4, k5, k6, k7, and k8 solved in step S24, for all pixels f(x, y) in the instrument distortion map, a spatial transformation is performed through the distortion transformation function to obtain g(x', y'), where f represents the instrument distortion map and g represents the instrument correction map.

6. The method for eliminating image distortion of a pointer instrument based on a calibration plate according to claim 1, 2 or 3, characterized in that: In step S3, determining the pointer deflection angle specifically includes: S31, segmenting the instrument correction image using an image segmentation method to obtain a pointer binary image; S32, fitting the pointer center line using the least squares method based on the pointer binary image; S33. Calculate the deflection angle of the pointer according to the fitted pointer center line.

7. The method for eliminating image distortion of a pointer instrument based on a calibration plate according to claim 6, characterized in that: In step S31, the image segmentation method includes but is not limited to threshold segmentation and region segmentation.

8. The method for eliminating image distortion of a pointer instrument based on a calibration plate according to claim 6, characterized in that: Step S33 specifically includes: The pointer deflection angle θ is calculated based on the pointer centerline obtained in step S32: Where k is the slope of the straight line where the center line of the pointer lies.

Images