A dashboard recognition method based on super-resolution and key points

Abstract

This invention discloses a dashboard recognition method based on super-resolution and key points, comprising the following steps: preprocessing images to obtain real and high-resolution images; modeling and applying statistical learning algorithms to obtain fuzzy kernel pooling, noise probability distribution, and noise type distribution parameters; randomly combining the fuzzy kernel pooling, noise probability distribution, and noise type distribution parameters to obtain a feature pool; after feature pooling, the high-resolution images are used to obtain a real training set, and the high-resolution images are subjected to three bidirectional down / up samplings to obtain a high-resolution dataset; the real training set and the high-resolution dataset are used to train the model, and feature extraction is performed; the feature extraction output is fed into the network reconstruction layer, and the instrument readings are read according to the positional relationship between the instrument scale and the pointer. This invention not only solves the influence of angle and lighting on image processing, but also greatly improves the accuracy of AI recognition in experiments.

Description

Technical Field

[0001] This invention relates to the field of AI recognition, and in particular to a dashboard recognition method based on super-resolution and key points. Background Technology

[0002] In electrical experiments, the readings of ammeters and voltmeters are frequently identified, and zeroing operations are often required. Due to factors such as lighting, shooting angle, and camera resolution, the scales on the instrument panel may be unclear, and the image may be blurry. Neither manual identification nor AI algorithms can extract crucial information from the image, leading to incorrect readings. Current mainstream solutions involve traditional methods such as locating all scales and pointers on the instrument panel, instrument panel matching, and pointer angle determination based on visual networks. However, these solutions have significant limitations.

[0003] 1. Traditional image algorithms: are greatly affected by image quality and angle;

[0004] 2. Dashboard matching: significant error.

[0005] 3. The method based on pointer angle has a large error. Summary of the Invention

[0006] The purpose of this invention is to provide a dashboard recognition method based on super-resolution and key points to improve image processing performance and enhance the accuracy of AI recognition.

[0007] To address the aforementioned technical problems, this invention provides a dashboard recognition method based on super-resolution and key points, comprising the following steps:

[0008] By using a production camera and a high-definition camera to photograph the instrument panel and preprocessing the images, both real and high-definition images of the instrument panel area are obtained.

[0009] Modeling was performed using real and high-resolution image datasets. Through statistical learning algorithms, parameters for fuzzy kernel pooling, noise probability distribution, and noise type distribution were obtained under different conditions.

[0010] The feature pool for generating real training images is obtained by randomly combining all possible parameters of the fuzzy kernel pool, noise probability distribution, and noise type distribution.

[0011] After the high-resolution images are processed by feature pooling, the real training set is obtained. The high-resolution images are then subjected to three bidirectional downsampling / upsampling to obtain the high-resolution dataset.

[0012] The model is trained using real training sets and high-definition datasets, and features are extracted and output.

[0013] The feature extraction output is fed into the network reconstruction layer to output quality assessment and key point assessment, and the instrument reading is read according to the positional relationship between the instrument scale and the pointer.

[0014] Furthermore, the preprocessing includes the following steps:

[0015] The dashboard area of ​​the instrument is cropped by object detection, and the coordinate information of the four corner points of the dashboard area is obtained by key point detection model.

[0016] The dashboard area is calibrated using information from the four corner points.

[0017] Furthermore, the transformation formula for correcting the dashboard area is as follows:

[0018]

[0019] Where (x, y, 1) are the coordinates before transformation, and (X, Y, Z) are the coordinates after transformation.

[0020] Furthermore, the calculation formulas for the fuzzy kernel pool, noise probability distribution, and noise type distribution parameters are obtained by setting different conditions:

[0021]

[0022] Among them, X i Represents the i-th object, 1≤i≤n; C j X represents the j-th object, where 1 ≤ j ≤ k; it Let C represent the t-th attribute of the i-th object, where 1 ≤ t ≤ m; jt This represents the t-th attribute of the j-th object.

[0023] Furthermore, the feature extraction output is performed using seven 3×3 CNN networks.

[0024] Furthermore, the loss function for the quality assessment is:

[0025]

[0026] Among them, y i f(x) is the true value. i (This is an estimated value.)

[0027] Furthermore, the loss function for the keypoint evaluation is:

[0028]

[0029] in, f represents the keypoint confidence error. The confidence error of the x-axis of the key point. Let W(p) be the confidence error of the keypoint ordinate, W(p) be the bias coefficient, J be the set of abscissas, and C be the set of ordinates. To predict the x-coordinate of key points, To predict the ordinate of key points, The x-coordinate of the actual key point The vertical coordinates of the actual key points.

[0030] Compared with the prior art, the present invention has at least the following beneficial effects:

[0031] This invention processes captured images into a consistent image through image correction, effectively solving the problem of the shooting angle affecting the detection results. In addition, a super-resolution algorithm is used to restore unclear parts of the image, making the dashboard clearly visible and improving the accuracy of subsequent instrument readings.

[0032] Furthermore, this invention utilizes key points to precisely locate each scale and pointer, and accurately calculates the instrument readings based on the relationship between the pointer and the scale, greatly improving the accuracy of AI recognition. Attached Figure Description

[0033] Figure 1 This is a flowchart of an embodiment of the dashboard recognition method based on super-resolution and key points of the present invention;

[0034] Figure 2 This is a flowchart of an embodiment of the dashboard recognition method based on super-resolution and key points of the present invention. Detailed Implementation

[0035] The dashboard recognition method based on super-resolution and key points of the present invention will be described in more detail below with reference to the schematic diagrams, which illustrate preferred embodiments of the invention. It should be understood that those skilled in the art can modify the invention described herein while still achieving the advantageous effects of the invention. Therefore, the following description should be understood as being of general knowledge to those skilled in the art and is not intended to limit the invention.

[0036] The invention is described more specifically by way of example in the following paragraphs with reference to the accompanying drawings. The advantages and features of the invention will become clearer from the following description and claims. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.

[0037] like Figure 1 and Figure 2 As shown in the figure, this invention proposes a dashboard recognition method based on super-resolution and key points, including the following steps:

[0038] (1) The instrument is photographed by a production camera and a high-definition camera and the images are preprocessed to obtain real images and high-definition images of the instrument panel area respectively.

[0039] Specifically, images of the instrument are captured at the same location using two cameras: one is the camera used in actual production, and the other is a high-definition camera. The instrument panel area is cropped using object detection, and the coordinates of the four corner points of the instrument panel area are obtained using a key point detection model. Then, the instrument panel area is corrected using this corner point information. The transformation formula for correcting the four corner point information of the instrument panel is as follows:

[0040]

[0041] Where (x, y, 1) are the coordinates before transformation, and (X, Y, Z) are the coordinates after transformation. The corrected image ensures that all dashboard areas are identical, thus solving the problem of being unaffected by angle. After correction, real images and high-definition images are obtained using cameras used in actual production and high-definition cameras, respectively.

[0042] In one embodiment, when performing rotation correction on an image, the transformation matrix... for After transformation Where (x, y, 1) are the coordinates before transformation, and (X, Y, 1) are the coordinates after transformation.

[0043] (2) Modeling was performed using real images and high-definition image datasets. Through statistical learning algorithms, different conditions were set to obtain parameters for fuzzy kernel pool, noise probability distribution, and noise type distribution.

[0044] Specifically, modeling is performed using real and high-resolution image datasets. Through statistical learning algorithms, different conditions are set to derive the blur kernel pooling, noise probability distribution, and noise type distribution, as shown in the following formulas:

[0045]

[0046] Among them, X i Represents the i-th object, 1≤i≤n; C j X represents the j-th object, where 1 ≤ j ≤ k; it Let C represent the t-th attribute of the i-th object, where 1 ≤ t ≤ m; jt This represents the t-th attribute of the j-th object.

[0047] In this step, the fuzzy kernel pool is the set of kernel sizes for different types of fuzzing; the noise distribution probability is the probability of occurrence for different types of noise, and by modeling, we can know which type of noise is more likely to occur; the noise type is hypothetical and only indicates that there are multiple types of noise.

[0048] (3) Randomly combine all possible parameters of the fuzzy kernel pool, noise probability distribution, and noise type distribution to obtain the feature pool for generating real training images. After the high-definition images are processed by the feature pool, the real training set is obtained. The high-definition images are then subjected to three bidirectional downsampling / upsampling to obtain the high-definition dataset.

[0049] Specifically, after a high-resolution image is processed through a parameter feature pool, multiple real images for training can be generated. This is a one-to-many relationship, which can be controlled by parameters. Thus, the real training set is generated. Additionally, the high-resolution image undergoes three downsampling / upsampling (bidirectional) cycles, effectively reducing its resolution to more closely resemble the real image. This approach primarily enriches the data, resulting in the high-resolution dataset.

[0050] (4) Train the model using the real training set and the high-definition dataset, and extract and output the features.

[0051] Specifically, the real training set and the high-resolution dataset are fed into the model for training. The real training set and the high-resolution dataset are then fed into the feature extraction layer, where seven 3×3 CNN networks are used to extract features.

[0052] (5) The feature extraction output is fed into the network reconstruction layer to output quality assessment and key point assessment, and the instrument reading is read according to the position relationship between the instrument scale and the pointer.

[0053] Specifically, multiple feature extraction outputs are fed into the network reconstruction layer, which mainly adopts a network layer with a ResNet-like structure. The output of this part is divided into two branches: quality evaluation and key point evaluation.

[0054] The loss function for quality assessment is: y i f(x) is the true value. i The value is the predicted value. The loss function used for quality assessment is the squared loss function, which calculates the square of the difference between the algorithm's prediction and the actual training data label information. The smaller the loss value, the smaller the difference between the predicted value and the actual value.

[0055] The loss function for keypoint evaluation is: in, f represents the keypoint confidence error. The confidence error of the x-axis of the key point. Let W(p) be the confidence error of the keypoint ordinate, W(p) be the bias coefficient, J be the set of abscissas, and C be the set of ordinates. To predict the x-coordinate of key points, To predict the ordinate of key points, The x-coordinate of the actual key point This represents the ordinate of the actual keypoint. The smaller the loss function value for keypoint evaluation, the more accurate the keypoints for obtaining dashboard scales and pointers.

[0056] Combining these two methods not only yielded high-definition images but also revealed key points on the instrument's scale and pointer. Furthermore, the instrument readings were obtained based on the positional relationship between the scale and pointer. It should be noted that the instruments used are not limited to the ammeter and voltmeter used in the experiment.

[0057] In summary, this invention processes captured images into consistent images through image correction methods, effectively solving the problem of the shooting angle affecting the detection results. In addition, the use of a super-resolution algorithm restores unclear parts of the image, making the dashboard clearly visible and improving the accuracy of subsequent instrument readings.

[0058] Furthermore, this invention utilizes key points to precisely locate each scale and pointer, and accurately calculates the instrument readings based on the relationship between the pointer and the scale, greatly improving the accuracy of AI recognition.

[0059] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A dashboard recognition method based on super-resolution and key points, characterized in that, Includes the following steps: By using a production camera and a high-definition camera to photograph the instrument panel and preprocessing the images, both real and high-definition images of the instrument panel area are obtained. Modeling was performed using real and high-resolution image datasets. Through statistical learning algorithms, parameters for fuzzy kernel pooling, noise probability distribution, and noise type distribution were obtained under different conditions. The feature pool for generating real training images is obtained by randomly combining all possible parameters of the fuzzy kernel pool, noise probability distribution, and noise type distribution. After the high-resolution images are processed by feature pooling, the real training set is obtained. The high-resolution images are then subjected to three bidirectional downsampling / upsampling to obtain the high-resolution dataset. The model is trained using real training sets and high-definition datasets, and features are extracted and output. The feature extraction output is fed into the network reconstruction layer to output quality assessment and key point assessment, and the instrument reading is read according to the positional relationship between the instrument scale and the pointer. The preprocessing includes the following steps: The dashboard area of ​​the instrument is cropped by object detection, and the coordinate information of the four corner points of the dashboard area is obtained by key point detection model. The dashboard area is calibrated using information from the four corner points. The calculation formulas for the fuzzy kernel pool, noise probability distribution, and noise type distribution parameters are obtained by setting different conditions respectively: ； in, Indicates the first One object, ; Indicates the first One object, ; Indicates the first The t-th property of an object ; Indicates the first The t-th property of an object.

2. The dashboard recognition method based on super-resolution and key points as described in claim 1, characterized in that, The transformation formula for correcting the dashboard area is: ； Where (x, y, 1) are the coordinates before transformation, and (X, Y, Z) are the coordinates after transformation.

3. The dashboard recognition method based on super-resolution and key points as described in claim 1, characterized in that, The feature extraction output is performed using seven 3×3 CNN networks.

4. The dashboard recognition method based on super-resolution and key points as described in claim 1, characterized in that, The loss function for the quality assessment is: ； in, For the true value, This is an estimated value.

5. The dashboard recognition method based on super-resolution and key points as described in claim 1, characterized in that, The loss function for the keypoint evaluation is: ； in, , , For key point confidence error, The confidence error of the x-axis of the key point. The confidence error of the ordinate of the key point. Here, J represents the bias coefficient, and C represents the set of horizontal coordinates and vertical coordinates. To predict the x-coordinate of key points, To predict the ordinate of key points, For practical key points x-coordinate The vertical coordinates of the actual key points.

Images