Pointer instrument reading method and system based on deep learning

Abstract

This application discloses a method and system for reading a pointer instrument based on deep learning, which relates to the field of intelligent instrument reading technology. The method includes the following steps: collecting image information of a pointer instrument; locating a reading area of ​​the pointer instrument in the image information using a neural network model; performing binarization processing and Blod analysis on the reading area to determine the pointer center position and scale position of the pointer instrument; determining the pointer position of the pointer instrument based on the scale position and the pointer center position, and calculating the scale angle range and pointer angle; and calculating the instrument reading of the pointer instrument based on the scale angle range and the pointer angle using a local search method. This application has the effect of being able to read multiple types of instruments without parameter adjustment.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent instrument reading, and in particular to a method and system for reading pointer instruments based on deep learning. Background Art

[0002] Power plants and substations are both crucial components of the power system. Power plants provide voltage and current, while substations transform and distribute these voltages and currents. During power transmission, numerous analog instruments are required to monitor and obtain real-time data. The overall operating status of the power plant or substation is analyzed by analyzing the readings from these instruments.

[0003] In recent years, some solutions have emerged for using machines to read instruments. Compared with manual reading, using machines to read instruments will save a lot of reading time. For example, by detecting instruments through SURF feature points, the image information of the instrument is usually collected first, and then the instrument image in the image information is captured. The instrument image is then processed and analyzed based on the instrument features of the same type of instrument, and finally the reading of the pointer instrument in the instrument image is obtained.

[0004] Regarding the above-mentioned related technologies, the inventors believe that there are the following defects: the instrument reading method in the related technology requires obtaining the dial characteristics of the instrument type to be tested in advance, then configuring processing parameters according to the dial characteristics, and then performing image analysis processing on the instrument image of the instrument to be tested based on the processing parameters. However, the dial characteristics of different types of instruments are also different. When it is necessary to read different types of instruments, the processing parameters need to be adjusted in advance. Therefore, it has limitations when reading multiple types of instruments. Summary of the Invention

[0005] In order to improve the limitations of reading multiple types of instruments, the present application provides a pointer instrument reading method and system based on deep learning.

[0006] In a first aspect, the present application provides a pointer instrument reading method based on deep learning, the method comprising the following steps:

[0007] Collect image information of pointer instruments;

[0008] Locating a reading area where the pointer instrument exists in the image information using a neural network model;

[0009] Performing binarization processing and Blod analysis on the reading area to determine the center position of the pointer and the scale position of the pointer instrument;

[0010] Calculating the scale angle range and the pointer angle based on the scale position and the pointer center position;

[0011] An instrument reading of the pointer instrument is obtained by calculation based on the scale angle range and the pointer angle through a local search method.

[0012] By adopting the above technical solution, after collecting the image information of the pointer instrument, the image features of the pointer instrument in the image information can be deeply extracted through the neural network model, so that the reading area where any type of pointer instrument is located can be located in the image information, and then the reading area is binarized to highlight the pointer features and scale features in the reading area. Even if the pointer length, maximum range, scale position or scale size of different types of pointer instruments are different, the pointer center position and scale position can be determined through Blod analysis, and the scale angle range and pointer angle can be calculated based on the pointer center position and scale position. The scale angle range determines the maximum range of the pointer instrument, and the pointer angle represents the current reading of the pointer instrument. Therefore, the current instrument reading of the pointer instrument can be calculated through the local search method.

[0013] Optionally, locating the reading area where the pointer instrument exists in the image information by using a neural network model includes the following steps:

[0014] Adjusting the size of the image information to obtain standard image information of a standard size;

[0015] Extracting features of the standard image information using a backbone network with the fully connected layer removed to obtain image features of the standard image information;

[0016] Performing a convolution operation on the image features through a residual structure to obtain a feature tensor of the standard image information;

[0017] A reading area where the pointer instrument exists is located in the standard image information according to the feature tensor.

[0018] By adopting this technical solution, before processing image information based on a neural network model, the image information must first be resized to a standard size suitable for the neural network model. A backbone network, minus the fully connected layers, is then used as a feature extractor to extract image features from the standard image information. After performing a convolution operation on the image features using a residual structure, feature tensors for each depth of image information are obtained. The feature tensors from the deep network are used for analyzing large-scale instruments, while the feature tensors from the shallow network are used for analyzing medium-scale and small-scale instruments, respectively. Finally, the feature tensors are used to locate the reading area containing the pointer instrument in the standard image information.

[0019] Optionally, performing binarization processing and Blod analysis on the reading area to determine the pointer center position and scale position of the pointer instrument includes the following steps:

[0020] Binarization is performed on the reading area in the RGB color space to obtain an RBG binary image;

[0021] Converting the reading area to an HSV color space, and performing a binarization process on the reading area in the HSV color space to obtain an HSV binarized image;

[0022] Performing Blod analysis on the RGB binary image to determine the center position of the pointer of the pointer instrument;

[0023] The Blod analysis is performed on the HSV binary image to determine the scale position of the pointer instrument.

[0024] By adopting the above technical solution, since the collected image information is an RGB image, the reading area is first binarized in the RGB color space, then converted to an HSV image in the HSV color space, and binarized again to obtain two binary images. In the RGB color space, the three color components are highly correlated, so continuous color changes are not intuitive. Therefore, the RGB binary image can more clearly display the pointer image, while the HSV color space is a color space with good uniformity, which is conducive to processing continuously changing colors. The colors of the multiple scales in the pointer instrument are continuously changed in combination with the background color, so the instrument scale can be more clearly displayed in the HSV binary image. By performing Blod analysis on the RGB binary image and the HSV binary image respectively, the pointer center position and scale position of the pointer instrument can be determined.

[0025] Optionally, performing Blod analysis on the RGB binary image to determine the center position of the pointer of the pointer instrument includes the following steps:

[0026] Performing denoising on the RGB binary image;

[0027] Performing connectivity analysis on the RGB binary image after the denoising process and identifying multiple connected domains;

[0028] Calculate the eigenvalue of each connected domain;

[0029] Compare each characteristic value with the preset pointer characteristic value respectively;

[0030] Marking the connected domain with the same eigenvalue as the pointer eigenvalue as a pointer connected domain;

[0031] The center position of the pointer of the pointer instrument is determined according to the center of gravity of the pointer connected domain.

[0032] By adopting the above technical solution, denoising processing can remove interfering noise points in the RGB binary image. Since the pointer is a continuous color area in the RGB binary image, the pointer image will be identified as a whole connected domain in the connectivity analysis. Since the length of the pointer image determines the length of the connected domain, the connected domain representing the pointer can be screened out from all connected domains based on the pointer eigenvalue, and the pointer center position of the pointer instrument can be determined based on the center of gravity of the pointer connected domain.

[0033] Optionally, the scale position includes a starting scale position and an ending scale position, and performing the Blod analysis on the HSV binary image to determine the scale position of the pointer instrument includes the following steps:

[0034] Performing denoising on the HSV binary image;

[0035] Performing connectivity analysis on the HSV binary image after denoising, and identifying multiple target connected domains;

[0036] Calculate the connected domain area of ​​each target connected domain respectively;

[0037] Select two target connected domains with the largest areas from all target connected domains as target scale connected domains;

[0038] Splitting the two target scale connected domains into a starting scale connected domain and an ending scale connected domain based on preset geometric parameters;

[0039] The starting scale position and the ending scale position are determined according to the positions of the starting scale connected domain and the ending scale connected domain.

[0040] By adopting the above technical solution, denoising processing can remove interference noise in the HSV binary image. In the HSV binary image, each scale is a continuous color area. Therefore, in the connectivity analysis, each scale image will be identified as a separate connected domain. In the instrument panel of the pointer instrument, the starting scale and the ending scale lines are usually longer or wider than the middle scale lines. Therefore, they can be screened according to the area of ​​the connected domain. The scales corresponding to the largest connected domain area are the scales at both ends. The two scale connected domains are then distinguished by preset geometric parameters and disassembled into the starting scale connected domain and the ending scale connected domain. Finally, the positions of the starting scale and the ending scale in the reading area are determined according to the positions of the two connected domains.

[0041] Optionally, the step of calculating the instrument reading of the pointer instrument based on the scale angle range and the pointer angle by a local search method comprises the following steps:

[0042] determining a pointer position of the pointer instrument according to the pointer angle;

[0043] Performing a local whole-scale search on an area adjacent to the pointer position based on the pointer angle and a preset angle threshold to obtain a pointer adjacent scale area;

[0044] Calculating the polar coordinate angles of all the whole scales in the area adjacent to the scale by using a polar coordinate search algorithm;

[0045] Analyze and screen all polar coordinate angles based on a non-maximum suppression algorithm to obtain two target polar coordinate angles, where two integral scales corresponding to the two target polar coordinate angles are closest to the pointer position and are located on both sides of the pointer position;

[0046] The instrument reading of the pointer instrument is calculated in combination with the scale angle range, the pointer angle and the target polar coordinate angle.

[0047] By adopting the above technical solution, the pointer position is first determined, and then a local whole-scale search is performed based on the pointer position to find the pointer's adjacent scale area. This facilitates reading the integer portion when subsequently calculating the instrument reading and can make the instrument reading accurate to decimals. After searching for the pointer's adjacent scale area, a polar coordinate system is first established and the polar coordinate angles of all the whole-scales adjacent to the pointer position are calculated using the polar coordinate search algorithm. The decimal portion is between two whole-scales, so after determining the polar coordinate angles of the two nearest whole-scales on both sides of the pointer, a more accurate instrument reading can be calculated by combining the scale range and the pointer angle.

[0048] Optionally, the calculating the instrument reading of the pointer instrument by combining the scale angle range, the pointer angle, and the target polar coordinate angle comprises the following steps:

[0049] Calculating a whole-scale reading of the pointer instrument based on the scale angle range and the pointer angle;

[0050] Calculate the fractional reading of the pointer instrument based on the target polar coordinate angle and the pointer angle;

[0051] The whole scale reading and the non-integer reading are added together to obtain the instrument reading of the pointer instrument.

[0052] By employing the above technical solution, the maximum range of a pointer instrument can be determined based on the scale angle range. Combined with the pointer angle within the scale angle range, the full-scale reading of the pointer instrument, i.e., the integer portion of the instrument reading, can be calculated. Furthermore, a local calculation using the two target polar coordinate angles and the pointer angle can be performed to calculate the non-integer reading of the pointer instrument, i.e., the decimal portion of the instrument reading. Finally, the integer and decimal portions are added together to obtain the pointer instrument reading.

[0053] In the second aspect, the present application also provides a pointer instrument reading system based on deep learning, including a memory, a processor, and a program stored on the memory and executable on the processor, which can implement the pointer instrument reading method based on deep learning as described in the first aspect when loaded and executed by the processor.

[0054] By adopting the above technical solution, the image information of the pointer instrument is first collected through the program call, and then the image features of the pointer instrument in the image information are deeply extracted through the neural network model, so that the reading area where any type of pointer instrument is located can be located in the image information, and then the reading area is binarized to highlight the pointer features and scale features in the reading area. Even if the pointer length, maximum range, scale position or scale size of different types of pointer instruments are different, the pointer center position and scale position can be determined through Blod analysis, and the scale angle range and pointer angle can be calculated based on the pointer center position and scale position. The scale angle range determines the maximum range of the pointer instrument, and the pointer angle represents the current reading of the pointer instrument. Therefore, the current instrument reading of the pointer instrument can be calculated through the local search method.

[0055] In summary, this application includes at least one of the following beneficial technical effects:

[0056] 1. First, collect the image information of the pointer instrument, and then use the neural network model to deeply extract the image features of the pointer instrument in the image information, so that the reading area where any type of pointer instrument is located can be located in the image information. Then, the reading area is binarized to highlight the pointer features and scale features in the reading area. Even if the pointer length, maximum range, scale position or scale size of different types of pointer instruments are different, the pointer center position and scale position can be determined through Blod analysis, and the scale angle range and pointer angle can be calculated based on the pointer center position and scale position. The scale angle range determines the maximum range of the pointer instrument, and the pointer angle represents the current reading of the pointer instrument. Therefore, the current instrument reading of the pointer instrument can be calculated through the local search method.

[0057] 2. First determine the pointer position, then perform a local whole-scale search based on the pointer position to find the pointer's adjacent scale area. This will facilitate reading the integer part when calculating the instrument reading later, and can make the instrument reading accurate to decimals. After searching for the pointer's adjacent scale area, first establish a polar coordinate system and calculate the polar coordinate angles of all the whole-scales near the pointer position using the polar coordinate search algorithm. The decimal part is between two whole-scales, so after determining the polar coordinate angles of the two nearest whole-scales on both sides of the pointer, a more accurate instrument reading can be calculated by combining the scale range and pointer angle. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flowchart of a pointer instrument reading method based on deep learning in one embodiment of the present application.

[0059] Figure 2 This is a flow chart of locating a reading area through a neural network model according to one embodiment of the present application.

[0060] Figure 3 This is a flowchart of binarization and Blod analysis of a reading area according to one embodiment of the present application.

[0061] Figure 4 It is a flowchart of determining the center position of a pointer of a pointer instrument according to one embodiment of the present application.

[0062] Figure 5 It is a schematic diagram of a flow chart for determining the scale position of a pointer instrument according to one embodiment of the present application.

[0063] Figure 6 This is a flow chart of calculating the instrument reading of a pointer instrument based on the scale angle range and the pointer angle according to one embodiment of the present application.

[0064] Figure 7 This is a flow chart of calculating an instrument reading by combining the scale angle range, pointer angle, and target polar coordinate angle according to one embodiment of the present application. DETAILED DESCRIPTION

[0065] The following is combined with Figure 1-7 This application is described in further detail.

[0066] The embodiment of the present application discloses a pointer instrument reading method based on deep learning.

[0067] Reference Figure 1 ,The pointer instrument reading method based on deep learning includes the following steps:

[0068] 101, collect image information of the pointer instrument.

[0069] Among them, a high-definition camera can be used to collect image information including pointer instruments.

[0070] 102, locating the reading area of ​​the pointer instrument in the image information through a neural network model.

[0071] Among them, the image information is subjected to feature extraction and convolution processing through a neural network model, so that the reading area can be located according to the feature tensor of the pointer instrument in the image information.

[0072] 103. Perform binarization processing and Blod analysis on the reading area to determine the pointer center position and scale position of the pointer instrument.

[0073] Among them, the reading area after binarization processing only contains two color values ​​0 and 255, so the instrument scale and instrument pointer can be highlighted, which is conducive to using Blod analysis to determine the pointer center position and scale position of the pointer instrument in the reading area.

[0074] 104 , calculating the scale angle range and the pointer angle based on the scale position and the pointer center position.

[0075] Among them, the polar coordinate system is established using the found center position of the pointer. The binary scale is searched with the polar angle as the step size through the change of the polar coordinate system, and the cumulative score of the white points at each polar angle is recorded. The area with high cumulative score is the pointer area and the scale area, so that the scale angle range can be calculated according to the scale area, and then the pointer angle can be calculated according to the scale angle range and the polar coordinate system.

[0076] 105, the instrument reading of the pointer instrument is calculated based on the scale angle range and the pointer angle and through a local search method.

[0077] The implementation principle of this embodiment is:

[0078] After collecting the image information of the pointer instrument, the image features of the pointer instrument in the image information can be deeply extracted through the neural network model, so that the reading area where any type of pointer instrument is located can be located in the image information, and then the reading area can be binarized to highlight the pointer features and scale features in the reading area. Even if the pointer length, maximum range, scale position or scale size of different types of pointer instruments are different, the pointer center position and scale position can be determined through Blod analysis, and the scale angle range and pointer angle can be calculated based on the pointer center position and scale position. The scale angle range determines the maximum range of the pointer instrument, and the pointer angle represents the current reading of the pointer instrument. Therefore, the current instrument reading of the pointer instrument can be calculated through the local search method.

[0079] exist Figure 1In step 102 of the embodiment shown, the feature tensor in the image information can be deeply extracted based on the neural network model, so that instruments of different scales in the image information can be located. Figure 2 The illustrated embodiment is described in detail.

[0080] Reference Figure 2 , locating the reading area through the neural network model includes the following steps:

[0081] 201 : Adjust the size of the image information to obtain standard image information of a standard size.

[0082] 202 , using the backbone network with the fully connected layer removed to extract features of the standard image information, to obtain image features of the standard image information.

[0083] 203, performing a convolution operation on the image features through the residual structure to obtain a feature tensor of standard image information.

[0084] Among them, the feature tensor is obtained by splicing the feature tensor of the residual structure with the previous sampling tensor. The feature tensor of the deep network is used for the analysis of large-scale instruments, and the feature tensors of the shallow network are used for the analysis of medium-scale and small-scale instruments respectively.

[0085] 204 , locating a reading area where a pointer instrument exists in the standard image information according to the feature tensor.

[0086] The implementation principle of this embodiment is:

[0087] Before processing image information based on a neural network model, the image information must be resized to a standard size suitable for the neural network model. A backbone network, minus the fully connected layers, is then used as a feature extractor to extract image features from the standard image information. After performing convolution operations on the image features using a residual structure, feature tensors for each depth of the image are obtained. The feature tensors from the deep network are used to analyze large-scale instruments, while those from the shallow network are used for medium-scale and small-scale instruments, respectively. Finally, the feature tensors are used to locate the reading area containing the pointer instrument in the standard image information.

[0088] exist Figure 1 In step 103 of the embodiment shown, the image information is binarized from the RGB color space and the HSV color space, and Blod analysis is performed on the two binarized images to determine the center position of the pointer and the scale position of the pointer instrument. Figure 3 The illustrated embodiment is described in detail.

[0089] Reference Figure 3,binarization processing and Blod analysis of the reading area include the following steps:

[0090] 301, performing binarization processing on the reading area in the RGB color space to obtain an RBG binary image.

[0091] Among them, the threshold segmentation method is used to perform binary processing on the reading area.

[0092] 302 , converting the reading area into an HSV color space, and performing binarization processing on the reading area in the HSV color space to obtain an HSV binarized image.

[0093] Among them, the threshold segmentation method is used to perform binary processing on the reading area.

[0094] 303 , performing Blod analysis on the RGB binary image to determine the center position of the pointer of the pointer instrument.

[0095] 304 , performing Blod analysis on the HSV binary image to determine the scale position of the pointer instrument.

[0096] The implementation principle of this embodiment is:

[0097] Since the acquired image information is an RGB image, the reading area is first binarized in the RGB color space. This is then converted to an HSV image in the HSV color space and binarized again, resulting in two binary images. In the RGB color space, the three color components are highly correlated, making continuous color changes unintuitive. Therefore, the RGB binary image can more clearly display the pointer image. The HSV color space, on the other hand, is a color space with good uniformity, making it easier to handle continuously changing colors. The colors of the multiple scales in a pointer instrument continuously change in combination with the background color, so the instrument scales can be more clearly displayed in the HSV binary image. Blod analysis is performed on both the RGB and HSV binary images to determine the pointer center position and scale positions of the pointer instrument.

[0098] exist Figure 3 In step 303 of the embodiment shown, the center position of the pointer of the pointer instrument is determined through Blod analysis processes such as denoising, connectivity analysis and eigenvalue comparison. Figure 4 The illustrated embodiment is described in detail.

[0099] Reference Figure 4 , determining the center position of the pointer of the pointer instrument includes the following steps:

[0100] 401, denoising the RGB binary image.

[0101] Among them, denoising processing can remove noise in the RGB binary image, thereby avoiding the influence of noise on the connectivity analysis results.

[0102] 402 , performing connectivity analysis on the denoised RGB binary image and identifying multiple connected domains.

[0103] 403. Calculate the eigenvalue of each connected domain.

[0104] Among them, the characteristic values ​​include the area of ​​the connected domain, the maximum length of the connected domain, the maximum width of the connected domain, etc.

[0105] 404 , respectively compare each characteristic value with a preset pointer characteristic value.

[0106] 405 , marking a connected domain having the same eigenvalue as the pointer eigenvalue as a pointer connected domain.

[0107] 406 , determining the center position of the pointer of the pointer instrument according to the center of gravity of the pointer connected domain.

[0108] The implementation principle of this embodiment is:

[0109] Denoising can remove interfering noise points in RGB binary images. Since the pointer is a continuous color area in the RGB binary image, the pointer image will be identified as a whole connected domain in the connectivity analysis. Since the length of the pointer image determines the length of the connected domain, the connected domain representing the pointer can be screened out from all connected domains based on the pointer eigenvalue, and the pointer center position of the pointer instrument can be determined based on the center of gravity of the pointer connected domain.

[0110] exist Figure 3 In step 304 of the embodiment shown, the scale position of the pointer instrument is determined through Blod analysis processes such as denoising, connectivity analysis, and connected domain area screening. Figure 5 The embodiment shown is described in detail.

[0111] Reference Figure 5 , determining the scale position of the pointer instrument includes the following steps:

[0112] 501, denoising the HSV binary image.

[0113] Among them, the denoising process can remove the noise in the HSV binary image, thereby avoiding the influence of the noise on the connectivity analysis results.

[0114] 502 , performing connectivity analysis on the HSV binary image after denoising, and identifying multiple target connected domains.

[0115] 503, respectively calculate the connected domain area of ​​each target connected domain.

[0116] 504 , two target connected domains with the largest connected domain areas are selected from all target connected domains as target scale connected domains.

[0117] 505 , splitting the two target scale connected domains into a start scale connected domain and an end scale connected domain based on preset geometric parameters.

[0118] Among them, when the two target scale connected domains are both in the upper half, the HSV binary image is vertically separated by the preset geometric parameters, and the target scale connected domain in the left half is the starting scale connected domain; when the two target scale connected domains are both in the right half, the HSV binary image is horizontally separated by the preset geometric parameters, and the target scale connected domain in the lower half is the starting scale connected domain.

[0119] 506 , determining the starting scale position and the ending scale position according to the positions of the starting scale connected domain and the ending scale connected domain.

[0120] The implementation principle of this embodiment is:

[0121] Denoising can remove interference noise in HSV binary images. In HSV binary images, each scale is a continuous color area, so in connectivity analysis, each scale image will be identified as a separate connected domain. In the instrument panel of the pointer instrument, the starting scale and the ending scale lines are usually longer or wider than the middle scale lines. Therefore, they can be filtered according to the area of ​​the connected domain. The scale corresponding to the largest connected domain area is the scale at both ends. The two scale connected domains are then distinguished by preset geometric parameters and disassembled into the starting scale connected domain and the ending scale connected domain. Finally, the positions of the starting scale and the ending scale in the reading area are determined according to the positions of the two connected domains.

[0122] exist Figure 1 In step 105 of the embodiment shown, the local search method, polar coordinate search algorithm and non-maximum suppression algorithm are combined to calculate the instrument reading of the pointer instrument. Figure 6 The illustrated embodiment is described in detail.

[0123] Reference Figure 6 Calculating the instrument reading of a pointer instrument based on the scale angle range and the pointer angle includes the following steps:

[0124] 601 : Determine a pointer position of the pointer instrument according to the pointer angle.

[0125] 602 , performing a local whole-scale search on an area adjacent to the pointer position based on the pointer angle and a preset angle threshold to obtain an area adjacent to the pointer.

[0126] The angle critical value is a preset angle value, which is usually larger than the angle between two adjacent whole scales in a standard instrument panel. Assuming that the pointer angle is α and the angle critical value is β, the range of the local whole scale search is [α+β, α-β].

[0127] 603, using a polar coordinate search algorithm to calculate the polar coordinate angles of all whole scales in the pointer's proximity scale area.

[0128] Among them, the structure is used to record the white score value. When using the polar coordinate search algorithm, the search angle range is the local whole scale search range, the search step is a fixed step, and all pixels in the adjacent scale area are traversed. If the color value of the pixel point is 0, the white score value is increased by 1. After the traversal is completed, all whole scales are located according to the white score value, and then the polar coordinate angles of all whole scales are calculated according to the polar coordinate system established with the center position of the pointer circle.

[0129] 604 , analyzing and screening all polar coordinate angles based on a non-maximum suppression algorithm to obtain two target polar coordinate angles.

[0130] Among them, the two whole scales corresponding to the two target polar coordinate angles are closest to the pointer position and are located on both sides of the pointer position.

[0131] 605 , the instrument reading of the pointer instrument is calculated by combining the scale angle range, the pointer angle, and the target polar coordinate angle.

[0132] The implementation principle of this embodiment is:

[0133] First, determine the pointer position, then perform a local whole-scale search based on the pointer position to find the pointer's adjacent scale area. This facilitates reading the integer portion of the instrument reading in subsequent calculations and allows the instrument reading to be accurate to decimals. After searching for the pointer's adjacent scale area, first establish a polar coordinate system and calculate the polar coordinate angles of all the whole-scales near the pointer position using the polar coordinate search algorithm. The decimal portion is between two whole-scales, so after determining the polar coordinate angles of the two nearest whole-scales on both sides of the pointer, a more accurate instrument reading can be calculated by combining the scale range and pointer angle.

[0134] exist Figure 6 In step 605 of the embodiment shown, the whole scale reading and the part scale reading of the pointer instrument are calculated respectively, and then the reading of the pointer instrument is calculated by adding them. Figure 7 The illustrated embodiment is described in detail.

[0135] Reference Figure 7 Calculating the instrument reading by combining the scale angle range, pointer angle, and target polar angle includes the following steps:

[0136] 701, calculate and obtain the whole scale reading of the pointer instrument based on the scale angle range and the pointer angle.

[0137] The maximum range is determined according to the scale angle range. The angle value of the pointer within the scale angle range can be calculated by combining the pointer angle and the scale angle range. The calculated angle value is rounded to obtain the whole scale reading of the pointer instrument.

[0138] 702. Calculate a fractional reading of a pointer instrument based on the target polar coordinate angle and the pointer angle.

[0139] The angle range between two integral scales of the pointer instrument is determined by two target polar coordinate angles, and the non-integer scale reading of the pointer instrument is calculated based on the angle value of the pointer instrument in the angle range between the two integral scales.

[0140] 703, the whole scale reading and the non-integer reading are added together to calculate the instrument reading of the pointer instrument.

[0141] The implementation principle of this embodiment is:

[0142] The maximum range of a pointer instrument can be determined from the scale angle range. Combined with the pointer angle within the scale angle range, the full-scale reading (the integer portion of the instrument reading) can be calculated. Local calculations using the two target polar coordinate angles and the pointer angle can then be performed to calculate the non-integer reading (the decimal portion of the instrument reading). Finally, the integer and decimal portions are added together to obtain the pointer instrument reading.

[0143] The embodiment of the present application also discloses a pointer instrument reading system based on deep learning, including a memory, a processor and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to realize the following Figure 1-Figure 7 The pointer instrument reading method based on deep learning is shown.

[0144] The implementation principle of this embodiment is:

[0145] By calling the program, the image information of the pointer instrument is first collected, and then the image features of the pointer instrument in the image information are deeply extracted through the neural network model, so that the reading area where any type of pointer instrument is located can be located in the image information, and then the reading area is binarized to highlight the pointer features and scale features in the reading area. Even if the pointer length, maximum range, scale position or scale size of different types of pointer instruments are different, the pointer center position and scale position can be determined through Blod analysis, and the scale angle range and pointer angle can be calculated based on the pointer center position and scale position. The scale angle range determines the maximum range of the pointer instrument, and the pointer angle represents the current reading of the pointer instrument. Therefore, the current instrument reading of the pointer instrument can be calculated through the local search method.

[0146] The above are all preferred embodiments of the present application, and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A pointer instrument reading method based on deep learning, characterized in that: The steps include: Collect image information of pointer instruments; Locating a reading area where the pointer instrument exists in the image information using a neural network model; Performing binarization processing and Blob analysis on the reading area to determine the center position of the pointer and the scale position of the pointer instrument; Calculating the scale angle range and the pointer angle based on the scale position and the pointer center position; Obtaining an instrument reading of the pointer instrument by calculating based on the scale angle range and the pointer angle through a local search method; The step of calculating the instrument reading of the pointer instrument based on the scale angle range and the pointer angle by a local search method comprises the following steps: determining a pointer position of the pointer instrument according to the pointer angle; Performing a local whole-scale search on an area adjacent to the pointer position based on the pointer angle and a preset angle threshold to obtain a pointer adjacent scale area; Calculating the polar coordinate angles of all the whole scales in the area adjacent to the scale by using a polar coordinate search algorithm; Analyze and screen all polar coordinate angles based on a non-maximum suppression algorithm to obtain two target polar coordinate angles, where two integral scales corresponding to the two target polar coordinate angles are closest to the pointer position and are located on both sides of the pointer position; The instrument reading of the pointer instrument is calculated in combination with the scale angle range, the pointer angle and the target polar coordinate angle.

2. The pointer instrument reading method based on deep learning according to claim 1, characterized in that: The method of locating the reading area where the pointer instrument exists in the image information by using a neural network model comprises the following steps: Adjusting the size of the image information to obtain standard image information of a standard size; Extracting features of the standard image information using a backbone network with the fully connected layer removed to obtain image features of the standard image information; Performing a convolution operation on the image features through a residual structure to obtain a feature tensor of the standard image information; A reading area where the pointer instrument exists is located in the standard image information according to the feature tensor.

3. The pointer instrument reading method based on deep learning according to claim 1, characterized in that: The binarization and Blob analysis of the reading area to determine the center position and scale position of the pointer instrument include the following steps: Binarization is performed on the reading area in the RGB color space to obtain an RGB binary image; Converting the reading area to an HSV color space, and performing a binarization process on the reading area in the HSV color space to obtain an HSV binarized image; Performing Blob analysis on the RGB binary image to determine the center position of the pointer of the pointer instrument; The Blob analysis is performed on the HSV binary image to determine the scale position of the pointer instrument.

4. The pointer instrument reading method based on deep learning according to claim 3 is characterized in that: The performing of Blob analysis on the RGB binary image to determine the center position of the pointer of the pointer instrument comprises the following steps: Performing denoising on the RGB binary image; Performing connectivity analysis on the RGB binary image after the denoising process and identifying multiple connected domains; Calculate the eigenvalue of each connected domain; Compare each characteristic value with the preset pointer characteristic value respectively; Marking the connected domain with the same eigenvalue as the pointer eigenvalue as a pointer connected domain; The center position of the pointer of the pointer instrument is determined according to the center of gravity of the pointer connected domain.

5. The pointer instrument reading method based on deep learning according to claim 3, characterized in that: The scale position includes a starting scale position and an ending scale position. The performing of the Blob analysis on the HSV binary image to determine the scale position of the pointer instrument includes the following steps: Performing denoising on the HSV binary image; Performing connectivity analysis on the HSV binary image after denoising, and identifying multiple target connected domains; Calculate the connected domain area of ​​each target connected domain respectively; Select two target connected domains with the largest areas from all target connected domains as target scale connected domains; Splitting the two target scale connected domains into a starting scale connected domain and an ending scale connected domain based on preset geometric parameters; The starting scale position and the ending scale position are determined according to the positions of the starting scale connected domain and the ending scale connected domain.

6. The pointer instrument reading method based on deep learning according to claim 1, characterized in that: The step of calculating the instrument reading of the pointer instrument by combining the scale angle range, the pointer angle, and the target polar coordinate angle comprises the following steps: Calculating a whole-scale reading of the pointer instrument based on the scale angle range and the pointer angle; Calculating a non-integer scale reading of the pointer instrument based on the target polar coordinate angle and the pointer angle; The whole scale reading and the non-whole scale reading are added together to obtain the instrument reading of the pointer instrument.

7. A pointer instrument reading system based on deep learning, characterized in that: It includes a memory, a processor, and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to implement a pointer instrument reading method based on deep learning as described in any one of claims 1 to 6.

Images