Meter reading recognition method and system based on multi-strategy fusion and feature screening

Abstract

The application relates to the field of information technology, in particular to a meter reading identification method and system based on multi-strategy fusion and feature screening. The method comprises the following steps: constructing a meter identification framework, wherein the meter identification framework comprises an input module, a reading meter identification module, a liquid column linear meter identification module and a pointer table identification module; the reading meter identification module adopts an OCR engine to perform counting line screening according to the geometric features of a text area for reading meter identification, and then performs character recognition on the counting line to obtain a reading; the liquid column linear meter identification module identifies liquid columns and scale lines based on the horizontal width of each row of color areas for liquid column linear meter identification, performs liquid column range extraction and obtains a reading according to the aligned scale lines; and the pointer table identification module adopts three-level strategies of Hough line segment detection, HSV red detection and contour fitting in sequence to cooperatively perform pointer reading identification.

Description

Technical Field

[0001] This application relates to the field of information technology, specifically to a method and system for meter reading identification based on multi-strategy fusion and feature filtering. Background Technology

[0002] Various types of measuring instruments are widely used in industrial production and daily life, including pointer gauges such as pressure gauges, thermometers, and flow meters; liquid column linear gauges such as glass tube thermometers and level gauges; and mechanical roller digital gauges such as water meters, electricity meters, and gas meters.

[0003] Existing technologies have proposed various automatic meter reading recognition schemes based on image processing, such as using Hough transform to detect pointer angle, using OCR to recognize digital meter readings, and using connected component analysis to extract liquid column height. However, the above schemes still have the following shortcomings in practical applications:

[0004] First, the identification mode is limited. Existing solutions typically design algorithms for a single type of meter, making it impossible to handle pointer meters, liquid level meters, and digital meters simultaneously within the same system. Users need to switch between multiple dedicated tools when dealing with various meter types, which is cumbersome.

[0005] Secondly, digital meter readings are easily influenced by the accompanying text. The dial images of mechanical counters such as water meters and electricity meters often include, in addition to the roller reading, the model number, technical parameters (e.g., "220V, 1200imp / kWh"), and unit markings (e.g., "m"). 3 The text includes descriptive text such as "kWh" and production number. When conventional OCR engines perform text recognition on the entire image, they tend to merge the counter number with the adjacent descriptive text into a single text block, leading to misrecognition. For example, a counter with an actual reading of "00422" might be misrecognized as "100422" or "00422-1".

[0006] Third, pointer detection has poor robustness. Existing pointer detection methods mostly employ a single strategy, such as pure Hough line segment detection or pure contour fitting. However, meter images in industrial settings often present complex situations such as dial reflections, scale wear, low contrast, and similar colors of red and black pointers. A single strategy cannot cover all scenarios, leading to missed pointer detections or significant angular deviations.

[0007] Fourth, the liquid column and the scale are difficult to distinguish. In liquid column gauges such as glass tube thermometers and level gauges, the liquid column is a wide, filled colored area (usually red), while the scale lines are only thin lines. Although they are similar in color, they are significantly different in geometric shape. Conventional connected component analysis or area thresholding methods distinguish them based solely on the area of ​​the region, which can easily lead to misinterpreting fragments from a broken liquid column as scale lines, or merging multiple scale lines into a liquid column, making it difficult to achieve stable differentiation.

[0008] Therefore, it is necessary to study and improve meter reading recognition technology. Summary of the Invention

[0009] This specification describes a meter reading identification method and system based on multi-strategy fusion and feature filtering through several embodiments.

[0010] Firstly, the embodiments of this specification provide a meter reading identification method based on multi-strategy fusion and feature filtering, including the following steps:

[0011] A meter identification framework is constructed, which includes an input module, a reading meter identification module, a liquid column linear meter identification module, and an pointer meter identification module. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module.

[0012] The meter reading recognition module uses an OCR engine to filter count lines based on the geometric features of the text area, and then performs character recognition on the count lines to obtain the reading.

[0013] The liquid column linear meter identification module identifies the liquid column and scale lines based on the horizontal width of each row of color areas, extracts the liquid column range, and obtains the reading based on the aligned scale lines.

[0014] The pointer meter recognition module employs a three-level strategy—Hough line segment detection, HSV red detection, and contour fitting—to collaboratively recognize pointer readings.

[0015] Secondly, embodiments of this specification provide a meter reading identification system based on multi-strategy fusion and feature filtering, including:

[0016] The module constructs a meter identification framework, which includes an input module, a reading meter identification module, a liquid column linear meter identification module, and a pointer meter identification module. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module.

[0017] The meter reading recognition module uses an OCR engine to filter count lines based on the geometric features of the text area, and then performs character recognition on the count lines to obtain the reading.

[0018] The liquid column linear meter identification module identifies the liquid column and scale lines based on the horizontal width of each row of color areas for liquid column linear meter identification, extracts the liquid column range, and obtains the reading based on the aligned scale lines.

[0019] The pointer meter recognition module employs a three-level strategy—Hough line segment detection, HSV red detection, and contour fitting—to collaboratively recognize pointer readings.

[0020] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0021] By constructing a unified identification framework, the identification of three types of meters—pointer meters, liquid column linear meters, and digital meters—is integrated into the same system. The same image reading, preprocessing, and result display process can be reused by switching modes, avoiding the cumbersome operation of switching between different dedicated tools and significantly improving the convenience of the automated meter reading process.

[0022] To address the issue of misidentification caused by OCR merging counter digits with explanatory text, an innovative high-threshold bright area cropping algorithm for abnormally long readings is introduced. When an abnormally long reading is detected, multi-threshold step binarization and bright area cropping are automatically triggered, and the background explanatory text is removed before re-identification. This secondary verification of the OCR results from a geometric perspective reduces over-reliance on OCR confidence and improves the reliability of the recognition results.

[0023] It employs a three-level backtracking mechanism: Hough line segment detection, HSV red detection, and contour fitting. These are executed sequentially, and if the previous level fails, it automatically backtracks to the next level, covering typical complex scenarios such as black pointers, red pointers, low contrast, dial wear, and reflections.

[0024] Based on the geometric prior that the liquid column is a wide filling region (large horizontal span) and the scale lines are thin line marks (small horizontal span), it does not depend on the complete connectivity of the liquid column region. Even if there are breaks in the liquid column or local missing parts caused by reflection, it can still accurately extract the vertical range of the liquid column by scanning line by line. It is more robust than the traditional area threshold method or connected domain analysis method.

[0025] Other features and advantages of various embodiments of this specification will be further revealed in the following detailed description and accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram for identifying meter readings provided in this manual.

[0028] Figure 2This is a schematic diagram of the meter reading identification method provided in this manual.

[0029] Figure 3 This is a schematic diagram of the pre-set first identification sub-strategy method provided in this specification.

[0030] Figure 4 This is a schematic diagram of the pre-set second identification sub-strategy method provided in this specification.

[0031] Figure 5 This is a schematic diagram of the method for identifying liquid columns and graduation lines provided in this manual.

[0032] Figure 6 This is a schematic diagram of the pointer reading recognition method provided in this manual.

[0033] Figure 7 This is a schematic diagram of the meter reading recognition system provided in this manual. Detailed Implementation

[0034] The technical solutions of the embodiments of this specification will be explained and described below with reference to the accompanying drawings. However, the following embodiments are only preferred embodiments of this specification and not all of them. Other embodiments obtained by those skilled in the art based on the embodiments in the implementation methods without creative effort are all within the protection scope of this specification.

[0035] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0036] In the following description, terms such as “inner,” “outer,” “upper,” “lower,” “left,” and “right” are used only to facilitate the description of the embodiments and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this specification.

[0037] All data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0038] Before introducing the technical solutions described in this manual, the application scenarios and related technologies of the technical solutions will be introduced.

[0039] Various types of metering instruments are widely used in industrial production and daily life to monitor key physical quantities such as pressure, temperature, flow rate, electricity, water consumption, and gas consumption. Apart from IoT instruments, metering instruments mainly include pointer instruments, liquid column linear meters, and mechanical roller digital meters.

[0040] Automatic meter reading recognition technology based on computer vision is gradually becoming a technological development trend for reading these non-IoT meters.

[0041] The basic process of automatic meter reading recognition technology based on computer vision typically includes image acquisition, image preprocessing, dial area detection and localization, key information extraction (pointers, scales, numbers, etc.), and reading calculation and output. The technical approach used varies depending on the type of meter.

[0042] However, in actual equipment, multiple meters belonging to different types may appear in the same image acquisition area. Identifying these meters requires switching between multiple dedicated recognition models. Furthermore, since image sampling is not targeted at a single meter, it is difficult to align the image directly with each meter. Therefore, recognition algorithms are needed to improve recognition coverage and accuracy.

[0043] To adapt to the identification of multiple meter readings in the field using computer vision, this embodiment provides a meter reading identification method and system based on multi-strategy fusion and feature filtering. Please refer to the appendix. Figure 1 By constructing a unified meter identification framework, the framework includes an input module, a reading meter identification module, a liquid column linear meter identification module, and a pointer meter identification module.

[0044] The input module is responsible for receiving the meter image to be identified and identifying the type of meter it belongs to. Then, based on the identification result, the image is distributed to the corresponding pointer meter identification module, liquid column linear meter identification module, or reading meter identification module for processing.

[0045] For the recognition of reading meters (i.e. mechanical roller digital meters), the reading meter recognition module uses an OCR engine to perform text detection on the entire image and filters the counter rows based on the geometric features of the text area to accurately select the actual counter reading area from numerous OCR recognition results. Then, it performs text recognition on the area to obtain the final reading.

[0046] For the identification of liquid column linear meters, the liquid column linear meter identification module distinguishes between the liquid column and the scale line based on the horizontal width of the color area in each row. Since the liquid column is a wide filled area while the scale line is a thin line mark, the two show a significant difference in the horizontal span of each row. Based on this geometric prior, the range of the liquid column can be accurately extracted and the reading can be obtained according to the aligned scale line.

[0047] For pointer meter recognition, the pointer meter recognition module uses a three-level strategy of Hough line segment detection, HSV red detection and contour fitting to recognize pointer readings in sequence. When the previous level strategy fails, it automatically falls back to the next level to ensure that pointer positioning and reading calculation can still be stably completed in complex scenarios such as reflection, wear and tear and low contrast.

[0048] This embodiment achieves automated reading recognition of three main types of meters—pointer meters, liquid column linear meters, and digital meters—within the same system through the aforementioned unified framework and multi-strategy fusion mechanism.

[0049] Specifically, this embodiment first provides a meter reading identification method based on multi-strategy fusion and feature filtering. Please refer to the appendix. Figure 2 ,include:

[0050] Step S1: Construct a meter identification framework. The meter identification framework includes an input module, a reading meter identification module, a liquid column linear meter identification module, and a pointer meter identification module. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module.

[0051] The input module receives the meter images to be identified and automatically determines the type of meter in the image. Based on the visual characteristics of the meter image, such as the dial shape (round dial, rectangular LCD screen, or glass tube structure), and whether it has pointers or roller counters, the image is classified into one of three types: pointer meter, liquid column linear meter, or digital meter. After completing the type identification, the input module distributes the image or image region to the corresponding recognition module according to the classification result: pointer meter images are sent to the pointer meter recognition module, liquid column linear meter images are sent to the liquid column linear meter recognition module, and digital meter images are sent to the digital meter recognition module. When an image contains multiple meters, the input module segments the image and sends the image regions containing the meters to the pointer meter recognition module, liquid column linear meter recognition module, or digital meter recognition module respectively. If a meter cannot be identified or its type cannot be determined due to the image viewing angle, the image region is discarded, and the corresponding region from another image viewing angle completes the meter identification. Each meter in the image should have a unique code for identification, so that the identified reading is associated with that unique code for subsequent data processing.

[0052] Step S2: The meter reading recognition module uses an OCR engine to filter count lines based on the geometric features of the text area for meter reading recognition, and then performs character recognition on the count lines to obtain the reading.

[0053] When the reading obtained by the reading meter recognition module through full-image text detection exceeds a preset number of digits, it is determined to be an abnormally long reading.

[0054] Extract the grayscale ROI corresponding to the text block, calculate the adaptive threshold ladder based on the ROI brightness range, use a fixed threshold set when the brightness range is less than the preset brightness threshold, and generate multiple candidate thresholds proportionally and truncate them to the preset range when the brightness range is not less than the preset brightness threshold.

[0055] Each candidate threshold is binarized, the bounding rectangle of the bright area is extracted, and then filled with pixels around it, cropped and enlarged. OCR recognition is called again, and only the results with numbers below a preset number of digits are accepted as candidate results.

[0056] A majority vote is conducted on all candidate results, and the reading with the most votes and the highest confidence level is selected as the final counter reading.

[0057] The integrated OCR engine is invoked to perform full-image text detection on the meter image, obtaining all text blocks in the image and their corresponding text bounding boxes and recognition confidence scores. For each text block detected by the OCR, if the number of digits in its recognition result exceeds the preset number of digits (normal counter readings are usually 4 to 5 digits, so the preset number of digits can be set to 5 digits, i.e., 6 digits and above are considered abnormal), then the text block is determined to be an abnormally long reading, triggering a high-threshold bright area cropping process.

[0058] Unusually long readings typically originate from the OCR engine comparing the mechanical counter digits with adjacent descriptive text on the dial (such as model number "LXS-20E", parameter "220V 1200imp / kWh", or unit designation "m"). 3 (e.g., "etc.") are incorrectly merged into the same text block. To separate the counter numbers from the merged block, the grayscale ROI corresponding to the text block is extracted, and an adaptive threshold ladder is calculated based on the brightness distribution range of the ROI.

[0059] Specifically, when the brightness range of the ROI is less than the preset brightness threshold (e.g., 30 gray levels), it indicates that the image contrast is low. In this case, a set of fixed thresholds (e.g., [150, 170, 190]) are used for binarization. When the brightness range is not less than the preset brightness threshold, multiple candidate thresholds are generated according to a specific ratio of the minimum brightness value to the brightness range (e.g., 0.62, 0.70, 0.78, 0.86), and each threshold is truncated to a preset effective range (e.g., [130, 215]).

[0060] For each candidate threshold, binarization is performed on the ROI, extracting the bounding rectangle of the high-brightness region (i.e., the brightest foreground text region). The rectangle is then expanded by several pixels (e.g., 4 pixels) around its perimeter, and a local image is cropped. This image is then enlarged proportionally and re-applied for OCR recognition. Only results that yield 4 to 5 pure digits after re-recognition are accepted as valid candidates. For multiple candidate results generated from all candidate thresholds, a majority voting mechanism is implemented: the frequency of each digit in each candidate reading is counted, and the candidate with the highest number of votes is selected. If multiple candidates have the same number of votes, the OCR confidence of each candidate is further compared, and the candidate with the highest confidence is selected as the final counter reading for that text block. The cropping strategy effectively removes explanatory text attached to the counter digits, restoring the true 4 to 5 digit counter reading.

[0061] After cropping abnormally long readings, all text blocks containing numbers in the image are comprehensively evaluated, and the text lines most likely belonging to the counter readings are selected.

[0062] Methods for filtering lines by counting based on the geometric features of text regions using an OCR engine include:

[0063] Please see the appendix Figure 3 The first pre-defined identification sub-strategy includes:

[0064] Step S211: Obtain the meter image area and obtain multiple text areas within the meter image area output by the OCR engine.

[0065] Step S212: Filter text areas where the number of digits is within the preset range [N1, N2], the height of the text block occupies the height of the meter image area in a ratio between 0.02 and 0.35, and the aspect ratio is not less than 0.5.

[0066] Step S213: Remove text regions with a number of digits ≥ N3 and a height ratio > 0.12.

[0067] Step S214: Determine the digit score based on how close the number of digits is to the midpoint of the preset interval [N1, N2], determine the size score based on how close the text block height is to 0.185, determine the aspect ratio score based on how close the aspect ratio is to 0.5, and obtain the confidence score based on the confidence score corresponding to the text area output by the OCR engine.

[0068] Step S215: The weighted sum of the score by number of digits, the score by size, the score by aspect ratio, and the confidence score is used as the comprehensive score of the text region. The text region with the highest comprehensive score is used as the counting row.

[0069] The first recognition sub-strategy is used to handle cases where the counter digits are not split into multiple independent text blocks by the OCR. It acquires the meter image region and all text block information within that region output by the OCR engine. Each text block contains its recognized text content, bounding box coordinates (x, y, width, height), and confidence score.

[0070] All text blocks undergo initial screening, retaining those that simultaneously meet the following conditions: the number of digits in the recognized text falls within the preset range [N1, N2] (since normal counter readings are typically 4 to 5 digits, N1 can be 4 and N2 can be 5, but considering potential omissions or false detections by OCR, the actual range can be appropriately widened to 3 to 7 digits); the ratio of the text block height to the meter image area height is between 0.02 and 0.35 (this range covers common sizes from small to large numbers); and the aspect ratio of the text block is not less than 0.5 (ensuring the text block has a reasonable horizontal extension shape and excluding excessively long and narrow abnormal areas). Based on the initial screening, further remove text blocks with ≥N3 digits (e.g., N3=6) and a height ratio >0.12. These text blocks are usually abnormally long text blocks after merging explanatory text and should not be considered as candidates for counter rows.

[0071] After screening and elimination, a comprehensive score is calculated for each candidate text block. This score is obtained by weighted summation of four components: the digit count score, determined by how close the number of digits in the text block is to the midpoint of a preset interval [N1, N2] (i.e., the typical counter digit count); the closer the number of digits is to the midpoint of the interval, the higher the score; the size score, determined by how close the text block height is to 0.185 times the image height, where 0.185 is the statistical median of the proportion of counter row height in a large number of meter images; the aspect ratio score, determined by how close the aspect ratio of the text block is to 0.5; and the confidence score, directly taken from the recognition confidence of the text block output by the OCR engine. Each of the four components is multiplied by a preset weight and then summed to obtain the comprehensive score of the text block. The text block with the highest comprehensive score is selected as the counting row for this sub-strategy.

[0072] For example, the OCR simultaneously detected the text block "00322" (5 digits, height ratio approximately 0.18, aspect ratio approximately 0.6) and the text block "LXS-20E" (containing letters, not meeting the pure numeric condition). The former scored higher in all dimensions and was selected as the counting line; while the unit identifier "m" below "00322"... 3 "Although it is a combination of numbers and letters, it has been excluded in the pure number of digits selection criteria."

[0073] Please see the appendix Figure 4 A second pre-defined identification sub-strategy includes:

[0074] Step S221: Perform center clustering on the y-axis position for all text blocks of type number. The tolerance of clustering is 0.8 times the median height and not less than the preset minimum pixel height.

[0075] Step S222: Group the clusters according to their x-axis adjacency. The number of digits in each group is within a preset range [N1, N2], and the aspect ratio of each group is between 1.2 and 15.

[0076] Step S223: Determine the group digit score based on the degree to which the number of digits in the group is close to the midpoint of the preset interval [N1, N2], determine the height consistency score based on the consistency of the height of the digital text blocks in the group, determine the group width-to-height ratio score based on the degree to which the group's aspect ratio is close to 8.1, and obtain the group confidence score based on the confidence level corresponding to the text area output by the OCR engine.

[0077] Step S224: The weighted sum of the group number score, high consistency score, group width-to-height ratio score, and group confidence score is used as the comprehensive group score of the text region. The text region with the highest comprehensive group score is used as the counting row.

[0078] The second recognition sub-strategy is used to handle situations where the counter number is split into multiple independent number blocks by the OCR. Since the OCR engine may separate adjacent numbers when detecting text, for example, splitting the counter reading "00422" into two independent text blocks "00" and "422", the single-block filtering of the first sub-strategy cannot obtain the complete counter reading, and the second sub-strategy is required for merging processing.

[0079] All numeric text blocks are clustered based on their y-axis center position in the image: text blocks whose y-center coordinates differ from each other by less than a certain tolerance value are grouped into the same cluster. The tolerance is 0.8 times the median height of all numeric text blocks, and not less than a preset minimum pixel height (e.g., 10 pixels), to ensure that numbers in adjacent rows are not incorrectly clustered together.

[0080] For each cluster, the text blocks are further grouped according to their adjacent positions on the x-axis. Numerical text blocks that are close to each other in the x-direction and have no significant gaps are merged into the same group. After grouping, only groups that meet the following conditions are retained for subsequent scoring: the total number of digits after concatenating all text blocks in the group is within the preset interval [N1, N2] (3 to 7 digits are also acceptable); the overall aspect ratio of the group is between 1.2 and 15 (reflecting the geometric characteristics of the horizontal extension of the counter line).

[0081] For each candidate group, a comprehensive group score is calculated. The comprehensive group score is obtained by weighted summation of four components: the group digit count score, determined by how close the total number of digits within the group is to the midpoint of a preset interval [N1, N2]; the height consistency score, reflecting the consistency of the height of each digit text block within the group, with higher scores awarded for blocks of similar height to ensure that merged digits belong to the same line of text; the group aspect ratio score, determined by how close the overall aspect ratio of the group is to 8.1 (8.1 is a typical value for the overall aspect ratio of a mechanical counter line); and the group confidence score, which is the average of the confidence scores of each text block within the group. The comprehensive group score is obtained by multiplying the four components by their respective weights and summing them. The group with the highest comprehensive group score is selected as the counting line for that sub-strategy.

[0082] Choose between the single-block result of the first sub-strategy and the grouping result of the second sub-strategy.

[0083] When the overall group score is not higher than a preset multiple of the overall score, the count row identified by the first identification sub-strategy is used; when the overall group score is higher than a preset multiple of the overall score, the count row identified by the second identification sub-strategy is used.

[0084] Through the collaborative screening of the above two-level sub-strategies, the system adapts to both complete and split detection of counter digits by OCR, accurately screening out the actual counter reading lines. Then, text recognition is performed on the counting line area to obtain the final reading result.

[0085] The reading meter identification also includes a mechanical counter identification method, which includes:

[0086] Obtain the meter image region and perform binarization processing. Filter out digit-shaped contours and perform center clustering based on the y-direction position. Remove clusters with fewer than 3 digit-shaped contours after clustering or clusters whose aspect ratio is not within the interval [1.5, 15].

[0087] The binarized meter image region is connected by performing a wide-kernel horizontal closure operation in both inverted and normal phases to connect broken strokes, and connected components with aspect ratios in the range [1.5, 15] and rectangularity not less than 0.3 are selected.

[0088] In the HSV space, high-brightness, low-saturation white regions are selected, and wide-kernel closed-loop operations are performed to connect them. Regions with aspect ratios within the range [2, 12], rectangularity greater than 0.5, and internal Canny edge density between [0.03, 0.5] are selected.

[0089] For detected counter ROIs, OCR recognition is called first. If OCR fails, it falls back to the KNN classifier based on equal-width segmentation for mechanical counter recognition.

[0090] If OCR full-image detection and geometric feature collaborative screening fail to obtain effective counter readings, the meter reading recognition module is also equipped with a dedicated mechanical counter recognition process as a fallback solution. The counter region is located using three methods: digital row detection, connected component analysis, and color-assisted analysis. OCR recognition is prioritized for the located region; if OCR fails, the process falls back to a segmentation recognition scheme based on a KNN classifier.

[0091] First, the digit row detection method is used.

[0092] The meter image region is acquired and binarized. To accommodate meter images with different contrast orientations, multiple binarization methods are employed, including Otsu's normal thresholding, Otsu's inverted thresholding, adaptive normal thresholding, and adaptive inverted thresholding. For each binarization result, all contours in the image are extracted, and contours that may belong to numerical characters are selected based on their geometric features.

[0093] The filtering criteria include: the height of the outline as a percentage of the meter image area is between 0.02 and 0.95 (ensuring that the numeric characters are not too large or too small); the width of the outline is not less than 3 pixels and does not exceed 25% of the image width; the aspect ratio of the outline is between 0.2 and 1.5 (numeric characters are generally approximately square or slightly tall rectangles); the area of ​​the outline is not less than 0.05% of the meter image area; and the fill rate of the outline (the ratio of the outline area to the area of ​​its circumscribed rectangle) is between 0.15 and 0.95 to exclude areas that are too irregular in shape or too hollow.

[0094] All the numerical contours obtained after filtering are clustered according to their center coordinates in the y-direction.

[0095] The clustering tolerance is set to 0.6 times the median height of all candidate profiles to ensure that numbers in the same row are grouped into the same cluster, while numbers in different rows are separated. After clustering, each cluster is evaluated for quality, and clusters with fewer than 3 numeric profiles (counters typically have at least 3 digits) are removed. Clusters whose overall aspect ratio is outside the range [1.5, 15] are also removed (counter rows should be horizontally extending rectangular bands). For clusters that pass the evaluation, the uniformity of spacing between numeric profiles within the group is further assessed. The spacing between digits in a standard counter should be uniform; therefore, the smaller the standard deviation of the x-direction spacing between adjacent numeric profiles, the more likely the cluster is to be a counter row.

[0096] The cluster is scored based on multiple factors, including the number of digit contours, the aspect ratio of the cluster regions, the uniformity of spacing, and the area consistency of the digit contours themselves. The expanded bounding box corresponding to the highest-scoring cluster is taken as the counter ROI. The upper and lower boundaries of the ROI are determined based on the y-direction range of all digit contours within the cluster, while the left and right boundaries are appropriately extended outward from the leftmost and rightmost digit contours to ensure complete inclusion of all digit characters.

[0097] Secondly, the connected component method is adopted.

[0098] When the digital line detection method fails to effectively locate the counter region, it reverts to the connected component method for localization. Adaptive threshold binarization is performed on the grayscale image, and then processing is applied separately to the inverted and normal images to simultaneously adapt to both white-background black text and dark-background bright text display modes. For each binarized image, a wide-kernel horizontal closing operation is used to connect broken strokes caused by wear or uneven lighting. The width of the horizontal closing kernel is taken as the larger value between 1 / 20 of the image width and 15 pixels, and the height is taken as the larger value between 1 / 50 of the image height and 3 pixels. This ensures that the kernel shape has sufficient horizontal extension to connect adjacent broken parts in the horizontal direction, while remaining relatively narrow in the vertical direction to avoid cross-line adhesion.

[0099] After the closing operation is completed, all connected components are extracted and filtered according to the following conditions:

[0100] The aspect ratio of the bounding rectangle of the connected region is between 1.5 and 15 to match the overall horizontal extension shape of the counter row; the area of ​​the connected region is between 0.2% and 75% of the area of ​​the meter image region; the rectangularity of the connected region (the ratio of the area of ​​the connected region to the area of ​​its bounding rectangle) is not less than 0.3 to ensure that the shape of the connected region is close to that of a rectangle.

[0101] For connected components that meet the above conditions, their scores are further calculated. The scores take into account factors such as the degree to which the aspect ratio of the connected component is close to that of a typical counter row, the degree of rectangularity, and the size. The bounding rectangle of the connected component with the highest score is taken as the counter ROI.

[0102] Finally, a color-assisted method was used.

[0103] When the connected component method also fails to locate the counter region, a fallback to the color-assisted method is adopted. The visual characteristic of mechanical counter regions typically presenting a high-brightness, low-saturation white background is utilized for localization. The image is converted to the HSV color space, and pixels satisfying a saturation below 60 and a brightness value greater than 150 are selected to obtain a binary mask for the white background region. A wide kernel closing operation is then performed on this mask to connect the scattered white regions into complete connected components. Within these connected components, candidate regions with aspect ratios between 2 and 12 and a rectangularity greater than 0.5 are selected. To further verify whether the selected regions are valid counter regions, the Canny edge density within these regions is calculated. Multiple numeric characters within the counter row region produce edge responses of a certain density, while solid-color background regions have extremely low edge density. Therefore, only candidate regions with an internal Canny edge density between 0.03 and 0.5 are retained, excluding both background regions without text and non-counter regions with excessively dense text. The bounding rectangle of the region satisfying all conditions is taken as the counter ROI.

[0104] Once the counter ROI is located using any of the methods—digital line detection, connected component analysis, or color-assisted analysis—the OCR engine is preferentially invoked to perform text recognition on that ROI. If the OCR recognition is successful and yields a 3- to 7-digit result, this result is output as the final counter reading. If the OCR recognition fails or the number of digits in the result is outside the valid range, the process reverts to a segmentation and recognition scheme based on a KNN classifier.

[0105] The method for training the KNN classifier includes:

[0106] For each number, training images are generated using multiple fonts, weights, and sizes. These images are then uniformly scaled to a preset size and flattened into feature vectors.

[0107] An initial classifier is created using a preset architecture and trained using the feature vectors;

[0108] When configuring predictions, k-nearest neighbor voting is used, and distance is used as the confidence level.

[0109] The segmentation and recognition scheme first binarizes the Region of Interest (ROI), employing four methods—Otsu thresholding binarization, Otsu inversion binarization, adaptive thresholding binarization, and adaptive inversion binarization—to handle different contrast modes. For each binarization result, a segmentation method based on vertical projection is first attempted: the binarized image is projected vertically, the number of foreground pixels in each column is counted, and the segmentation points between each digit character are determined based on the valley positions of the projection histogram, thereby segmenting the ROI into individual digit image blocks.

[0110] If the number of segments obtained by the vertical projection method is not within the effective range of 3 to 7 bits, then the equal-width segmentation method is used instead: according to the preset counter bit range (3 to 7 bits), the ROI is evenly divided into a corresponding number of equal-width blocks in the horizontal direction, and each block is a digital image block. All segmented digital image blocks are uniformly scaled to a preset size (e.g., 20 pixels × 20 pixels) and flattened into feature vectors for input into the KNN classifier for recognition.

[0111] The KNN classifier is pre-trained. In this embodiment, the training data is generated synthetically rather than relying on real sample collection: for each digit from 0 to 9, four fonts provided by OpenCV, namely HERSHEY_SIMPLEX, HERSHEY_PLAIN, HERSHEY_DUPLEX and HERSHEY_TRIPLEX, are used to generate digital images with three line thicknesses of 1, 2, and 3 and five sizes of even-numbered pixels between 24 and 32.

[0112] Each combination of font, weight, and size generates a training image for that digit, resulting in 4 × 3 × 5 = 60 training samples per digit. All training samples are uniformly scaled to a preset size (e.g., 20 pixels × 20 pixels) and flattened into a one-dimensional feature vector, with each sample having a feature dimension of 400. A classifier is created using the KNearest class in OpenCV's machine learning module, trained using the aforementioned synthetic dataset. After training, the classifier internally stores the feature vectors of all training samples and their corresponding digit labels.

[0113] During the prediction phase, the KNN classifier receives the feature vector of the input digital image, finds the k nearest samples (k=5) from the training set that are closest to the input feature, performs a majority vote on the labels of these 5 samples, and outputs the number with the most votes as the prediction result. Simultaneously, the distance between the nearest neighbor sample and the input feature is used as the confidence level of the prediction result. When this confidence level is lower than a preset threshold, the recognition result is deemed unreliable and a prompt message is output for user manual confirmation.

[0114] Step S3: The liquid column linear meter recognition module identifies the liquid column and scale lines based on the horizontal width of each color area, extracts the liquid column range, and obtains the reading based on the aligned scale lines.

[0115] The liquid column linear meter identification module is used to identify meters such as glass tube thermometers and level gauges that use the height of the liquid column to indicate the reading.

[0116] Liquid columns and graduation lines are usually very similar in color (e.g., both are red), but they are fundamentally different in geometry. Liquid columns are filled areas of a certain width, while graduation lines are just thin line marks.

[0117] Based on this geometric prior, the method of statistically analyzing the horizontal span of the color region row by row is used to accurately distinguish the liquid column from the scale line, thereby extracting the vertical range of the liquid column and calculating the reading based on the aligned scale line.

[0118] Please see the appendix Figure 5 The method for identifying the liquid column and scale lines based on the horizontal width of each row of color areas includes:

[0119] Step S31: Obtain the meter image area, extract the liquid column area in the HSV space, scan the color area of ​​each row of the image, and count the maximum continuous color horizontal span of each row.

[0120] The meter image region is acquired, and the image is converted from the BGR color space to the HSV color space. The color distribution characteristics of the liquid column in the HSV space are then used for extraction. Taking a red liquid column as an example, a dual-segment red threshold is used for extraction: hue H is located in either the 0-10 or 160-180 range, saturation S is not less than 50, and brightness V is not less than 50. Pixels meeting these threshold conditions are marked as foreground (liquid column candidate pixels), and the remaining pixels are marked as background. To fill in any small voids or breaks that may exist within the liquid column region (e.g., local defects caused by liquid column reflection or bubbles), a vertical morphological closing operation is performed on the extracted binary image using a 5-pixel × 1-pixel kernel, iterated twice. This operation can connect the broken parts of the liquid column in the vertical direction without significantly changing the horizontal width characteristics of the liquid column.

[0121] After completing the above processing, the binary image is scanned line by line. For each line in the image, the length of all consecutive foreground pixel segments in that line is counted, and the maximum value is recorded as the maximum consecutive color horizontal span of that line.

[0122] Taking a row of pixels as an example, if the foreground pixels in that row are distributed as 5 consecutive red pixels, 3 background pixels, and 10 consecutive red pixels, then the maximum horizontal span of the consecutive colors in that row is 10. A corresponding horizontal span value is recorded for each row, thus obtaining the horizontal span sequence of the entire image distributed by rows.

[0123] Step S32: Designate rows with a horizontal span greater than or equal to the preset width as liquid column rows, and rows with a horizontal span less than the preset width as scale line rows.

[0124] The type of a row is determined based on the maximum horizontal span of consecutive colors in each row.

[0125] Liquid columns are wide, filled areas, with each row spanning their width having a substantial horizontal width (e.g., tens of bytes or even hundreds of pixels); while scale lines are thin, line-like markers, typically only 1 to 3 pixels wide horizontally. Based on this difference, a preset width threshold (e.g., 4 pixels) is set. For each row, if its maximum continuous color horizontal span is greater than or equal to this preset width, the row is classified as a liquid column row, considered to be within the vertical range covered by the liquid column; if its maximum continuous color horizontal span is less than this preset width, the row is classified as a scale line row, considered to be within the scale line's coverage area rather than the liquid column's coverage area.

[0126] Taking an image of a red glass tube thermometer as an example, the area where the liquid column is located occupies dozens of consecutive rows from the middle to the bottom of the image. Each row has a relatively wide continuous red segment (for example, a horizontal span of up to 25 pixels), so these rows are all classified as liquid column rows; while in the red scale line area at the top of the image, each row contains only a red line segment that is 1 to 2 pixels wide, so these rows are classified as scale line rows.

[0127] Step S33: Take the maximum continuous range of the liquid column as the vertical range of the liquid column.

[0128] After classification, a series of row numbers are obtained, labeled as liquid column rows. These liquid column rows may be continuously concentrated in the vertical direction (corresponding to the main body of the liquid column), or there may be a small number of isolated liquid column rows (corresponding to image noise or fragments caused by liquid column breakage).

[0129] Extract the maximum continuous range of the liquid column line, that is, the longest continuous segment among the liquid column lines that appear continuously in the vertical direction.

[0130] Specifically, all rows are traversed from smallest to largest row number. Adjacent rows with consecutive row numbers (row number difference of 1) are grouped into the same continuous segment. The starting and ending row numbers of each continuous segment, as well as the number of rows it contains, are recorded. The continuous segment with the most rows is taken as the vertical range of the liquid column, with its starting row number indicating the top position and its ending row number indicating the bottom position. By taking the largest continuous range instead of all liquid column rows, isolated noise rows outside the main body of the liquid column (such as small fragments scattered after the liquid column breaks or stray red pixels in the image) can be effectively filtered out, ensuring that the extracted vertical range of the liquid column accurately reflects the continuous height of the real liquid column.

[0131] For horizontally installed liquid column gauges (such as horizontally placed level gauges), a symmetrical processing logic is adopted: first, the installation direction is determined by judging the size of the outer rectangle of the largest connected contour. If the height of the outer rectangle of the largest connected contour is greater than 1.2 times the width, it is determined to be a vertical mode; otherwise, it is determined to be a horizontal mode.

[0132] In horizontal mode, the process is changed to column-by-column scanning. The maximum vertical span of the continuous color in each column is counted. Columns with a vertical span greater than or equal to the preset width are taken as liquid column columns. The maximum continuous range of the liquid column column is taken as the horizontal range of the liquid column. Then, the horizontal range of the liquid column is mapped to the reading.

[0133] After extracting the liquid column range, the final reading result is calculated and output based on the proportion of the liquid column vertical range to the tube vertical range and the corresponding reading range of the tube vertical range. The method for determining the tube vertical range will be explained later.

[0134] On the other hand, methods for obtaining readings based on aligned scale lines include:

[0135] Attempt to identify readings within a preset distance range of the aligned scale lines; if a reading is identified, use that reading as the reading of the aligned scale line.

[0136] If no reading is identified, the vertical range of the tube is identified. Based on the proportion of the vertical range of the liquid column to the vertical range of the tube and the reading range corresponding to the vertical range of the tube, the reading of the aligned scale line is obtained.

[0137] There are two methods for obtaining readings based on aligned scale lines.

[0138] The first approach involves extracting the vertical range of the liquid column and then attempting to identify the scale values ​​near the graduation line aligned with the top of the liquid column using OCR. Specifically, within a preset distance range at the top of the liquid column (e.g., a certain number of pixels above and below the top of the liquid column), the OCR engine searches for scale text blocks that it can recognize. If valid numerical text (such as 10, 20, 30, etc.) is detected within this range, the number is used as the reading for that graduation line, and the corresponding temperature or liquid level value is directly output. For example, using an image of a thermometer with a range of −20°C to 50°C, if the OCR recognizes the number 30 near the top of the liquid column, the current temperature can be directly determined to be 30°C without further calculation of the scale. This method can quickly obtain accurate readings when the scale numbers are clear and the OCR recognition is successful.

[0139] The second approach, if the graduations cannot be identified near the top of the liquid column, involves a proportional calculation based on the vertical range of the tube. First, the effective graduation range of the glass tube (i.e., the vertical range) needs to be determined. Then, based on the proportion of the liquid column's vertical range within this vertical range, and combined with the corresponding physical reading range, the current reading is calculated.

[0140] The determination of the vertical range of the tube employs a strategy of "basic estimation and multi-source extension" to ensure that the final tube range not only completely encompasses the liquid column but also accurately reflects the effective graduation range of the glass tube. Specifically, the methods for identifying the vertical range of the tube include:

[0141] The base area is defined by the vertical range of the meter image area from 2% down from the top to 5% down from the bottom of the liquid column.

[0142] A relaxed HSV threshold is used to expand from the liquid column region, and Canny edge detection is used to detect the glass tube wall and the top and bottom of the tube are determined based on the longest continuous edge segment.

[0143] The vertical range of the pipe is obtained based on the expanded area.

[0144] Based on the bottom position of the liquid column and the overall height of the meter image, a conservative initial tube range is given. The basic range is defined as the top boundary at a vertical position 2% of the image height downwards from the top of the meter image area, and the bottom boundary at a vertical position 5% of the image height downwards from the bottom of the liquid column.

[0145] The top of the base range is set near the top of the image (leaving a 2% margin to accommodate the scale extension at the top of the liquid column), and the bottom is set at a certain distance below the bottom of the liquid column. The logic for setting the base range is: the liquid column must be inside the glass tube, so extending a certain margin outward from the position of the liquid column as an anchor point can ensure that the initial tube range at least intersects with the liquid column.

[0146] The base scope is expanded using two independent sources, both of which follow the principle of "expanding only, not shrinking".

[0147] The first extension method is the relaxed HSV thresholding approach: In the HSV space, a more relaxed red threshold condition is applied than that used for liquid column extraction. Starting from the defined liquid column row region, the extension proceeds row by row upwards and downwards. If an adjacent row contains red pixels that satisfy the relaxed threshold condition, that row is included in the tube's range. Since red scale lines outside the liquid column region are also captured by this relaxed threshold, the extension can expand the tube's range from the liquid column region to a region encompassing all red scale lines.

[0148] The second extension source is the Canny edge detection method: Canny edge detection is performed on the meter image area to extract the edge responses of the two walls of the glass tube. The glass tube walls are usually represented as two clear vertical edge lines in the image. From all detected edge segments, the longest continuous edge segment is selected, and the positions of the top and bottom of the tube are determined according to the vertical extension range of the edge segment. The tube range is then extended to include the complete edge segment.

[0149] These two expansion sources are independent of each other. The former relies on color information, while the latter relies on edge information. When the boundary of the pipe range provided by either source exceeds the corresponding boundary of the base range, the new boundary that exceeds the base range replaces the original boundary. If the boundary provided by a source does not exceed the base range, the original boundary remains unchanged. Both expansions can only expand the pipe range outwards and cannot shrink the pipe range inwards.

[0150] Constraints are applied: The expanded tube extent must encompass the entire vertical range of the liquid column; that is, the top boundary of the tube extent must not be lower than the top of the liquid column, and the bottom boundary of the tube extent must not be higher than the bottom of the liquid column. If the expanded tube extent fails to meet this hard constraint (e.g., Canny edge detection fails to cover the top region of the liquid column), the tube extent boundary is forcibly expanded to include the entire range of the liquid column, ensuring that the tube extent is always logically valid and avoiding the physical paradox of the tube extent being smaller than the liquid column extent.

[0151] After obtaining the vertical range of the tube, calculate the ratio of the liquid column's vertical range to the total vertical range of the tube, i.e., ratio = (bottom position of the tube - top position of the liquid column) / (bottom position of the tube - top position of the tube). Then, multiply this ratio by the physical reading range corresponding to the vertical range of the tube (e.g., -20°C to 50°C on the Celsius scale), and add the initial value of the range to obtain the current reading. It also supports dual-scale output on both Celsius and Fahrenheit scales, calculating and outputting the readings on both scales based on the user-configured range parameters.

[0152] Step S4: The pointer table recognition module uses a three-level strategy of Hough line segment detection, HSV red detection and contour fitting to recognize pointer readings.

[0153] The pointer meter recognition module performs reading recognition for pointer-type instruments (such as pressure gauges, thermometers, flow meters, etc.). Given the diverse pointer shapes (black pointers, red pointers, bi-color pointers), complex dial conditions (reflection, wear, low contrast), and varied scale distributions (uniform scales, non-linear scales, double-circle scales), recognition is completed through four stages: dial circular area detection, three-level strategy collaborative pointer recognition, scale information extraction, and angle reading mapping.

[0154] Please see the appendix Figure 6 Methods for pointer reading recognition include:

[0155] Step S41: Obtain the meter image area and detect the circular area of ​​the dial.

[0156] The input meter image is used to detect circular areas on the dial. Since pointer-type meters typically have circular dials, accurately locating the center and radius of the dial is a prerequisite for subsequent pointer detection and angle calculation. The detection of circular areas on the dial employs a method combining multi-parameter Hough circle detection and multi-dimensional comprehensive scoring and filtering.

[0157] The methods for detecting the circular area of ​​the dial include:

[0158] Multiple sets of Hough circle detection with different parameters were used, and the detection results were combined.

[0159] Repeated circles are deduplicated; circles with a distance of less than 0.15 times the radius are considered the same circle.

[0160] A comprehensive scoring function is constructed, which is a weighted sum of the circle inner edge density score, the distance from the circle center to the image center score, and the proportion of the diameter to the short side of the image score.

[0161] The circle with the highest score is selected as the circular area of ​​the dial.

[0162] Multiple sets of Hough circle detection algorithms with different parameters are used to detect dials in different sizes and contrast conditions. Specifically, four different parameter combinations are configured, each with differences in gradient accumulator threshold, minimum radius, and maximum radius.

[0163] For example, one group uses a higher accumulator threshold to detect large circles with sharp edges, while another group uses a lower accumulator threshold to detect small circles with blurred edges. Each group of parameters runs Hough circle detection independently, and the candidate circles detected by each group are merged into a single candidate circle set.

[0164] The merged candidate circle set contains a large number of duplicate detections. The same dial may be detected simultaneously by multiple sets of parameters, resulting in multiple candidate circles with similar center positions and radii. The candidate circle set is deduplicated: for any two candidate circles, if the distance between their centers is less than 0.15 times the average of their radii, they are considered the same circle, and the one with the higher score is retained, while the other is discarded. Through this deduplication operation, each candidate circle retained in the set represents an independent dial candidate region.

[0165] To select the best watch face from multiple candidate circles, a comprehensive scoring function is constructed to quantitatively evaluate each candidate circle. The scoring function is obtained by weighted summation of three components: the edge density score (fill) represents the density of image edge points contained within the candidate circle. Since the watch face contains scale lines and pointers, the edge density is higher, while the background area has lower edge density. Therefore, a higher edge density indicates that the circle is more likely to be the true watch face region. This component has a weight of 0.45. The center-to-image-center score (center_s) is the inversely normalized score of the distance from the center of the circle to the image center. During shooting, the watch face is usually located near the center of the image, so the closer the center is to the image center, the higher the score. This component has a weight of 0.35. The size score (size_s) is the score of the candidate circle's diameter relative to the shorter side of the image. The watch face usually occupies a major part of the image; a larger size ratio results in a higher score. This component has a weight of 0.20. After normalizing the three components, they are weighted and summed to obtain the comprehensive score for each candidate circle. The candidate circle with the highest score is selected as the final watch face circular region, and its center coordinates (cx, cy) and radius r are output.

[0166] Step S42: The pointer position is obtained by using a three-level strategy of Hough line segment detection, HSV red detection and contour fitting in sequence.

[0167] Based on the detected circular area of ​​the dial, a three-level pointer detection strategy is executed sequentially according to a preset priority. The execution of the subsequent strategy is terminated when the previous strategy successfully outputs a valid pointer position, and the execution of the next strategy is automatically rolled back to the next level when the previous strategy fails.

[0168] The first-level strategy is Hough segment detection, which is suitable for detecting most pointers (black pointer, metallic pointer, white pointer). Hough segment detection includes:

[0169] Extract the ROI region of the reading instrument, perform adaptive threshold binarization on the ROI region, and construct an annular mask to shield the central axis and outer ring;

[0170] The detected line segments are clustered by angle, with a clustering tolerance of 15°. The largest cluster is selected for weighted averaging, with the weight being the line segment length.

[0171] Using the two endpoints of the weighted line segment as candidate needle tips, the number of foreground pixels in the neighborhood is counted to obtain the needle tip;

[0172] If the distance from the needle tip to the center is less than 0.15 times the radius, it is considered invalid and triggers HSV red detection.

[0173] Regions of Interest (ROIs) are extracted based on the center and radius of the dial. Typically, a square ROI is extracted with the center as the center and a side length of 1.05 times the radius, ensuring that the dial area is completely included. Adaptive threshold binarization is then performed on the ROIs, converting the image into a binary image of the foreground (hands and tick marks) and background.

[0174] To reduce the interference of the central axis fixing bolt and the outer ring scale lines on pointer detection, an annular mask is constructed to act on the binary image: the mask retains an annular area with the dial center as the center, an inner diameter of 0.12 times the radius, and an outer diameter of 0.95 times the radius as the effective detection range, while the remaining areas (central axis and outer ring) are shielded.

[0175] Within the effective area defined by the annular mask, HoughLinesP line segment detection is performed to extract all line segments in the image. The detected line segments vary in length and direction, including segments belonging to the pointer, tick marks, and cluttered edges of the dial. All detected line segments are clustered by angle, with a clustering tolerance of 15°, meaning segments with an angle difference within 15° are grouped into the same cluster. After angle clustering, the cluster with the most line segments is most likely the cluster representing the pointer's direction (because the pointer is typically represented by long, concentrated line segments, while tick marks are scattered in various directions). The cluster with the most line segments is selected, and a weighted average is calculated for all line segments within that cluster, with the weight equal to the length of each line segment, resulting in a weighted average line segment representing the pointer's direction.

[0176] After obtaining the weighted line segment, the position of the pointer tip needs to be determined. The pointer's geometry is a line segment extending from near the center of the dial to near the edge, with one end close to the center (tail end) and the other end far from the center (tip). Using the two endpoints of the weighted line segment as candidate pointer tips, the number of foreground pixels in the neighborhood of each endpoint is counted. The true pointer tip falls at the tip of the pointer, and its neighborhood has a relatively sparse distribution of foreground pixels (only the pointer tip itself); while the tail end is close to the center, and its neighborhood contains structures such as the central axis or the root of the pointer, resulting in a relatively dense distribution of foreground pixels. By comparing the relative number of foreground pixels in the neighborhood of the two endpoints, the end with fewer foreground pixels is identified as the pointer tip. If the distance from the pointer tip to the center is less than 0.15 times the radius, it indicates that the detection result is not a true pointer (it may be a false detection of the central axis or a short scale line), the first-level strategy is deemed invalid, and the second-level strategy is triggered.

[0177] The second-level strategy is HSV red detection, suitable for red pointers or scenarios where the first-level strategy fails to detect red due to low contrast. Red pointers may exhibit different hues in RGB images due to changes in lighting, but red has a stable hue distribution in the HSV color space. The ROI region is converted to the HSV color space, and a dual-segment red threshold is used to extract the red region: hue H is located in either the 0-10 or 160-180 range, saturation S is no less than 50, and brightness V is no less than 50.

[0178] Pixels meeting the above conditions are marked as red foreground. This module performs morphological closing operations on the extracted red binary image to connect breaks in the red regions caused by reflections or wear. After the closing operation is complete, the bounding rectangles of all connected components are extracted (minAreaRect is used to obtain the minimum bounding rectangle), and the major axis direction of this rectangle is the direction of the red connected component.

[0179] Among multiple red connected regions, the region with the largest area and pointing towards the edge of the dial is selected as the pointer candidate, and the average of the rectangular corner points at the far end of this region (the side furthest from the center) is used as the pointer tip position. If the second-level strategy also fails to extract a valid pointer tip (e.g., there is no red area or all red areas are too small), the third-level strategy is triggered.

[0180] The third-level strategy is contour fitting, which serves as the final solution. Adaptive threshold binarization is performed on the ROI region, followed by morphological opening to remove minor noise. All outer contours are extracted from the binary image after opening, and the largest contour with an area greater than radius r multiplied by 3 is selected as the pointer candidate contour. A cv2.fitLine line fitting is performed on all points of this contour to obtain a fitted straight line passing through the contour. Along the direction of this fitted line, a line segment is extended outward from the center. During the extension, the foreground density in the neighborhood of each point is checked. When the neighborhood foreground density suddenly decreases, that point is the pointer tip. This tip is output as the pointer position.

[0181] By executing the above three-level strategy in sequence, the position of the pointer tip is finally obtained, and the angle value of the pointer relative to the center of the dial is calculated by combining the position of the center of the dial.

[0182] Step S43: Use OCR to extract the numbers and scale lines in the meter image area, and obtain the start and end positions of the scale and the start and end readings of the scale based on the positions of the numbers and scale lines.

[0183] After locating the pointer, the scale information on the dial needs to be extracted. An OCR engine is used to detect and recognize text within the ROI area of ​​the dial, extracting all numeric text blocks and their positional information. Taking a pressure gauge as an example, the dial may have scale numbers such as 0, 5, 10, 15, 20, 25, and 30. The OCR engine will detect these numeric text blocks and return their respective bounding box coordinates and recognized content. Based on the positional distribution of these numeric text blocks, the start and end positions of the scale are determined. Typically, the scale number in the direction of the smallest angle is taken as the start scale, and the scale number in the direction of the largest angle is taken as the end scale. Simultaneously, the reading values ​​corresponding to the start and end scales are obtained from the OCR results, forming a correspondence between "scale position readings." Furthermore, for subdivided scales without marked numbers, their readings are inferred through the equal division relationship between adjacent scale numbers.

[0184] Step S44: Obtain the pointer reading based on the pointer position, the start and end positions of the scale, and the start and end readings of the scale.

[0185] After obtaining the pointer angle, it needs to be mapped to the actual reading. The mapping process relies on user-configured range parameters, including the start angle, end angle, and range. The pointer angle is compared with the start and end positions of the scale: when the pointer angle is within a continuous interval between the start and end angles, the module calculates the percentage of the pointer angle's offset relative to the start angle relative to the entire range angle. This percentage is multiplied by the range, and then added to the range start value to obtain the pointer reading.

[0186] For cases where the starting angle is greater than the ending angle (e.g., the dial scale starts at 135° from the upper left corner and goes counterclockwise to 45° from the upper right corner, crossing the 0° / 360° dividing point), a back-wrap process is triggered: the pointer angle is specially compared with the starting and ending angles. When the pointer angle crosses the 0° dividing point, its angle value is offset to compensate, ensuring that the offset calculation is performed within the correct angle range.

[0187] Taking a pressure gauge with a range of 0 to 30 kg / cm², a starting angle of 135°, and an ending angle of 45° as an example, when the pointer points directly upward (90°), the clockwise offset of the pointer from the starting angle to the current position is calculated through the back-wrap process, and then mapped to the range of 0 to 30, finally outputting the correct reading result.

[0188] On the other hand, this embodiment provides a meter reading identification system based on multi-strategy fusion and feature filtering. Please refer to the appendix. Figure 7 ,include:

[0189] The module constructs a meter identification framework, which includes an input module 100, a reading meter identification module 200, a liquid column linear meter identification module 300, and a pointer meter identification module 400. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module.

[0190] The meter reading recognition module 200 uses an OCR engine to filter counting lines based on the geometric features of the text area for meter reading recognition, and then performs character recognition on the counting lines to obtain the reading.

[0191] The liquid column linear meter recognition module 300, for the recognition of liquid column linear meters, identifies the liquid column and scale lines based on the horizontal width of each row of color areas, extracts the liquid column range, and obtains the reading based on the aligned scale lines.

[0192] The pointer meter recognition module 400 employs a three-level strategy—Hough line segment detection, HSV red detection, and contour fitting—to collaboratively recognize pointer readings.

[0193] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A meter reading identification method based on multi-strategy fusion and feature filtering, characterized in that, Including the following steps: A meter identification framework is constructed, which includes an input module, a reading meter identification module, a liquid column linear meter identification module, and an pointer meter identification module. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module. The meter reading recognition module uses an OCR engine to filter count lines based on the geometric features of the text area, and then performs character recognition on the count lines to obtain the reading. The liquid column linear meter recognition module identifies the liquid column and scale lines based on the horizontal width of each row of color areas, extracts the liquid column range, and obtains the reading based on the aligned scale lines. The pointer meter recognition module employs a three-level strategy—Hough line segment detection, HSV red detection, and contour fitting—to collaboratively recognize pointer readings.

2. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 1, characterized in that, When the reading obtained by the reading meter recognition module through full-image text detection exceeds a preset number of digits, it is determined to be an abnormally long reading. Extract the grayscale ROI corresponding to the text block, calculate the adaptive threshold ladder based on the ROI brightness range, use a fixed threshold set when the brightness range is less than the preset brightness threshold, and generate multiple candidate thresholds proportionally and truncate them to the preset range when the brightness range is not less than the preset brightness threshold. Each candidate threshold is binarized, the bounding rectangle of the bright area is extracted, and then filled with pixels around it, cropped and enlarged. OCR recognition is called again, and only the results with numbers below a preset number of digits are accepted as candidate results. A majority vote is conducted on all candidate results, and the reading with the most votes and the highest confidence level is selected as the final counter reading.

3. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 1, characterized in that, Methods for filtering lines by counting based on the geometric features of text regions using an OCR engine include: The first pre-defined identification sub-strategy includes: Obtain the meter image area, and then obtain multiple text regions within the meter image area output by the OCR engine; Filter text areas where the number of digits is within a preset range [N1, N2], the height of the text block occupies the height of the meter image area in a ratio between 0.02 and 0.35, and the aspect ratio is not less than 0.5; Remove text regions with a digit count ≥ N3 and a height ratio > 0.12; The digit score is determined based on how close the number of digits is to the midpoint of the preset interval [N1, N2]. The size score is determined based on how close the text block height is to 0.

185. The aspect ratio score is determined based on how close the aspect ratio is to 0.

5. The confidence score is obtained based on the confidence level of the text area output by the OCR engine. The weighted sum of the score based on the number of digits, the size score, the aspect ratio score, and the confidence score is used as the overall score for the text region. The text region with the highest overall score is used as the counting line. The pre-defined second identification sub-strategy includes: All text blocks of type number are clustered by center position according to the y-axis position. The tolerance of the clustering is 0.8 times the median height and not less than the preset minimum pixel height. Within a cluster, groups are formed based on x-axis adjacency, with the number of digits within each group falling within a preset range [N1, N2] and the aspect ratio of each group between 1.2 and 15. The group digit score is determined based on the degree to which the number of digits in the group is close to the midpoint of the preset interval [N1, N2]. The height consistency score is determined based on the consistency of the height of the digital text blocks in the group. The group aspect ratio score is determined based on the degree to which the aspect ratio of the group is close to 8.

1. The group confidence score is obtained based on the confidence of the text area output by the OCR engine. The overall group score of a text region is calculated by weighting the group number score, high consistency score, group aspect ratio score, and group confidence score. The text region with the highest overall group score is used as the counting row. When the overall group score is not higher than a preset multiple of the overall score, the count row identified by the first identification sub-strategy is used; when the overall group score is higher than a preset multiple of the overall score, the count row identified by the second identification sub-strategy is used.

4. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 1, characterized in that, The reading meter identification also includes a mechanical counter identification method, which includes: Obtain the meter image region and perform binarization processing. Filter out digit-shaped contours and perform center clustering based on the y-direction position. Remove clusters with fewer than 3 digit-shaped contours after clustering or clusters whose aspect ratio is not within the interval [1.5, 15]. The binarized meter image region is connected by performing a wide-kernel horizontal closure operation in both inverted and normal phases to connect broken strokes, and connected components with aspect ratios in the range [1.5, 15] and rectangularity not less than 0.3 are selected. In the HSV space, high-brightness, low-saturation white regions are selected, and wide-kernel closed-loop operations are performed to connect them. Regions with aspect ratios within the range [2, 12], rectangularity greater than 0.5, and internal Canny edge density between [0.03, 0.5] are selected. For detected counter ROIs, OCR recognition is called first. If OCR fails, it falls back to the KNN classifier based on equal-width segmentation for mechanical counter recognition.

5. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 4, characterized in that, The methods for training the KNN classifier include: For each number, training images are generated using multiple fonts, weights, and sizes. These images are then uniformly scaled to a preset size and flattened into feature vectors. An initial classifier is created using a preset architecture and trained using the feature vectors; When configuring predictions, k-nearest neighbor voting is used, and distance is used as the confidence level.

6. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 1, characterized in that, The method for identifying liquid columns and scale lines based on the horizontal width of each row of color regions includes: Obtain the meter image area, extract the liquid column area in HSV space, scan the color area of ​​each row of the image, and count the maximum continuous color horizontal span of each row. Rows with a horizontal span greater than or equal to the preset width are designated as liquid column rows, and rows with a horizontal span less than the preset width are designated as scale line rows. The maximum continuous range of the liquid column is taken as the vertical range of the liquid column.

7. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 6, characterized in that, Methods for obtaining readings based on aligned scale lines include: Attempt to identify readings within a preset distance range of the aligned scale lines; if a reading is identified, use that reading as the reading of the aligned scale line. If no reading is identified, the vertical range of the tube is identified. Based on the proportion of the vertical range of the liquid column to the vertical range of the tube and the reading range corresponding to the vertical range of the tube, the reading of the aligned scale line is obtained. Methods for identifying the vertical range of a tube include: The base area is defined by the vertical range of the meter image area from 2% down from the top to 5% down from the bottom of the liquid column. A relaxed HSV threshold is used to expand from the liquid column region, and Canny edge detection is used to detect the glass tube wall and the top and bottom of the tube are determined based on the longest continuous edge segment. The vertical range of the pipe is obtained based on the expanded area.

8. The meter reading identification method based on multi-strategy fusion and feature filtering according to claim 1, characterized in that, Methods for pointer reading recognition include: Obtain the meter image area and detect the circular area of ​​the meter dial. A three-level strategy, consisting of Hough line segment detection, HSV red detection, and contour fitting, is used in sequence to identify the pointer and obtain its position. OCR is used to extract the numbers and scale lines in the meter image area, and the start and end positions of the scale and the start and end readings of the scale are obtained based on the positions of the numbers and scale lines. The pointer reading is obtained based on the pointer position, the start and end positions of the scale, and the start and end readings of the scale. Hough segment detection includes: Extract the ROI region of the reading instrument, perform adaptive threshold binarization on the ROI region, and construct an annular mask to shield the central axis and outer ring; The detected line segments are clustered by angle, with a clustering tolerance of 15°. The largest cluster is selected for weighted averaging, with the weight being the line segment length. Using the two endpoints of the weighted line segment as candidate needle tips, the number of foreground pixels in the neighborhood is counted to obtain the needle tip; If the distance from the needle tip to the center is less than 0.15 times the radius, it is considered invalid and triggers HSV red detection.

9. A meter reading identification method based on multi-strategy fusion and feature filtering according to claim 8, characterized in that, Methods for detecting the circular area of ​​the dial include: Multiple sets of Hough circle detection with different parameters were used, and the detection results were combined. Repeated circles are deduplicated; circles with a distance of less than 0.15 times the radius are considered the same circle. A comprehensive scoring function is constructed, which is a weighted sum of the circle inner edge density score, the distance from the circle center to the image center score, and the proportion of the diameter to the short side of the image score. The circle with the highest score is selected as the circular area of ​​the dial.

10. A meter reading identification system based on multi-strategy fusion and feature filtering, characterized in that, include: The module constructs a meter identification framework, which includes an input module, a reading meter identification module, a liquid column linear meter identification module, and a pointer meter identification module. The input module receives meter images and identifies the meter type, and then sends them to the pointer meter identification module, the liquid column linear meter identification module, or the reading meter identification module. The meter reading recognition module uses an OCR engine to filter count lines based on the geometric features of the text area, and then performs character recognition on the count lines to obtain the reading. The liquid column linear meter identification module identifies the liquid column and scale lines based on the horizontal width of each row of color areas, extracts the liquid column range, and obtains the reading based on the aligned scale lines. The pointer meter recognition module employs a three-level strategy—Hough line segment detection, HSV red detection, and contour fitting—to collaboratively recognize pointer readings.

Images