Knowledge-Guided Meter Reading Method and System Based on Visual Language Model

Abstract

This invention relates to the fields of computer vision and artificial intelligence, specifically to a knowledge-guided meter reading method and system based on a large visual language model. The method includes: a template acquisition step: acquiring a meter image and retrieving a structured reading prompt template containing metadata and inference logic instructions based on the type; a multimodal input step: inputting the image and template as multimodal data into a large visual language model; an inference generation step: extracting visual features and aligning them with the template's attention, executing thought chain inference in response to instructions, and generating a natural language description result containing intermediate inference evidence; and a parsing output step: performing structured parsing on the result and extracting meter reading values ​​that conform to format constraints. This invention achieves a deep integration of business logic and visual perception, establishes human-computer trust, and avoids logical fallacies inherent in purely visual models.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically to a knowledge-guided meter reading method and system based on a large visual language model. Background Technology

[0002] In industrial production and maintenance inspections, various analog meters generate a large amount of status data, and these meters are read and identified through automated means. The accuracy of meter readings largely depends on the precision of the image recognition system in resolving the mapping relationship between visual features and physical values. In existing technologies, reading systems mainly rely on deep neural networks to perform simple visual feature segmentation and key point localization on the dial image, and calculate values ​​based on geometric rules. When encountering situations such as dial reflection, blurriness, or dirt, the lack of contextual logic guidance prevents the model from effectively eliminating interference, resulting in relatively low reliability of meter readings. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a knowledge-guided meter reading method and system based on a large visual language model. Specifically, the technical solution of this invention is as follows:

[0004] Acquire the image of the meter to be identified, and retrieve the preset structured reading prompt template based on the current meter type characteristics;

[0005] The structured reading prompt template includes meter metadata definitions described in natural language and step-by-step reading reasoning logic instructions.

[0006] The meter image and the structured reading prompt template are used as multimodal data inputs to a preset visual language model;

[0007] The visual features of the meter image are extracted using the large visual language model, and the visual features are aligned with the structured reading prompt template to locate the visual coordinates of key parts of the meter. The reading reasoning logic instruction is combined with the preset geometric calculation rules to perform the thought chain reasoning to generate a natural language description result containing intermediate reasoning basis.

[0008] The natural language description results are parsed in a structured manner to extract the meter reading values ​​that conform to the numerical format constraints defined in the structured reading prompt template.

[0009] Preferably, the step of constructing the structured reading prompt template includes:

[0010] Define role setting information, which is used to configure the response identity parameters of the visual language big model;

[0011] Define the meter metadata information, which includes the measuring range, scale unit, multiplier relationship, reading carry rules, and numerical format constraints.

[0012] Define a thought chain guidance instruction, which is used to constrain the visual language large model to generate the natural language description result in the order of visual localization, feature recognition, rule matching and numerical calculation.

[0013] Preferably, the step of performing visual semantic alignment of the meter image using a large visual language model includes:

[0014] The meter image is converted into a visual feature vector using the encoding layer of the aforementioned large visual language model;

[0015] The textual concepts of the names of key parts of the meter contained in the structured reading prompt template are mapped to the pixel areas corresponding to the visual feature vectors to establish a semantic association between the text description and the image features.

[0016] Based on the semantic association, the dial area, the scale line position, and the pointer pointing area are located in the meter image.

[0017] Preferably, the step of performing thought chain reasoning in conjunction with the reading reasoning logic instruction includes:

[0018] Based on the reading reasoning logic instruction, identify two adjacent scale values ​​where the pointer is located in the meter image;

[0019] Determine the positional offset of the pointer relative to the two adjacent scale values, and determine the current indication value based on a preset distance determination logic or interpolation calculation rule;

[0020] Based on the multiplication ratio defined in the structured reading prompt template, the current indicated value is multiplied to generate the natural language description result.

[0021] Preferably, the method further includes a zero-sample adaptation step:

[0022] When the meter type corresponding to the meter image changes to a new type that has not been trained, the parameters of the visual language large model remain unchanged;

[0023] Update the meter metadata definitions and reading inference logic instructions in the structured reading prompt template to adapt to the characteristics of the new type of meter;

[0024] The updated structured reading prompt template guides the visual language big model to read new types of meter images.

[0025] Preferably, the step of performing structured parsing on the natural language description results to extract meter reading values ​​that conform to preset business logic constraints includes:

[0026] The numerical portion is extracted from the natural language description result using regular expressions or format extraction algorithms as candidate readings.

[0027] Verify whether the candidate readings and the intermediate derivation basis satisfy the logical verification rules defined in the structured reading prompt template;

[0028] If the verification result is satisfactory, the candidate reading is output as the meter reading value.

[0029] If the verification result is not met, an error message will be generated and a manual review process will be triggered.

[0030] Preferably, the logic verification rules include a multi-pointer reading consistency rule, a maximum range limit rule, and the reading carry rule.

[0031] A knowledge-guided meter reading system based on a large visual language model includes:

[0032] The data acquisition module is used to acquire the image of the meter to be identified and retrieve the preset structured reading prompt template based on the current meter type characteristics;

[0033] The reasoning interaction module is used to input the meter image and the structured reading prompt template as multimodal data into a preset visual language big model, and use the visual language big model to perform visual feature extraction, attention alignment and thought chain reasoning to generate a natural language description result containing intermediate reasoning basis.

[0034] The result parsing module is used to perform structured parsing on the natural language description results and extract the meter reading values ​​that conform to the numerical format constraints defined in the structured reading prompt template.

[0035] The system updates the structured reading prompt template to adapt to different types of meters without retraining the large visual language model.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. This invention achieves a deep integration of business logic and visual perception by introducing a structured reading prompt template and a thought chain reasoning mechanism. It transforms the reading experience of domain experts into contextual constraints that the visual language big model can understand. This solution can not only output reading results, but also provide a complete derivation process. When faced with interference such as dial reflection, stains, or blurred visual features, the visual language big model can use contextual logic to make robust inferences, effectively avoiding logical fallacies caused by pixel noise in traditional pure visual models, and establishing a traceable human-machine trust system.

[0038] 2. This invention establishes a semantic association between text description and image features by aligning visual features with a structured reading prompt template. Combined with the distance judgment logic or interpolation calculation rules defined in the structured reading prompt template, it achieves closed-loop verification from semantic understanding to mathematical calculation. At the same time, it introduces numerical format constraints that conform to the template definition to ensure that the final reading is an accurate value verified by reasoning logic instructions, rather than a simple probability generation, thereby effectively suppressing numerical illusion.

[0039] 3. This invention constructs logical verification rules and performs maximum range limits, multi-pointer reading consistency checks, and carry logic checks on candidate readings output by the visual language large model based on the logic defined in the structured reading prompt template. This mechanism can automatically intercept erroneous readings that do not conform to physical laws or business logic, and trigger manual intervention when an anomaly is detected, ensuring data reliability and security in industrial application scenarios.

[0040] 4. This invention, through zero-sample adaptation technology and a method for updating structured reading prompt templates, adapts to new types of meter features by only adjusting the metadata definition in the prompt template while keeping the parameters of the large visual language model unchanged. This design significantly reduces the marginal cost and maintenance complexity of new equipment access, solves the pain point of the complex and inexhaustible range of meter models in industrial fields, and achieves compatibility and coverage for various types of meters. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0042] Figure 1 This is a flowchart of the method of the present invention;

[0043] Figure 2 This is a structural diagram of the system of the present invention. Detailed Implementation

[0044] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0045] Example 1:

[0046] Please see Figure 1 Knowledge-guided meter reading methods based on a visual language big data model include:

[0047] Acquire the image of the meter to be identified, and retrieve the preset structured reading prompt template based on the current meter type characteristics;

[0048] The structured reading prompt template includes meter metadata definitions described in natural language and step-by-step reading reasoning logic instructions.

[0049] Meter images and structured reading prompt templates are used as a large visual language model for multimodal data input.

[0050] Visual features of meter images are extracted using a large visual language model. These visual features are then aligned with a structured reading prompt template to locate the visual coordinates of key parts of the meter. Finally, reading reasoning logic instructions are combined with preset geometric calculation rules to execute thought chain reasoning, thereby generating natural language description results containing intermediate derivation basis.

[0051] The natural language description results are parsed in a structured manner to extract the meter reading values ​​that conform to the numerical format constraints defined in the structured reading prompt template.

[0052] In the specific execution of this embodiment, the method constructs a software-defined instrument system defined by prompt words, aiming to inject the reading logic of domain experts into the visual language large model; the visual language large model of the present invention specifically refers to a generative model that adopts a transformer architecture and opens access permissions to attention layer weights, such as an open-source model combining LLaVA-1.5 (Large Language and Vision Assistant v1.5), MiniGPT-4 (Mini GPT-4), or CLIP (Contrastive Language-Image Pre-trained Model) with Vicuna (Alpaca Model) architecture, or an application programming interface service that provides an intermediate layer attention tensor output interface; the model needs to have the ability to output cross-attention weights during the inference stage so that subsequent steps can perform pixel-level visual localization.

[0053] About the structured readout prompt template It uses JavaScript Object Notation (JSON) for storage and includes fields such as meter_type (meter type), scale_range (scale range), and read_step (reading steps). This template refers to a predefined text or code structure used to transform unstructured human reading experience into contextual constraints that can be understood by large language models. As a carrier of the cognitive reasoning layer, it guides the model on how to observe and think. The content comes from a knowledge base written by domain experts for different meter types.

[0054] The system determines the type characteristics of the meter through an image classifier or manual selection, and retrieves the corresponding structured reading prompt template from a pre-set knowledge base. The visual language big data model refers to a deep neural network with image encoding and text generation capabilities. The system constructs a big data model containing images of meters to be identified. and retrieved structured reading prompt templates Multimodal input sequences This allows the model to possess prior knowledge of the specific meter before processing the image; the thought chain reasoning refers to the process by which the model explicitly generates a series of intermediate reasoning steps before generating the final answer. The model extracts images through a visual encoder. High-dimensional visual features :

[0055]

[0056] in, This represents the set of real numbers, meaning that every element in the feature tensor is a real number. The number of feature channels, Given the height and width of the feature map, a multi-head attention mechanism is used to define the template. Textual concepts and The spatial regions are semantically aligned, and the natural language description results are generated step by step by strictly following the instructions in the template. The result is a sequence of text strings containing numerical values ​​and reasoning processes; in structured parsing, the system does not directly use... Instead of using a final result, the parser extracts key information from the text and extracts the final meter reading from the natural language text according to the numerical format defined in the template. ;

[0057] This embodiment achieves a deep integration of business logic and visual perception by introducing a structured reading prompt template and thought chain reasoning. The solution not only outputs numbers but also provides a complete derivation process, enabling maintenance personnel to trace the cause of errors and establishing human-machine trust. At the same time, since the model is injected with reading rules as strong priors, when the image has reflections or stains that cause visual features to be blurred, the model can use contextual logic to make more robust inferences, thereby avoiding the logical fallacies caused by pixel noise in traditional pure vision models.

[0058] Example 2:

[0059] The steps to build a structured reading prompt template include:

[0060] Define role setting information, which is used to configure the response identity parameters of the visual language large model;

[0061] Define the meter metadata information, which includes the measuring range, scale unit, multiplier relationship, reading carry rules, and numerical format constraints;

[0062] Define a thought chain guidance instruction, which is used to constrain the visual language large model to generate natural language description results in the order of visual localization, feature recognition, rule matching and numerical calculation.

[0063] In this embodiment, to ensure that the visual language model strictly executes the preset logic, the system employs a slot-filling-based prompt word construction algorithm; role setting information Instantiated as a system-level prompt: You are an expert in reading industrial meters. Please make inferences strictly based on the given metadata and do not output any uncertain guesses. This setting alters the probability distribution during model decoding through an attention mechanism, suppressing the generation of illusions. Meter metadata information. It is serialized into a standard JSON string format and embedded into the context data area of ​​the prompt template;

[0064] This injection of structured data allows the model to directly reference specific numerical boundaries in subsequent inference, solving the problem of general models lacking domain common sense; (Thinking chain guidance instructions) The information is compiled into an ordered list of natural language steps; the system constructs the above three parts of information into a complete input sequence; and the operators involved in the above sequence construction are implemented, specifically, concatenation. The operation is not a simple string concatenation, but a tensor concatenation based on multimodal alignment: the system defines the input embedding sequence. as follows:

[0065]

[0066] in, The text embedding tensor representing the prefix cue words, For prefix length, For the embedded dimension; This represents the image feature embedding tensor after projection alignment. Indicates the length of the image feature sequence; This represents a suffix text embedding tensor containing thought chain instructions. The suffix length;

[0067] Construct the complete input sequence; system-defined image placeholders. The insertion position in the text prompt word will It is analyzed into three parts: prefix text Image placeholders, suffix text ; Use a word segmenter to and They are respectively converted into a sequence of token identifiers (TokenID) and mapped to a text embedding tensor. and Using a visual encoder to process images The vector is converted into a visual feature vector, and its dimensions are aligned to the text embedding dimensions through a projection layer. ; Perform on the sequence length dimension This operation ensures that image features are logically and precisely embedded into the contextual structure defined by the prompt words.

[0068] Example 3:

[0069] The steps for visual semantic alignment of meter images using a large visual language model include:

[0070] The meter image is converted into a visual feature vector by using the encoding layer of the key part names of the meter contained in the visual language big model.

[0071] The textual concepts in the structured reading prompt template are mapped to the pixel regions corresponding to the visual feature vectors to establish a semantic association between the textual description and the image features.

[0072] Based on semantic association, the dial area, scale line position, and pointer pointing area are located in the meter image.

[0073] In this embodiment, the visual feature vector refers to the high-dimensional matrix obtained after the image is encoded by a deep convolutional network or Transformer:

[0074]

[0075] in, and These represent the spatial height and width of the feature map, respectively. This represents the number of channel dimensions of the feature vector. To achieve pixel-level precise localization, this embodiment employs an attention backtracking algorithm based on token index mapping. This step is executed using a large visual language model with access permissions to model weights and parameters. The word segmenter of the large visual language model converts the text concept to be localized into a set of TokenIDs. The text concept to be located refers to the aforementioned key parts of the meter. Depending on the type of meter, these include the dial used to define the reading area, the scale lines used to provide numerical reference, and the pointer used to indicate the current value. The system iterates through the token list of the input sequence. Construct a set of sequence indexes that match this concept:

[0076]

[0077] in, This represents the position index of the token in the sequence, thus solving the problem of mapping vocabulary IDs to sequence positions; the system extracts the cross-attention tensor of the last layer of the model. :

[0078]

[0079] in, This indicates the number of heads in the multi-head attention mechanism. This indicates the number of tokens in the input text sequence. and These represent the spatial height and width of the image feature map, respectively. Corresponding to the length of the image feature sequence after flattening;

[0080] Calculate the convergent attention heatmap This heatmap semantically represents the response intensity of the pointer concept at various locations in the image; to obtain precise coordinates, the system will flatten the image... The vector is reshaped into a two-dimensional matrix corresponding to the size of the feature map space. Bilinear interpolation is performed to upsample to the original image size, and the dynamic threshold is calculated using the Otsu's method. ;

[0081] The system first bases its index set From tensor Extract slices of the corresponding text concepts and average the multi-head attention and multi-token attention. The calculation formula is as follows:

[0082]

[0083] in, Representing the heatmap in spatial coordinates The response value at that location, This indicates the number of heads in the multi-head attention mechanism. Represents a set The number of elements in Indicates the first The first attention point in the Each token position corresponds to image coordinates Attention weights;

[0084] The specific calculation process is as follows: Statistically reshaped The grayscale histogram for each possible threshold Calculate the pixel ratio of the background and foreground classes respectively. and average gray level Calculate the between-class variance:

[0085]

[0086] in, Thresholds The percentage of background and foreground pixels in the total pixels. The average gray values ​​of the background and foreground classes are selected respectively, so that... The largest The value is used as the final binarization threshold. This adaptively separates high-response regions from background noise, generating a binary mask.

[0087]

[0088] in, and This constitutes the conditional operation logic, indicating if the heatmap response value of a pixel... Greater than the threshold Then the pixel in the mask The value in is assigned to 1; otherwise, it is assigned to 0.

[0089] To eliminate the interference of discrete noise points or multi-peak attention distributions on centroid calculation, the system uses a binary mask. Perform connected component analysis and retain the connected regions with the largest area. As an effective positioning area, among which For a binary mask, when pixel The value is 1 if it belongs to the target region, and 0 otherwise. To ensure the robustness of the algorithm in extreme cases, the system verifies the mask region before calculating the centroid. Total number of pixels:

[0090]

[0091] That is, the total number of pixels in the target area; if Then calculate the geometric centroid of the mask region:

[0092]

[0093] By collapsing the fuzzy attention distribution into definite physical coordinates, a mathematical transformation from semantic understanding to geometric localization is achieved; if If so, an error flag will be output and manual intervention will be requested;

[0094] To establish a robust geometric calculation baseline, the system is based on the dial area mask from the aforementioned positioning steps. Calculate its geometric centroid as the global reference origin. If the template does not explicitly define the dial area, a linear least squares circle fitting algorithm will be executed to determine it. Specifically, based on the expansion algebraic equation of a circle:

[0095]

[0096] in, , Let the coordinates be the points on the circumference. , The x and y coordinates are the center of the circle. Let be the radius of the circle; construct a system of linear equations that pass through a fixed point:

[0097]

[0098] Among them, construct the design matrix , The total number of coordinate points of the identified tick marks, its th Row vector is defined as This corresponds to the set of coordinates of all identified tick marks. The first in One point;

[0099] Construct the target vector , its first Each element is defined as Solving for the parameter vector using a pseudo-inverse matrix Where the parameter vector is:

[0100]

[0101] Right now The first two components are the coordinates of the center of the circle. superscript This represents the matrix transpose, where the third component contains the radius. The information, therefore extract The first two components are the center of the fitted circle. This algorithm avoids the convergence uncertainty of iterative search and provides a precise geometric origin with analytical solutions for subsequent polar coordinate calculations of circular meters.

[0102] Example 4:

[0103] The steps for performing thought chain reasoning by combining reading reasoning logic instructions include:

[0104] Based on the reading reasoning logic instructions, identify the two adjacent scale values ​​where the pointer is located in the meter image;

[0105] Determine the position offset of the pointer relative to two adjacent scale values, and determine the current indication value based on preset distance determination logic or interpolation calculation rules;

[0106] Based on the multiplication relationships defined in the structured reading prompt template, the current indicated value is multiplied to generate a natural language description result.

[0107] In generation The reasoning process within the thought chain is actually an interaction between the large visual language model and the external geometric computation module; the system in Reverse lookup to generate a string in the token sequence and The corresponding Token index range is used, and the centroid coordinates of these two values ​​on the image are extracted using the attention aggregation method described in the previous embodiment. and Simultaneously, the attention mechanism of Example 3 is used to locate the pixel region of the pointer tip concept and calculate the centroid to obtain the coordinates of the pointer tip. ;

[0108] The system performs differentiated interpolation logic based on the geometry type field in the meter metadata:

[0109] If this field is identified as linear, the system constructs a scale direction vector. and pointer projection vector ; In calculating relative offset To prevent division by zero due to coincident or misidentified scale points causing the vector magnitude to be zero, a minimum regularization term is introduced into the formula. The value is :

[0110]

[0111] Among them, symbols Represents the vector dot product. The L2 norm (Euclidean norm) of a vector is used to represent the vector.

[0112] If the field is identified as circular, the system retrieves the dial center obtained in Example 3. Construct reference vector , and pointer vector ; Calculate the argument of a vector using the four-quadrant arctangent function:

[0113]

[0114] in, This is the arctangent function in the four quadrants, and its return value range is... The subscripts y and x represent vectors respectively. The vertical and horizontal components in a rectangular coordinate system.

[0115] System calculation vector and To correctly distinguish between clockwise and counterclockwise directions, the smallest signed angle between two vectors needs to be determined using the algebraic form of the cross product of two-dimensional vectors instead of the norm form:

[0116]

[0117] and vectors with pointer vector The smallest signed angle between them:

[0118]

[0119] Among them, symbols Represents the cross product of vectors, symbol Represents the dot product of vectors; This represents the L2 norm of the vector; it's used to eliminate jitter noise near the zero mark, i.e., noise caused by tiny pointer deviations. Become close For large values, the system performs the following logical judgment:

[0120] like and ,in, The preset jitter tolerance angle, typically 0.05 radians, is then forced to... ;

[0121] Otherwise if If so, it is judged as abnormal or reversed;

[0122] Under normal circumstances, the proportion is calculated as follows:

[0123]

[0124] Finally, the boundary clamping logic is introduced:

[0125]

[0126] Current indicator value The calculation formula is revised to be based on interpolation of physical values:

[0127]

[0128] The calculation result After multiplier After correction This information is then backfilled into the natural language description.

[0129] This embodiment, through this closed-loop design, ensures that the final reading is not merely a probability generation of the language model, but has undergone rigorous geometric projection calculation verification, effectively avoiding numerical illusion.

[0130] Example 5:

[0131] The method also includes a zero-shot adaptation step:

[0132] When the meter type corresponding to the meter image changes and becomes a new type that has not been trained, keep the parameters of the large visual language model unchanged;

[0133] Update the meter metadata definitions and reading inference logic instructions in the structured reading prompt template to adapt to the characteristics of new types of meters;

[0134] The updated structured reading prompt template guides the visual language big model to read new types of meter images.

[0135] In this embodiment, zero-sample adaptation refers to fitting the model weights without updating them. In certain situations, the system can complete new tasks simply by changing the input context; when a previously unseen meter type appears on-site, the system does not require data re-labeling or model fine-tuning; maintenance personnel only need to modify the prompt template. Adjust the dial shape description and range parameters; updated template. It was immediately fed into the large visual language model; since the model already possessed general object and number cognition concepts, it was able to combine... Guided by the instructions, the new meter is immediately and correctly identified;

[0136] This embodiment enables the access of new meters with marginal costs approaching zero. Compared with traditional solutions that require massive samples and long training periods, this solution only requires writing prompt words to go online, giving the system a strong ability to adapt to heterogeneous scenarios quickly and effectively solving the pain point of complex and inexhaustible meter models in industrial fields.

[0137] Example 6:

[0138] The steps for performing structured parsing on the natural language description results and extracting meter readings that conform to preset business logic constraints include:

[0139] Using regular expressions or format extraction algorithms, the numerical part is separated from the natural language description results as candidate readings;

[0140] Verify whether the candidate readings and intermediate derivations meet the logical verification rules defined in the structured reading prompt template;

[0141] If the verification result is satisfactory, the candidate reading is output as the meter reading value.

[0142] If the verification result is not met, an error message will be generated and a manual review process will be triggered.

[0143] The logic verification rules include multi-pointer reading consistency rules, maximum range limit rules, and reading carry rules.

[0144] In this embodiment, the system outputs the model's... Perform analysis and extract candidate readings. To prevent the model from generating illusions, this embodiment introduces a logical consistency check function. The verification logic is as follows:

[0145]

[0146] in, The source is the extraction result, and the physical meaning is the candidate reading value; The source is meter metadata, and the physical meaning is the maximum range limit defined in the template; symbol This represents the logical AND operation; The source is single-pointer logic judgment, and the physical meaning is the result of the logic consistency judgment of the thought chain, which is used to check whether the intermediate steps contradict the final conclusion.

[0147] In practice, the system uses pre-compiled regular expressions from... Extracting range assertions describing pointer positions And extract the final numerical conclusion. ;like The value falls into If the defined interval is within the specified range, the function outputs True; otherwise, it outputs False. The source is multi-pointer linkage verification. Regarding the multi-pointer reading consistency rules in this embodiment, a specific calculation and verification method is defined: the system uses a multi-key-value matching regular expression to extract a structured set of multi-dial readings, and forcibly converts the extracted numerical strings into double-precision floating-point numbers, denoted as: low-order dial reading. and high dial reading Among them, the preset high-position dial and low-position dial satisfy the following: The number system has a base-based transmission relationship, and the minimum scale value of the higher dial corresponds to the full-scale rotation of the lower dial.

[0148] It should be noted that, to ensure the validity of the consistent formula, the input variables... and It must be a continuous indicator value containing decimal place information, and this value comes from the interpolation algorithm in Example 4 during generation. The system calculates the precise time result, rather than simply rounding up the reading; considering the design characteristic of mechanical meters where the dial rotates counterclockwise, the system needs to read the rotation direction parameter from the metadata in advance;

[0149] If defined as counterclockwise, let... To get the full-scale value of the lower dial, let Updated to Perform numerical reversal processing to unify the rotation direction logic; consistency check requires that the decimal part of the high-order dial must match the progress of the low-order dial.

[0150] Considering the circular characteristics of a mechanical watch dial, where the values ​​0.9 and 0.0 are geometrically continuous, direct subtraction could lead to boundary errors. Therefore, this embodiment uses the minimum circular distance formula for discrimination: Let the normalized linear difference value be... for:

[0151]

[0152] in, Indicates to The modulo operation is performed to obtain the decimal part, which represents the progress ratio of the high-order dial; mod1 refers to the remainder operation with 1 as the divisor. For consecutive double-precision floating-point numbers, the integer part representing the complete scale or number of revolutions is filtered out, and only the decimal part in the range [0,1) is retained. The ratio of the low-level dial reading to its full-scale range represents the progress ratio of the low-level dial; the absolute value of the difference between the two is the consistency deviation of the carry logic.

[0153] The verification criteria are as follows:

[0154]

[0155] in, This is the preset tolerance threshold; The determination method is based on the minimum scale division of the meter and the mechanical transmission error. The specific calculation formula is as follows:

[0156]

[0157] in, This is the static relaxation coefficient, and its value range is typically [value range missing]. This is used to adjust the tolerance for reading deviations; An integer representing the total number of graduations on the lower dial; this parameter is derived from the structured readout prompt template. The metadata field, the total number of dial ticks, is parsed and obtained. The relaxation coefficient, which is associated with the meter's accuracy class, can be read from a system-preset static configuration file containing a mapping table between accuracy classes and relaxation coefficients. However, in the preferred embodiment, the system no longer relies on the fixed coefficients in the static configuration file, but instead performs dynamic error baseline calculations during runtime to adapt to complex field environments. The specific steps are as follows:

[0158] The first step is to obtain the metrological tolerance standard; based on the metrological verification procedure corresponding to the meter type, find the maximum permissible error value for a specific accuracy class across the entire measurement range. ;

[0159] The second step is to use the coordinates of adjacent scale points identified in Example 4. and the corresponding physical scale values Calculate the dynamic resolution of the current image. :

[0160]

[0161] in, Represents the absolute value of a number. Represents the L2 norm of a vector; its physical meaning is the physical scale value represented by each pixel, with the unit being: scale value / pixel;

[0162] The third step is to obtain the inherent localization error constant of the visual model. and the maximum permissible error specified in the metrological verification procedure. ; Calculate the overall physical tolerance threshold:

[0163]

[0164] in, The localization error constant of the visual model, in pixels, typically 2-5 pixels; The dimensionless safety factor is preferably within the range of [value range missing]. This is used to cover additional positioning errors caused by perspective distortion. The maximum permissible error is expressed in physical scale values, determined according to metrological verification procedures. Dynamic resolution, unit: physical scale value / pixel;

[0165] The fourth step is to convert this physical tolerance relative to the minimum scale value. Dimensionless proportionality constant:

[0166]

[0167] in, The minimum physical value of the current meter division, i.e., the actual physical quantity represented by one scale, is obtained from the structured reading prompt template metadata. This calculation process is performed in real time with each reading to adapt to changes in image resolution at different shooting distances and zoom levels, aiming to tolerate minor asynchrony caused by mechanical gear backlash, while intercepting logical errors across scale levels. The system will use the calculated dynamic threshold. As a real-time error benchmark, it replaces the aforementioned preset static coefficients. In other words:

[0168]

[0169] And verify the formula again:

[0170]

[0171] in, The function here represents a minimum value operation, which compares the linear deviation values. Its supplementary angle deviation value The smaller of the two values ​​is automatically selected as the basis for determining the actual deviation.

[0172] This computational mechanism aims to address the periodic boundary jump problem of circumferential readings on mechanical dials, ensuring that when crossing the zero mark, the algorithm can correctly identify the minimum neighborhood deviation in geometric position, rather than the maximum linear difference in numerical value.

[0173] If the inequality is true, it is determined that the multi-pointer reading logic is consistent, and True is returned; otherwise, False is returned.

[0174] when When the value is False, the system does not output a reading, but instead issues an alarm to prompt manual intervention. This embodiment constructs a deterministic logic verification mechanism, which effectively intercepts erroneous readings that do not conform to physical laws or business logic.

[0175] Example 7:

[0176] Please see Figure 2 A knowledge-guided meter reading system based on a visual language big data model includes:

[0177] The data acquisition module is used to acquire the image of the meter to be identified and retrieve the preset structured reading prompt template based on the current meter type characteristics;

[0178] The reasoning interaction module is used to input meter images and structured reading prompt templates as multimodal data into a preset visual language big model. The visual language big model is used to perform visual feature extraction, attention alignment and thought chain reasoning to generate natural language description results containing intermediate reasoning basis.

[0179] The results parsing module is used to perform structured parsing of natural language description results and extract meter readings that conform to the numerical format constraints defined in the structured reading prompt template.

[0180] The system adapts to different types of meters by updating structured reading prompt templates without retraining the large visual language model.

[0181] In the hardware architecture and functional implementation of this embodiment, the data acquisition module is used to connect to a field camera or image database to obtain meter images. It also includes a metadata management unit that retrieves the corresponding structured reading prompt template based on the meter ID. The inference interaction module is deployed on a high-performance computing server or cloud inference cluster, loading a pre-trained large visual language model, and is responsible for... and Assemble into multimodal cue words and perform inference generation This module supports streaming output to monitor the thought process in real time; the result parsing module is a deterministic software program that performs regular expression extraction and logical validation rules. And is responsible for writing the final cleaned data into the industrial IoT platform;

[0182] This embodiment achieves decoupling and collaboration between probability-based cognitive reasoning and rule-based business logic through modular design. Its greatest system-level advantage lies in the versatility of the architecture. With the same set of code and model weights, it can cover the reading needs of water, electricity, gas, heat and industrial instruments in all scenarios by simply updating the external prompt word library. There is no need to develop independent algorithm models for each type of meter, which greatly reduces the maintenance cost and complexity of the system.

[0183] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A knowledge-guided meter reading method based on a large visual language model, characterized in that: include: Acquire the image of the meter to be identified, and retrieve the preset structured reading prompt template based on the current meter type characteristics; The structured reading prompt template includes meter metadata definitions described in natural language and step-by-step reading reasoning logic instructions. The meter image and the structured reading prompt template are used as multimodal data inputs to a preset visual language model; The visual features of the meter image are extracted using the large visual language model, and the visual features are aligned with the structured reading prompt template to locate the visual coordinates of key parts of the meter. The reading reasoning logic instruction is combined with the preset geometric calculation rules to perform the thought chain reasoning to generate a natural language description result containing intermediate reasoning basis. The natural language description results are parsed in a structured manner to extract the meter reading values ​​that conform to the numerical format constraints defined in the structured reading prompt template; The step of performing structured parsing on the natural language description results to extract meter readings that conform to preset business logic constraints includes: The numerical portion is extracted from the natural language description result using regular expressions or format extraction algorithms as candidate readings. Verify whether the candidate readings and the intermediate derivation basis satisfy the logical verification rules defined in the structured reading prompt template; If the verification result is satisfactory, the candidate reading is output as the meter reading value. If the verification result is not met, an error message will be generated and a manual review process will be triggered. The logical verification rules include multi-pointer reading consistency rules, maximum range limit rules, and reading carry rules; The calculation and verification method for the multi-pointer reading consistency rule is as follows: The extracted numerical string is forcibly converted to a double-precision floating-point number, which is recorded as the low-order dial reading. and high dial reading Let the normalized linear difference value be... for: Indicates to Perform a modulo operation to obtain the decimal part; mod1 refers to taking the remainder of the value with 1 as the divisor. This represents the ratio of the lower dial reading to its full-scale range. The verification criteria are as follows: ;in, This is the preset tolerance threshold; Tolerance The calculation formula is as follows: Based on the identified coordinates of adjacent tick points and the corresponding physical scale values Calculate the dynamic resolution of the current image. : in, Represents the absolute value of a number. The L2 norm of a vector; Obtain the inherent localization error constant of the visual model and the maximum permissible error specified in the metrological verification procedure. ; Calculate the comprehensive physical tolerance threshold : Convert the physical tolerance to the minimum scale value dimensionless proportionality coefficient : The calculated dynamic dimensionless proportionality coefficient As a real-time error benchmark, let the tolerance for: 。 2. The knowledge-guided meter reading method based on a large visual language model according to claim 1, characterized in that, The steps for constructing the structured reading prompt template include: Define role setting information, which is used to configure the response identity parameters of the visual language big model; Define the meter metadata information, which includes the measuring range, scale unit, multiplier relationship, reading carry rules, and numerical format constraints. Define a thought chain guidance instruction, which is used to constrain the visual language large model to generate the natural language description result in the order of visual localization, feature recognition, rule matching and numerical calculation.

3. The knowledge-guided meter reading method based on a large visual language model according to claim 1, characterized in that, The steps of extracting visual features from the meter image using the large visual language model and aligning the visual features with the structured reading prompt template for attention include: The meter image is converted into a visual feature vector using the encoding layer of the aforementioned large visual language model; The textual concepts of the names of key parts of the meter contained in the structured reading prompt template are mapped to the pixel areas corresponding to the visual feature vectors to establish a semantic association between the text description and the image features. Based on the semantic association, the dial area, the scale line position, and the pointer pointing area are located in the meter image.

4. The knowledge-guided meter reading method based on a large visual language model according to claim 2, characterized in that, The steps of performing thought chain reasoning in conjunction with the reading reasoning logic instructions include: Based on the reading reasoning logic instruction, identify two adjacent scale values ​​where the pointer is located in the meter image; Determine the positional offset of the pointer relative to the two adjacent scale values, and determine the current indication value based on a preset distance determination logic or interpolation calculation rule; Based on the multiplication ratio defined in the structured reading prompt template, the current indicated value is multiplied to generate the natural language description result.

5. The knowledge-guided meter reading method based on a large visual language model according to claim 1, characterized in that, The method also includes a zero-sample adaptation step: When the meter type corresponding to the meter image changes to a new type that has not been trained, the parameters of the visual language large model remain unchanged; Update the meter metadata definitions and reading inference logic instructions in the structured reading prompt template to adapt to the characteristics of the new type of meter; The updated structured reading prompt template guides the visual language big model to read new types of meter images.

6. A knowledge-guided meter reading system based on a large visual language model, characterized in that: The knowledge-guided meter reading method based on a large visual language model, as described in any one of claims 1 to 5, further includes: The data acquisition module is used to acquire the image of the meter to be identified and retrieve the preset structured reading prompt template based on the current meter type characteristics; The reasoning interaction module is used to input the meter image and the structured reading prompt template as multimodal data into a preset visual language big model, and use the visual language big model to perform visual feature extraction, attention alignment and thought chain reasoning to generate a natural language description result containing intermediate reasoning basis. The result parsing module is used to perform structured parsing on the natural language description results and extract the meter reading values ​​that conform to the numerical format constraints defined in the structured reading prompt template. The system updates the structured reading prompt template to adapt to different types of meters without retraining the large visual language model.

Images