Delay electric energy meter intelligent identification method of multi-mode large model fusion special model

Abstract

The invention belongs to the technical field of electric energy meter automatic verification, and particularly relates to a delay electric energy meter intelligent identification method of a multi-mode large model fusion special model. Comprising the following steps: S1, acquiring an electric energy meter image data set, synchronously extracting or labeling corresponding text semantic information, and constructing a multi-modal large-model general identification framework oriented to an electric energy meter key information identification task; s2, aiming at the problem of insufficient precision of a multi-modal large model in some fine-grained tasks, designing and training a corresponding lightweight special identification model; s3, constructing a perception, analysis and decision closed-loop workflow; s4, end-to-end processing from image input to structured report output is carried out, a lightweight feedback closed loop mechanism is built in, and continuous optimization of models and rules is carried out; according to the method, the multi-mode large model and the special identification model are combined, so that high-precision identification, structured extraction, consistency verification and abnormality diagnosis of key elements in the electric energy meter image are realized.

Description

Technical Field

[0001] This invention belongs to the field of automated verification technology for electricity meters, specifically involving a method for intelligent identification of delayed electricity meters based on a multimodal large-scale model fusion dedicated model. Background Technology

[0002] Currently, the verification of electricity meters largely relies on manual operation, which is not only inefficient but also prone to errors. The development of artificial intelligence technology, especially the application of deep learning in image recognition, offers new solutions to this problem. However, existing technologies often focus only on single-mode data processing, such as using only image data, which struggles to handle complex real-world scenarios, such as challenges arising from different lighting conditions, shooting angles, and differences in electricity meter models. Therefore, effectively combining multiple information sources to improve the accuracy and robustness of identifying key information from electricity meters has become an urgent problem to be solved. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method for intelligent identification of delayed energy meters by fusing a multimodal large model with a dedicated model. By combining a multimodal large model and a dedicated identification model, this method can achieve high-precision identification, structured extraction, consistency verification, and anomaly diagnosis of key elements in energy meter images.

[0004] The objective of this invention is achieved as follows: a method for intelligent identification of delayed energy meters based on multimodal large-scale model fusion and dedicated model, comprising the following steps:

[0005] S1. Collect the image dataset of electricity meters and simultaneously extract or annotate the corresponding text semantic information to construct a multimodal large model general recognition framework for the task of recognizing key information of electricity meters.

[0006] S2. To address the insufficient accuracy of multimodal large models in certain fine-grained tasks, design and train a corresponding lightweight dedicated recognition model.

[0007] S3. Construct a closed-loop workflow of perception, analysis, and decision-making; the perception layer calls multi-modal large model and special model in parallel to obtain multi-dimensional identification results such as asset number, reading, shooting time, and sealing status; the analysis layer performs logical verification on the identification results based on the preset business rule base (formulated with the participation of power operation and maintenance experts); the decision-making layer integrates multi-source information to generate a structured verification report, marks abnormal items, and supports manual review interface.

[0008] S4 encapsulates the multimodal large model, dedicated recognition model, and business rule base module into an integrated, independently running intelligent recognition program, performing end-to-end processing from image input to structured report output, and incorporates a lightweight feedback closed-loop mechanism for continuous optimization of the model and rules.

[0009] Furthermore, the electricity meter image dataset in step S1 covers electricity meter image data of different brands, models, installation environments, lighting conditions, and shooting angles; the text semantic information not only comes from the image itself, but also includes task instruction templates, which together constitute the text modality in the multimodal input.

[0010] Furthermore, in step S1, before annotating the semantic information of the text, it is necessary to determine the annotation standards and structured field definitions. Each image is annotated in a refined multi-level manner. The field names include asset number, current description, current reading, shooting time, nameplate area, upper left seal, upper right seal, lower right seal, and bottom seal.

[0011] Furthermore, step S2 addresses the insufficient accuracy of large multimodal models in fine-grained tasks by designing and training a lightweight dedicated recognition model, including:

[0012] Fine-tune the sealed areas in the electricity meter image;

[0013] End-to-end training is performed using object detection (such as YOLOv13);

[0014] An attention mechanism and sample augmentation training strategy are introduced to improve the robustness of the model in complex scenarios such as low light, occlusion, and reflection.

[0015] Furthermore, the detailed annotation of the sealing area in the energy meter image includes the definition and description of the sealing area boundary box annotation and the sealing status label; the annotation objects of the sealing area boundary box include the seals on the upper left, upper right, lower right and bottom of the energy meter casing, and the sealing status label includes intact, damaged and missing;

[0016] The introduced attention mechanism is the CBAM attention module. The sample augmentation training strategy addresses the typical challenges of field images of electrical energy performance. During training, each image is dynamically combined with 2-3 augmentation operations, and the augmentation intensity and frequency are increased for damaged and missing samples to alleviate class imbalance.

[0017] Furthermore, in step S3, the perception layer concurrently calls the multimodal large model and the dedicated model to obtain multidimensional recognition results, including:

[0018] The same energy meter image is simultaneously sent to two independent inference channels A and B;

[0019] After inference, the structured JSON output information of channels A and B is obtained, and the output of channel B is mapped to status.

[0020] The results of multimodal and specialized model recognition are fused.

[0021] Furthermore, the business rule base in step S3 is used to automatically verify the consistency and diagnose anomalies of the multidimensional recognition results output by the perception layer. The rule base is encoded in JSON Schema or rule engine (such as Drools) form and supports dynamic loading and version management. The core rules include basic numerical rationality rules, time consistency verification rules, and rules linking the sealing status and business status.

[0022] Furthermore, the feedback closed-loop mechanism in step S4 includes:

[0023] Perform manual review and then return the manually reviewed and corrected samples to the training set;

[0024] Regularly fine-tune the parameters of the multimodal model and the dedicated model;

[0025] The business rule base is dynamically updated to adapt to changes in policies or equipment standards.

[0026] Furthermore, the step of performing manual review and returning the manually reviewed and corrected samples to the training set includes:

[0027] The maintenance personnel can view the JSON / HTML reports output by the program and correct any errors, such as changing the seal status or correcting the readings.

[0028] Submit a patch sample via command-line tool; the patch sample is stored in a local cache directory and aggregated monthly.

[0029] A script is automatically triggered monthly to merge and download feedback samples and update them to the main training set.

[0030] Furthermore, the dynamic updating of the business rule base includes updating the rule base after expert review if the data submitted by manual feedback reflects a misjudgment of a certain rule.

[0031] The beneficial effects of this invention are as follows: The intelligent identification method for delayed energy meters based on the fusion of multimodal large-scale model and dedicated model of this invention includes the following steps: S1, collecting energy meter image datasets and simultaneously extracting or labeling corresponding text semantic information to construct a general recognition framework for multimodal large-scale model for key information identification tasks of energy meters; S2, addressing the insufficient accuracy of multimodal large-scale model in certain fine-grained tasks, such as seal integrity judgment, designing and training a corresponding lightweight dedicated recognition model; S3, constructing a closed-loop workflow of perception, analysis, and decision-making; wherein the perception layer concurrently calls the multimodal large-scale model and dedicated model to obtain multi-dimensional identification results such as asset number, reading, shooting time, and seal status; the analysis layer performs logical verification of the identification results based on a preset business rule base (formulated with the participation of power operation and maintenance experts); and the decision-making layer integrates multi-source information to generate a structured verification report and marks anomalies. The system integrates a multimodal large model, a dedicated recognition model, and a business rule base module into a single, independently running intelligent recognition program. This program performs end-to-end processing from image input to structured report output and incorporates a lightweight feedback loop mechanism for continuous model and rule optimization. Through these steps, the present invention's intelligent recognition method for delayed energy meters, which integrates a multimodal large model with a dedicated model, constructs an intelligent energy meter recognition system that combines automatic sensing, intelligent analysis, and autonomous decision-making. The core architecture uses an intelligent agent that collaborates with both the multimodal large model and the dedicated recognition model. A four-step progressive technical approach enables high-precision recognition, structured extraction, consistency verification, and anomaly diagnosis of key energy meter elements. This effectively improves the accuracy and robustness of key energy meter information recognition, maintaining high recognition performance even in complex environments. Furthermore, by combining the multimodal large model with the dedicated model, efficient recognition and structured extraction of elements such as energy meter asset number, reading, time, and seal status are achieved, significantly enhancing the automation level and efficiency of energy meter verification. In addition, this method also supports intelligent matching and anomaly diagnosis functions, which helps to identify and solve problems in a timely manner and ensure the normal operation of the electricity meter management system. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 A flowchart illustrating the intelligent identification method for delayed energy meters based on a dedicated model for multimodal large-scale model fusion. Detailed Implementation

[0034] The present invention will now be further described with reference to the accompanying drawings.

[0035] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be noted that in the embodiments of the present invention, all directional indications (such as up-down-left-right-forward-backward...) are only used to explain the relative positional relationship and movement between the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly. The connection can be a direct connection or an indirect connection.

[0038] like Figure 1 As shown, the intelligent identification method for delayed energy meters based on a multimodal large model fusion dedicated model of the present invention includes the following steps:

[0039] S1. Collect the image dataset of electricity meters and simultaneously extract or annotate the corresponding text semantic information to construct a multimodal large model general recognition framework for the task of recognizing key information of electricity meters.

[0040] S2. To address the insufficient accuracy of multimodal large models in certain fine-grained tasks (such as seal integrity judgment), a corresponding lightweight dedicated recognition model is designed and trained.

[0041] S3. Construct a closed-loop workflow of perception, analysis, and decision-making; the perception layer calls multi-modal large model and special model in parallel to obtain multi-dimensional identification results such as asset number, reading, shooting time, and sealing status; the analysis layer performs logical verification on the identification results based on the preset business rule base (formulated with the participation of power operation and maintenance experts); the decision-making layer integrates multi-source information to generate a structured verification report, marks abnormal items, and supports manual review interface.

[0042] S4 encapsulates the multimodal large model, dedicated recognition model, and business rule base module into an integrated, independently running intelligent recognition program, performing end-to-end processing from image input to structured report output, and incorporates a lightweight feedback closed-loop mechanism for continuous optimization of the model and rules.

[0043] Furthermore, in one embodiment, the electricity meter image dataset in step S1 covers electricity meter image data of different brands, models, installation environments, lighting conditions, and shooting angles; the text semantic information not only comes from the image itself, but also includes task instruction templates, which together constitute the text modality in the multimodal input.

[0044] Furthermore, in one embodiment, in step S1, before annotating the text semantic information, it is necessary to determine the annotation standards and structured field definitions. Each image is annotated in a refined multi-level manner, and the field names include asset number, current description, current reading, shooting time, nameplate area, upper left seal, upper right seal, lower right seal, and bottom seal.

[0045] In one specific embodiment, the Qwen3-VL-32B model is used as the basic architecture when constructing a multimodal large-scale general recognition framework for the task of identifying key information of electricity meters. This model is a large-scale visual language model in the Tongyi Qianwen series that supports joint understanding of images and text, and has powerful cross-modal alignment and instruction following capabilities. To adapt to the vertical domain task of electricity meter recognition, low-rank adaptation (LoRA) technology is used to efficiently fine-tune the model, avoiding the high computational overhead caused by full-parameter training, while retaining its generalization ability. Specifically:

[0046] (1) Dataset Construction

[0047] A total of 40,000 on-site images of electricity meters were collected and cleaned, covering the mainstream single-phase electricity meters used in various cities.

[0048] Lighting conditions include direct strong light, weak light, backlight, and nighttime supplemental lighting;

[0049] Shooting angles include frontal view, oblique view (within ±30°), and partial obstruction;

[0050] Environmental interference includes real-world operation and maintenance scenarios such as glass reflection, stains, moisture, and label application.

[0051] The data was divided into a training set (32,000 images), a validation set (4,000 images), and a test set (4,000 images) in an 8:1:1 ratio.

[0052] (2) Labeling Standards and Structured Field Definitions

[0053] Each image is annotated with fine-grained multi-level annotations, as shown in Table 1. All annotations are manually verified, and the error rate is controlled to be less than 0.1%.

[0054] Table 1

[0055]

[0056]

[0057] (3) Text semantic information extraction and instruction construction

[0058] The text semantic information not only comes from the image itself, but also includes the task instruction template, which together constitute the text modality in the multimodal input. Identify the input picture strictly according to the following requirements, and use a complete JSON object as the only output, without outputting any explanatory text or additional symbols.

[0059] Task requirements:

[0060] 1. First, judge whether the picture of the electric energy meter is complete;

[0061] If the main body of the electric energy meter and the key areas (barcode, display screen, seal) in the picture are clearly visible, it is judged as complete, and output {"picture": "qualified"}; otherwise, output {"picture": "unqualified"}.

[0062] 2. Identify the complete number after "NO" on the barcode of the electric energy meter;

[0063] Output the field "no", and if it cannot be clearly identified, output "0" (string).

[0064] 3. Identify all Chinese characters (such as "current", "time", "total forward", "electricity quantity", etc.) and numbers on the electronic display screen of the electric energy meter;

[0065] Output the field "word", which is an array containing all the Chinese characters visible on the electronic display screen;

[0066] Output the field "number", which is a string, retaining the original display format, such as "250921" or "25.09.21".

[0067] 4. Identify the shooting watermark time in the lower left corner of the picture, and output the fields "time" and the format "YYYY.MM.DD HH:MM", such as "2025.09.21 14:37".

[0068] 5. Judge whether the small round seal in the upper left corner of the electric meter is intact. When it is intact, output {"left": "qualified"}, and when the seal is missing, damaged or due to incomplete shooting, output {"left": "unqualified"}.

[0069] 6. Judge whether the small round seal in the upper right corner of the electric meter is intact. When it is intact, output {"right": "qualified"}, and when the seal is missing, damaged or due to incomplete shooting, output {"right": "unqualified"}.

[0070] 7. Judge whether the small round seal in the lower right corner of the electricity meter is intact. When it is intact, output {"program": "qualified"}; when the seal is missing, damaged, or incomplete due to shooting, output {"program": "unqualified"}.

[0071] 8. Judge whether the small round object tied with a wire on the middle screw at the bottom of the electricity meter is intact. When it is intact, output {"down": "qualified"}; when the seal is missing, damaged, or incomplete due to shooting, output {"down": "unqualified"}.

[0072] Only one JSON object is allowed to be output, and the field order is no, word, number, time, left, right, program, down.

[0073] (4) Fine-tune and train based on Qwen3-VL-32B + LoRA.

[0074] The base model uses Qwen3-VL-32B, its visual encoder is ViT, the language model uses a Dense architecture, and it supports multi-resolution image input.

[0075] Only insert LoRA adapters (rank = 64, alpha =​​​​​​​​​​​​​​​​​​​​An attention mechanism and sample augmentation training strategy are introduced to improve the robustness of the model in complex scenarios such as low light, occlusion, and reflection.

[0083] Furthermore, in one embodiment, the detailed annotation of the sealed area in the energy meter image includes the definition and description of the sealing area bounding box annotation and the sealing status label; the annotation objects of the sealing area bounding box include the seals on the upper left, upper right, lower right and bottom of the energy meter casing, and the sealing status label includes intact, damaged and missing;

[0084] The bounding box format is Pascal VOC format, i.e. (xmin, ymin, xmax, ymax), or COCO format, i.e. (x_top_left, y_top_left, width, height).

[0085] Each image contains four sealed areas, and each sealed area is independently labeled with a bounding box;

[0086] The bounding box should tightly surround the seal body and should not contain background distractions (such as wires or case textures), but slight occlusion edges are allowed.

[0087] The sealing status labels are clearly defined as shown in Table 2.

[0088] Table 2

[0089]

[0090] If the seal is obscured due to the shooting angle (e.g., blocked by a hand or tool) and its state cannot be determined, the sample will not be included in the training set.

[0091] The introduced attention mechanism is the CBAM attention module. The sample augmentation training strategy addresses the typical challenges of field images of electrical energy performance. During training, each image is dynamically combined with 2-3 augmentation operations, and the augmentation intensity and frequency are increased for damaged and missing samples to alleviate class imbalance.

[0092] CBAM is lightweight and plug-and-play, and it models both channel attention and spatial attention, making it suitable for deployment in object detection or classification backbone networks. It has a strong ability to focus on local details (such as seal damage or reflective areas).

[0093] Specifically, in this embodiment, the integration location is embedded after each residual block in the backbone network, with a total of 4 CBAM modules inserted, corresponding to the Stage2-Stage5 outputs.

[0094] Channel attention learns the importance weights of each channel through global average pooling + MLP activation layer, suppresses irrelevant background channels (such as shell texture), and enhances seal-related feature channels.

[0095] Spatial attention is applied along the channel dimension using pooling and convolutional activation layers to learn the importance weights of different locations in the feature map space, generating a spatial weight mask that focuses on the sealed area and weakens occlusions or reflective interference areas.

[0096] Furthermore, in one embodiment, the process in step S3 where the perception layer concurrently invokes the multimodal large model and the dedicated model to obtain multidimensional recognition results such as asset number, display value, shooting time, and sealing status includes:

[0097] The same energy meter image is simultaneously sent to two independent inference channels A and B;

[0098] After inference, the structured JSON output information of channels A and B is obtained, and the output of channel B is mapped to status.

[0099] The results of multimodal and specialized model recognition are fused.

[0100] Among them, channel A (multimodal large model) input image + instruction prompt (Prompt) calls the fine-tuned Qwen3-VL-32B+LoRA model;

[0101] Channel B (dedicated model) takes the same image as input and calls the trained dedicated detection model for the sealing state (based on YOLOv13+CBAM architecture).

[0102] Specifically, after inference, the output of channel A can be the following structured JSON information:

[0103] {"picture": "Pass", "no": "12312312312", "word": ["Current", "Time"], "number": "25.09.22", "time": "2025.11.1911:59", "left": "Pass", "right": "Pass", "program": "Pass", "down": "Pass"};

[0104] The output of channel B can be the following structured information:

[0105] [{"position": "left", "status": "damaged", / / Value: good / damaged / missing "confidence": 0.94, "bbox": [x1, y1, x2, y2]}, {"position": "right", "status": "good", "confid ence": 0.98, "bbox": [x1, y1, x2, y2]}, {"position": "program", "status": "missing", "confidence": 0.91}, {"position": "down", "status": "good", "confidence": 0.96}];

[0106] Perform status mapping on the output of channel B:

[0107] "good" → "qualified";

[0108] "damaged" or "missing" → "unacceptable";

[0109] Specifically, the process of integrating multimodal and specialized model recognition results is as follows:

[0110] Parse the raw output of the large model and extract the left, right, program, and down fields;

[0111] Parse the output of the dedicated model and build a seal status dictionary by position: seal_status = {"left": "unqualified", "right": "qualified", "program": "unqualified", "down": "qualified"};

[0112] If the dedicated model detects a seal at a certain location, the corresponding field in the large model will be forcibly replaced.

[0113] If the dedicated model does not detect a certain position (e.g., there is no down field), the original value of the large model is retained;

[0114] The fused result is as follows: {"picture": "Unqualified", "no": "12312312312", "word": ["current", "time"], "number": "25.09.22", "time": "2025.11.1911:59", "left": "Unqualified", "right": "qualified", "program": "Unqualified", "down": "qualified"};

[0115] In addition, the dedicated model is only allowed to be corrected when confidence ≥ 0.8, in order to avoid false corrections due to low confidence.

[0116] The analysis layer performs logical verification on the identification results based on a pre-defined business rule base (developed with the participation of power operation and maintenance experts). For example: whether the asset number conforms to the State Grid coding standard; whether the current reading is less than the historical record to determine if it is running backwards; whether the shooting time is later than the electricity meter's verification validity period to determine if it is being used beyond its expiration date; and whether the abnormal seal status occurs simultaneously with a sudden change in the reading to determine suspected electricity theft.

[0117] Furthermore, in one embodiment, the business rule base in step S3 is used to automatically verify the consistency and diagnose anomalies of the multidimensional recognition results output by the perception layer. The rule base is encoded in JSON Schema or rule engine (such as Drools) form and supports dynamic loading and version management. The core rules include basic numerical rationality rules, time consistency verification rules, and rules linking the sealing status and business status.

[0118] The basic numerical rationality rules are shown in Table 3 below.

[0119] Table 3

[0120]

[0121] If the reading is negative, it may be due to an OCR error or screen malfunction, and should be marked as invalid data. Time consistency verification rules: Energy meter images typically contain two types of time information:

[0122] Screen Time: The current date and time displayed on the LCD screen of the electricity meter.

[0123] The watermark time is a timestamp automatically embedded by the device when the image is captured, which can come from a mobile phone / inspection terminal. Strong constraint rules are set for these two types of times as shown in Table 4 below.

[0124] Table 4

[0125]

[0126] Example of processing logic:

[0127] Screen time: "2025.11.19 11:59:01";

[0128] Watermark time: "2025-11-19 12:00:22";

[0129] Time difference = 81 seconds → violates R04 → marked as "time out of sync".

[0130] The rules for linking the sealing status with the business status are shown in Table 5 below.

[0131] Table 5

[0132] Furthermore, in one embodiment, the feedback closed-loop mechanism in step S4 includes: performing manual review and returning the manually reviewed and corrected samples to the training set;

[0133] Regularly fine-tune the parameters of the multimodal model and the dedicated model;

[0134] The business rule base is dynamically updated to adapt to changes in policies or equipment standards.

[0135] Furthermore, in one embodiment, the step of performing manual review and returning the manually reviewed and corrected samples to the training set includes:

[0136] The maintenance personnel review the JSON / HTML reports output by the program and correct any errors (such as modifying the seal status or correcting the readings).

[0137] Submit a patch sample via command-line tool; the patch sample is stored in a local cache directory and aggregated monthly.

[0138] A script is automatically triggered monthly to merge and download feedback samples and update them to the main training set.

[0139] Furthermore, in one embodiment, the dynamic updating of the business rule base includes updating the rule base after expert review if the manually submitted data reflects a misjudgment of a certain rule (such as R04 not being applicable under a specific phenotype).

[0140] The integrated, independently operating intelligent recognition program in step S4 adopts modular decoupling and mainly includes the following components as shown in Table 6 below.

[0141] Table 6

[0142]

[0143] To enable continuous evolution of the model and rules, the program incorporates a lightweight feedback loop mechanism, the process of which is as follows:

[0144] Manual review: Operations and maintenance personnel review the JSON / HTML reports output by the program and correct any errors (such as changing the seal status or correcting the readings).

[0145] Feedback submission: Submit corrected samples via command-line tool, in the format {"image_path":"data / meter_001.jpg","corrected_result":{corrected complete fields},"feedback_type":"model_error" / / or"rule_misjudge"};

[0146] Caching and aggregation: Corrected samples are stored in the local feedback_cache / directory and aggregated monthly.

[0147] Regular retraining is performed, with a script automatically triggered monthly to merge and download feedback samples and update them to the main training set;

[0148] When the number of images incorrectly identified by the model reaches a certain level, such as 10,000 images for the multimodal large model and 1,000 images for the dedicated model, incremental fine-tuning can be performed on the multimodal large model and the dedicated model respectively.

[0149] Version release generates a new version package containing updated model weights and rule files for on-site upgrades.

[0150] If the manually submitted data consistently reflects misjudgments of a particular rule (e.g., R04 is not applicable under a specific phenotype), the rule base will be updated after expert review. Through this mechanism, the system possesses continuous learning and evolution capabilities, making it suitable for typical power business scenarios such as large-scale electricity meter extension verification, rotation management, and anti-theft electricity inspection.

[0151] In summary, the present invention provides a method for intelligent identification of delayed energy meters by fusing multimodal large-scale models with dedicated models, comprising the following steps: S1, collecting energy meter image datasets and simultaneously extracting or labeling corresponding textual semantic information to construct a general recognition framework for multimodal large-scale models oriented towards key information identification tasks of energy meters; S2, addressing the insufficient accuracy of multimodal large-scale models in certain fine-grained tasks (such as seal integrity judgment), designing and training corresponding lightweight dedicated recognition models; S3, constructing a closed-loop workflow of perception, analysis, and decision-making; wherein the perception layer concurrently calls the multimodal large-scale model and the dedicated model to obtain multi-dimensional identification results such as asset number, reading, shooting time, and seal status; the analysis layer performs logical verification of the identification results based on a preset business rule base (formulated with the participation of power operation and maintenance experts); and the decision-making layer integrates multi-source information to generate a structured verification report, marks abnormal items, and... Supports manual review interface; S4, encapsulates the multimodal large model, dedicated recognition model, and business rule base module into an integrated, independently running intelligent recognition program, performing end-to-end processing from image input to structured report output, and incorporating a lightweight feedback closed-loop mechanism for continuous optimization of the model and rules; through the above steps, the present invention's multimodal large model fusion dedicated model intelligent recognition method for delayed energy meters constructs an intelligent energy meter recognition system integrating automatic perception, intelligent analysis, and autonomous decision-making. With an intelligent agent working collaboratively with the multimodal large model and dedicated recognition model as its core architecture, it achieves high-precision recognition, structured extraction, consistency verification, and anomaly diagnosis of key elements of the energy meter through a four-step progressive technical route; it can effectively improve the accuracy and robustness of key information recognition of the energy meter, maintaining high recognition performance even in complex environments. Furthermore, by combining the multimodal large model with the dedicated model, it achieves efficient recognition and structured extraction of elements such as energy meter asset number, reading, time, and seal status, greatly improving the automation level and efficiency of energy meter verification. In addition, this method also supports intelligent matching and anomaly diagnosis functions, which helps to identify and solve problems in a timely manner and ensure the normal operation of the electricity meter management system.

[0152] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0153] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for intelligent identification of delayed energy meters based on multimodal large model fusion and dedicated model, characterized in that, Includes the following steps: S1. Collect the image dataset of electricity meters and simultaneously extract or annotate the corresponding text semantic information to construct a multimodal large model general recognition framework for the task of recognizing key information of electricity meters. S2. To address the insufficient accuracy of multimodal large models in certain fine-grained tasks, design and train a corresponding lightweight dedicated recognition model. S3. Construct a closed-loop workflow for perception, analysis, and decision-making; The perception layer calls multimodal large models and special models in parallel to obtain multidimensional recognition results; the analysis layer performs logical verification on the recognition results based on a preset business rule base; and the decision layer integrates multi-source information to generate a structured verification report and marks abnormal items. S4 encapsulates the multimodal large model, dedicated recognition model, and business rule base module into an integrated, independently running intelligent recognition program, performing end-to-end processing from image input to structured report output, and incorporates a lightweight feedback closed-loop mechanism for continuous optimization of the model and rules.

2. The method for intelligent identification of delayed energy meters based on a multimodal large-scale model fusion dedicated model as described in claim 1, characterized in that, The electricity meter image dataset in step S1 covers electricity meter image data of different brands, models, installation environments, lighting conditions and shooting angles; the text semantic information not only comes from the image itself, but also includes task instruction templates, which together constitute the text modality in the multimodal input.

3. The method for intelligent identification of delayed energy meters based on a multimodal large-scale model fusion dedicated model as described in claim 2, characterized in that, In step S1, before annotating the semantic information of the text, it is necessary to determine the annotation standards and structured field definitions. Each image is annotated in a refined multi-level manner. The field names include asset number, current description, current reading, shooting time, nameplate area, upper left seal, upper right seal, lower right seal and bottom seal.

4. The method for intelligent identification of delayed energy meters based on a multimodal large-scale model fusion dedicated model as described in claim 1, characterized in that, Step S2 addresses the issue of insufficient accuracy of large multimodal models in fine-grained tasks by designing and training a lightweight, dedicated recognition model, including: Fine-tune the sealed areas in the electricity meter image; End-to-end training is performed using object detection; By introducing attention mechanisms and sample augmentation training strategies, the robustness of the model in complex scenarios can be improved.

5. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 4, characterized in that, The detailed annotation of the sealed area in the energy meter image includes the definition and description of the sealing area boundary box annotation and the sealing status label; the sealing area boundary box annotation objects include the seals on the upper left, upper right, lower right and bottom of the energy meter shell, and the sealing status label includes intact, damaged and missing; The introduced attention mechanism is the CBAM attention module. The sample augmentation training strategy addresses the typical challenges of field images of electrical energy performance. During training, each image is dynamically combined with 2-3 augmentation operations, and the augmentation intensity and frequency are increased for damaged and missing samples.

6. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 1, characterized in that, In step S3, the perception layer concurrently calls the multimodal large model and the dedicated model to obtain multidimensional recognition results, including: The same energy meter image is simultaneously sent to two independent inference channels A and B; After inference, the structured JSON output information of channels A and B is obtained, and the output of channel B is mapped to status. The results of multimodal and specialized model recognition are fused.

7. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 1, characterized in that, The business rule base in step S3 is used to automatically verify the consistency and diagnose anomalies of the multidimensional recognition results output by the perception layer. The rule base is encoded in JSON Schema or rule engine form. The core rules include basic numerical rationality rules, time consistency verification rules, and rules linking the sealing status and business status.

8. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 1, characterized in that, The feedback closed-loop mechanism in step S4 includes: Perform manual review and then return the manually reviewed and corrected samples to the training set; Regularly fine-tune the parameters of the multimodal model and the dedicated model; The business rule base is dynamically updated to adapt to changes in policies or equipment standards.

9. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 8, characterized in that, The step of performing manual review and returning the manually reviewed and corrected samples to the training set includes: The operations and maintenance personnel review the JSON / HTML reports output by the program and correct any errors. Submit a patch sample via command-line tool; the patch sample is stored in a local cache directory and aggregated monthly. A script is automatically triggered monthly to merge and download feedback samples and update them to the main training set.

10. The method for intelligent identification of delayed energy meters based on a multimodal large model fusion dedicated model as described in claim 8, characterized in that, The dynamically updated business rule base includes a system whereby if manual feedback data consistently reflects misjudgments of a particular rule, the rule base will be updated after expert review.

Images