Power equipment intelligent identification and operation and maintenance management method and system based on multi-modal feedback iteration

By employing a multimodal feedback iterative approach and combining an image and text data Mask R-CNN model, a power equipment identification system was constructed. This system addresses the issues of low identification efficiency and poor adaptability in existing technologies, achieving high accuracy and low cost in equipment management.

CN121998892APending Publication Date: 2026-05-08STATE GRID SHANDONG ELECTRIC POWER CO +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202511792615.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-12-01
Publication Date
2026-05-08

AI Technical Summary

Technical Problem

Existing power equipment identification technologies suffer from low efficiency, large errors, and poor adaptability in manual identification. Furthermore, deep learning-based methods lack semantic understanding capabilities and struggle to adapt to complex scenarios and subtle changes in equipment.

Method used

A multimodal feedback iterative approach is adopted to construct a multimodal standard image library and an AI comparison model library. The model is trained using Mask R-CNN and combined with image and text data to identify and compare device components, thereby achieving real-time image acquisition, AI recognition, comparison verification, and model iteration, and improving recognition accuracy and adaptability.

Benefits of technology

It achieves high accuracy and adaptability in power equipment identification, reduces reliance on manual labor and operation and maintenance costs, and supports refined management under the smart grid.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121998892A_ABST
    Figure CN121998892A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of power equipment management, in particular to a power equipment intelligent identification and operation and maintenance management method and system based on multi-modal feedback iteration, and the method comprises the steps: collecting material codes, text data and multi-angle standard image data of power equipment, and building an equipment standard map database; training and generating an AI model for identifying and comparing the equipment parts based on the standard atlas library, and constructing an AI comparison model library; the method comprises the following steps: collecting a real-time image of a to-be-identified device through a mobile terminal during material warehouse-in and warehouse-out or inventory work; calling an AI model in the AI comparison model library to identify the real-time image, comparing the identified equipment part with standard information in a standard map library, and outputting a comparison result; and verifying the comparison result, and if the identified part is inconsistent with the standard information or part missing exists, giving an alarm in real time. Intelligent upgrading of electrical equipment identification and operation and maintenance is realized, and the operation and maintenance cost is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of power equipment management technology, specifically to a method and system for intelligent identification and operation and maintenance management of power equipment based on multimodal feedback iteration. Background Technology

[0002] Efficient and accurate identification and operation and maintenance management of power equipment are crucial foundations for ensuring the safe and stable operation of the power grid. However, in the current process of digital and intelligent transformation in the power industry, equipment identification and management still face numerous challenges.

[0003] Currently, the mainstream solutions mainly rely on the following two technical paths: First, there are recognition methods based on manual inspection and simple image processing. Manual recognition is prone to missed detections and false detections due to factors such as fatigue and distraction, making it difficult to guarantee recognition accuracy and consistency. Although some solutions have introduced computer vision technology, they mostly use traditional feature extraction algorithms (such as SIFT, SURF, etc.), which have limited feature representation capabilities, poor adaptability to complex scenarios such as changes in lighting, shooting angles, and component occlusion, and weak generalization ability, making it difficult to meet the needs of refined management of power equipment.

[0004] Second, there are automated recognition methods based on deep learning models. While these methods have been widely used in recent years, they still have limitations. First, most methods rely solely on single image modal data, failing to effectively utilize the abundant, equipment-related textual information present in power systems. This results in a lack of semantic understanding in the models, leading to low confidence and high error rates when faced with similar-looking but different-model equipment, or when image quality is poor. Second, existing models are typically static; once trained, their performance is fixed. They cannot iterate and optimize using the massive amounts of field data generated in daily operations, making it difficult to adapt to subtle changes in different substations and operating environments. Recognition performance gradually degrades over time. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a method and system for intelligent identification and operation and maintenance management of power equipment based on multimodal feedback iteration.

[0006] In a first aspect, the technical solution of the present invention provides a method for intelligent identification and operation and maintenance management of power equipment based on multimodal feedback iteration, which is applied to the site of material entry and exit or inventory operation, and includes the following steps: S1. Collect material codes, text data, and multi-angle standard image data of power equipment to establish a standard image library for equipment; based on the standard image library, train and generate an AI model for equipment component identification and comparison, and construct an AI comparison model library; S2. During material entry / exit or inventory operations, real-time images of the equipment to be identified are collected via a mobile terminal; the AI ​​model in the AI ​​comparison model library is called to identify the real-time images, and the identified equipment components are compared with the standard information in the standard image library, and the comparison results are output. S3. Verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, issue a real-time alarm. S4. The verified images and annotation information in the on-site recognition results are used as new training samples. The incremental training of the AI ​​model is triggered periodically to optimize the recognition performance of the model in specific work scenarios, and the optimized model is updated to the AI ​​comparison model library.

[0007] By constructing a multimodal standard image library and an AI comparison model library, a closed-loop process is achieved at the material entry / exit or inventory site: real-time image acquisition → AI recognition → standard comparison → anomaly alarm → sample feedback → model iteration. This overcomes the bottlenecks of low efficiency, large errors, and poor adaptability of traditional manual identification. This method possesses strong scene adaptability and self-evolution capabilities, improving the accuracy of power equipment identification and the level of operational automation, reducing reliance on manual labor and maintenance costs, and supporting the refined and intelligent management needs of equipment in the context of smart grids.

[0008] As a further limitation of the technical solution of the present invention, in S1, the training and generation of the AI ​​model for equipment component identification and comparison specifically involves: training the Mask R-CNN model to achieve target detection and instance segmentation of each component in the power equipment image.

[0009] The Mask R-CNN model combines object detection and instance segmentation capabilities, enabling precise localization and pixel-level recognition of various components in power equipment, thus solving the problem of traditional image recognition's difficulty in distinguishing component details. Addressing the complexity and strong interrelationships of power equipment components, the model effectively identifies the categories and shapes of different components, improving recognition accuracy and reducing misjudgments caused by ambiguous component identification. The model's strong feature extraction capabilities are adaptable to multi-angle and multi-scenario image data of power equipment, providing a precise recognition foundation for subsequent comparison and operation and maintenance management, and supporting refined control.

[0010] As a further limitation of the technical solution of the present invention, the steps for training the Mask R-CNN model are as follows: S11a. Obtain standard image data of the device from the standard image library, and perform pixel-level annotation on the power equipment and key components in the image to generate annotation information including component category, bounding box and instance segmentation mask; S12a. Input standard image data into the backbone network of the initialized Mask R-CNN model to extract multi-scale image feature maps; S13a. Using the image feature map extracted by the backbone network, candidate target regions are generated through the region proposal network of the Mask R-CNN model. S14a. After processing the candidate target regions through the RoIAlign layer of the Mask R-CNN model, input them into the object detection and instance segmentation head, and perform the following operations in parallel: For each candidate target region, perform multi-classification to accurately identify the component type and fine-tune its bounding box position; A binary mask is generated for each candidate target region to achieve pixel-level component instance segmentation; S15a. During training, the model parameters are optimized by minimizing the total loss function of the multi-task task. The loss function is a weighted sum of the classification loss, the bounding box regression loss, and the mask segmentation loss. S16a. Export the trained model as an ONNX format file and deploy it to the AI ​​comparison model library for real-time recognition and retrieval.

[0011] The refined annotation information generated by primitive-level annotation provides rich details for model training, enabling the model to more accurately capture component features and improve the accuracy and reliability of recognition. Multi-scale feature extraction and candidate region selection processes ensure that the model can effectively identify equipment components of different sizes and locations, adapting to the diversity of power equipment images. The optimization of the multi-task loss function simultaneously improves the accuracy of component classification, bounding box localization, and instance segmentation, avoiding overall performance bottlenecks caused by single-task optimization. The export and deployment of the ONNX format model ensures efficient model access in different system environments, meeting the response requirements of real-time recognition at the work site.

[0012] As a further limitation of the technical solution of the present invention, when training the Mask R-CNN model, joint feature learning is performed by fusing text data, specifically including the following steps: S11b: Convert the equipment's material code and text data into a text feature vector through a text embedding layer; S12b. The text feature vector is fused with the image feature map extracted by the backbone network to generate a multimodal fusion feature map that incorporates semantic context information. S13b. Using the multimodal fusion feature map, perform the training process from S13a to S15a. During the training process, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are optimized in an integrated manner through the backpropagation algorithm, and finally a device recognition model that can understand the semantic information of the device and fuses multimodal features is generated.

[0013] The fusion of text embedding and image features enables joint learning of multimodal data, allowing the model to simultaneously capture the device's visual features and semantic information, addressing the lack of contextual support in traditional single-image recognition. An integrated parameter optimization mechanism ensures deep fusion of text and image features, enhancing the model's adaptability to complex scenes, such as accurate recognition even when components are occluded or lighting conditions change. The model, incorporating semantic information, better understands the relationship between device encoding, text descriptions, and components, reducing recognition errors caused by insufficient image information and further improving comparison accuracy.

[0014] As a further limitation of the technical solution of the present invention, the multi-task total loss function in S15a is composed of the weighted sum of classification loss, bounding box regression loss and mask segmentation loss, as shown in the following formula:

[0015] For classifying losses, For bounding box regression loss, For mask segmentation loss, These are the weight coefficients for classification loss, bounding box regression loss, and mask segmentation loss, respectively.

[0016] The weighted summation multi-task total loss function allows for flexible adjustment of the weights of each loss term based on the core requirements of power equipment identification, with a focus on optimizing the identification accuracy of key components. The coordinated optimization of classification loss, bounding box regression loss, and mask segmentation loss ensures that the model achieves high quality in component category judgment, location localization, and pixel-level segmentation, avoiding performance imbalance in a single dimension. The loss function design makes model training more targeted, accelerates convergence, improves training efficiency, and simultaneously ensures the model's generalization ability, adapting to the identification needs of different types of power equipment.

[0017] As a further limitation of the technical solution of the present invention, the step in S13b, which optimizes the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head in an integrated manner through backpropagation algorithm, includes: S13b.1 Calculate the gradient of the total loss L with respect to the final output layer of the model; S13b.2. Propagate the gradient backward along the model architecture from top to bottom, passing through the following paths in sequence: Prediction head path: The gradient flows through the target detection and instance segmentation head, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Region Proposal Network Path: The gradient flows through the region proposal network, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Multimodal fusion path: Gradients from the prediction head path and the region proposal network path converge at the multimodal fusion feature map and continue to propagate backward; Backbone network path: The merged gradients flow through the backbone network of the Mask R-CNN model to calculate the gradients of the image feature extraction parameters; Text embedding layer path: The merged gradients flow through the text embedding layer simultaneously, and the gradients of the text feature extraction and mapping parameters are calculated. S13b.3. Using the optimizer algorithm, based on all the gradients calculated in S13b.2, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are updated in a one-time, synchronous manner.

[0018] Multi-path gradient backpropagation ensures synchronized updates of parameters across modules such as the backbone network and text embedding layers, avoiding insufficient feature fusion caused by optimizing a single module. Gradient convergence at the multimodal fusion feature map strengthens the correlation learning between text and image features, allowing the model to gain a deeper understanding of the intrinsic relationships between multi-dimensional device data. Simultaneous updates of all module parameters at once guarantee the consistency and stability of the model architecture, improving training efficiency and effectiveness, and providing more reliable algorithmic support for accurate recognition.

[0019] As a further limitation of the technical solution of the present invention, step S2 includes: S21. At the material entry and exit or inventory site, real-time images of the electrical equipment to be identified are captured by a camera and uploaded to the system server. S22. After receiving the real-time image, the system server calls the AI ​​model corresponding to the target device in the AI ​​comparison model library to perform forward inference on the real-time image and output the recognition result; the recognition result includes the category, bounding box position and pixel-level segmentation mask of all power equipment components identified in the image; S23. Based on the business document or user selection, retrieve the standard component list and standard images corresponding to the equipment material codes from the standard image library; S24. Compare the identification results with the standard component list and perform the following analysis: Verify that the identified component categories fully cover all required components in the standard component list; Based on the bounding box position and the segmentation mask, analyze whether each component exists, whether its position is correct, and whether its shape is complete. S25. Generate a structured comparison report, the report including at least: Correctly identify components that conform to the standard list; Components that exist in the standard list but are not identified in the live images; Components that are identified but have abnormal shapes or do not conform to the standard list; The identified component bounding boxes and masks are overlaid on the real-time image and displayed side-by-side with the standard image.

[0020] The mobile terminal image acquisition method adapts to the mobile needs of the work site, making data collection more convenient and efficient, breaking the limitations of traditional fixed equipment acquisition. The structured comparison report clearly presents the identification status of various components, intuitively displaying compliant, missing, and abnormal components, allowing business personnel to quickly grasp the equipment status and improve decision-making efficiency. The side-by-side display of real-time and standard images, along with mask overlay effects, provides business personnel with intuitive comparison evidence, reducing the difficulty and error of manual verification and minimizing human oversight.

[0021] As a further limitation of the technical solution of the present invention, step S3 includes: S31. Automatically match and load the corresponding verification rules based on the current job type and equipment type; S32. Receive the comparison result, and based on the loaded verification rules, automatically trigger one or more of the following verifications: Integrity verification: If the list of missing items in the comparison results is not empty, it is determined that a component is missing; Consistency verification: The list of anomalies in the comparison results is compared with the tolerance standard in the verification rules. When the degree of anomaly exceeds the preset threshold, it is determined that the components are inconsistent. Compliance verification: Verify whether the identified component types and quantities comply with current business procedures; S33. Based on the verification results, execute the corresponding real-time alarm strategy: Level 1 Alarm: For situations requiring manual confirmation, the alert will be highlighted and accompanied by text prompts on the mobile terminal and application front-end interface; Level 2 Alarm: For confirmed missing or inconsistent components, an audible alert will be issued while triggering a warning on the front-end interface, and subsequent business operations will be automatically suspended. Level 3 alarm: For anomalies that affect operational safety, an alarm work order will be automatically generated and pushed to the backend management system and the designated person in charge while triggering Level 1 and Level 2 warnings; S34. Regardless of whether an alarm is triggered, the final result of this identification, the verification conclusion, and the operator's handling opinion will be stored in the database as a feedback record.

[0022] The dynamic matching and verification rule design adapts to the needs of different job types and equipment types, improving the targeting and flexibility of verification and avoiding inaccurate verification issues caused by uniform rules. A multi-dimensional verification mechanism comprehensively covers integrity, consistency, and compliance, ensuring the reliability of identification results and minimizing the omission of anomalies. A tiered alarm strategy adopts differentiated handling methods based on the severity of anomalies, ensuring timely response to critical issues while avoiding excessive alarms that could disrupt workflows, thus improving the refinement of operation and maintenance management. Complete storage of feedback records provides rich data support for subsequent model optimization, while also improving management transparency and facilitating the traceability of work processes and processing results.

[0023] As a further limitation of the technical solution of the present invention, step S4 includes: S41. The system automatically filters feedback records that have been verified and confirmed on-site; records that are determined to be correctly identified are used as positive samples, and their images and annotation information are directly stored in the incremental training sample library; records that trigger alarms, or those with identification errors or missing information, are labeled and corrected, and then stored as negative samples in the incremental training sample library. S42. Automatically trigger the model incremental training task according to the preset period, or when the number of new samples in the incremental training sample library reaches the set threshold. S43. When performing incremental training tasks, the specific tasks include: Load the model from the current online service as pre-trained weights from the AI ​​comparison model library; The new samples in the incremental training sample library are mixed with a portion of the samples in the original training dataset to form the dataset used for this incremental training. The model is trained using a learning rate lower than that used during initial training to fine-tune the model parameters; S44. Use a fixed validation set to evaluate the performance of the new model after incremental training. If the overall performance of the new model is not lower than that of the original model, a new version number will be assigned to the new model and it will be deployed to the test environment in the AI ​​comparison model library.

[0024] Classification and labeling correction of positive and negative samples ensure high-quality incremental training samples, providing accurate data for model optimization and improving model iteration efficiency. A dual trigger mechanism of preset cycle and sample quantity threshold ensures timely and reasonable model iteration, avoiding resource waste from frequent training or performance lag caused by insufficient training. Incremental training based on pre-trained weights and low-learning-rate fine-tuning quickly adapts to the recognition needs of new scenarios and devices while ensuring stable model performance, extending the model's lifespan. Rigorous performance evaluation and version management ensure the reliability of deployed models, preventing performance degradation from affecting job quality, while also supporting model traceability and rollback, improving system stability.

[0025] Secondly, the technical solution of the present invention also provides an intelligent identification and operation and maintenance management system for power equipment based on multimodal feedback iteration, which is applied to the site of material entry and exit or inventory operations. The system includes: The standard image and model library construction module is used to collect material codes, text data and multi-angle standard image data of power equipment to establish a standard image library of equipment; and based on the standard image library, to train and generate AI models for equipment component identification and comparison, and to build an AI comparison model library. The on-site operation and real-time comparison module is used to collect real-time images of the equipment to be identified through a mobile terminal during material entry and exit or inventory operations; it calls the AI ​​model in the AI ​​comparison model library to identify the real-time images, compares the identified equipment components with the standard information in the standard image library, and outputs the comparison results. The intelligent verification and alarm module is used to verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, a real-time alarm will be issued. The feedback iteration and model optimization module is used to take the verified images and annotation information in the on-site recognition results as new training samples, periodically trigger incremental training of the AI ​​model to optimize the recognition performance of the model in specific work scenarios, and update the optimized model to the AI ​​comparison model library.

[0026] As can be seen from the above technical solutions, this application has the following advantages: The construction of a standard image library and an AI comparison model library enables standardized management of power equipment data, providing a unified basis for accurate identification and solving the problems of scattered data and lack of unified standards in traditional identification. Real-time image acquisition at the work site and comparison using the model break the time and space limitations of manual identification, improving the efficiency of material entry and exit and inventory operations, and reducing reliance on manual labor. The feedback-iterative incremental training mechanism allows the AI ​​model to continuously adapt to changes in the work scenario, constantly optimizing its recognition performance and solving the pain points of traditional models' weak generalization ability and difficulty in adapting to complex scenarios. The entire process covers data construction, real-time identification, comparison verification, and model optimization, forming a closed-loop management system, realizing the intelligent upgrade of power equipment identification and operation and maintenance, and reducing operation and maintenance costs. Attached Figure Description

[0027] To more clearly illustrate the technical solution of this application, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention.

[0029] Figure 2 A block diagram of a system provided in an embodiment of the present invention. Detailed Implementation

[0030] To make the purpose, features, and advantages of this application more apparent and understandable, specific embodiments and accompanying drawings will be used to clearly and completely describe the technical solution protected by this application. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this application and in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0032] like Figure 1 As shown, this embodiment of the invention provides a method for intelligent identification and operation and maintenance management of power equipment based on multimodal feedback iteration. This embodiment is applied to the on-site operation of power material entry and exit and inventory counting, relying on the WMS warehouse management system and AI technology architecture. The specific steps are as follows: S1. Collect material codes, text data, and multi-angle standard image data of power equipment to establish a standard image library for equipment; based on the standard image library, train and generate an AI model for equipment component identification and comparison, and construct an AI comparison model library; We collected nearly five years of power equipment operation data, covering material codes, text data (specifications, supplier descriptions, and technical parameters) and multi-angle standard image data (front, side, and close-ups of key components) for key equipment such as transformers and switchgear. Equipment experts supplemented the data with comprehensive training photos of the equipment. After annotation and approval, the text data and annotation information were stored in the database, and the standard image files were stored on the OSS server, constructing a structured equipment standard image library to ensure data interoperability with material master data and warehouse master data in the WMS system.

[0033] Based on standard map library data, a deep learning framework is used to train an AI model for equipment component recognition and comparison. The trained models are classified and stored according to equipment type to build an AI comparison model library, which is then integrated with the WMS system's policy management module to support the subsequent use of the corresponding model by material code during operations.

[0034] S2. During material entry / exit or inventory operations, real-time images of the equipment to be identified are collected via a mobile terminal; the AI ​​model in the AI ​​comparison model library is called to identify the real-time images, and the identified equipment components are compared with the standard information in the standard image library, and the comparison results are output. Operators use a mobile app (or a high-definition camera) developed with the Vue.js framework to collect images of the devices to be identified in real time during material entry and exit inspections and inventory scanning. The image data is then uploaded to the system server via a proxy interface / router.

[0035] After receiving the image, the system server retrieves the matching AI model from the AI ​​comparison model library based on the material code of the equipment to be identified, and performs real-time image recognition. After recognition, it retrieves the standard component list (such as the core, windings, oil tank, and other essential components of a transformer) and standard images of the corresponding equipment from the equipment standard image library, compares the identified equipment components with the standard information, and outputs the comparison results including component matching status and differences.

[0036] S3. Verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, issue a real-time alarm. By integrating the quality inspection management module of the WMS system, the results are automatically verified, focusing on checking for issues such as missing components (e.g., a switch cabinet lacking an operating handle) and inconsistent component models (e.g., bolt specifications not conforming to standards). The verification rules are compatible with business procedures such as inventory status transfer and ownership transfer. If the verification finds inconsistent or missing components, the system's real-time alarm mechanism is triggered, and the alarm information is synchronized to the WMS system front-end interface and the operator's mobile terminal to prevent abnormal equipment from entering subsequent warehousing or maintenance stages.

[0037] S4. The verified images and annotation information in the on-site recognition results are used as new training samples. The incremental training of the AI ​​model is triggered periodically to optimize the recognition performance of the model in specific work scenarios, and the optimized model is updated to the AI ​​comparison model library.

[0038] The images and annotation information verified in the on-site recognition results are classified and processed: correctly identified records are directly used as positive samples, and incorrect or missing records that trigger alarms are manually corrected and annotated as negative samples and stored in the incremental training sample library to ensure that the sample data is linked with the feedback record database.

[0039] Incremental training of the model is automatically triggered monthly at a preset cycle (or when the number of incremental samples reaches a threshold of 1000). During training, the current online service model in the AI ​​comparison model library is loaded as pre-training weights, and incremental samples are mixed with the original training data. The model parameters are fine-tuned using a low learning rate (e.g., 1 / 10 of the initial learning rate). After training, the model performance is evaluated through the application monitoring module. If the recognition accuracy (e.g., part recognition accuracy) is not lower than the original model, a version number is assigned to the new model, and it is updated to the AI ​​comparison model library, thereby optimizing the model's performance under specific operational scenarios such as changes in lighting and part occlusion.

[0040] In some embodiments, in S1, training and generating an AI model for equipment component identification and comparison specifically involves training a Mask R-CNN model to achieve target detection and instance segmentation of each component in the power equipment image.

[0041] Based on a standard equipment image library, a Mask R-CNN model was selected to train an AI model for equipment component recognition and comparison. The training process relied on a GPU server and the PyTorch framework to achieve target detection and instance segmentation of various components in power equipment images. Standard image data (with a uniform resolution of 1024×1024) and corresponding text data (preprocessed into structured vectors) from the device's standard image library are used as inputs to the Mask R-CNN model to ensure that the input data format is consistent with the data reception requirements of the model's computation nodes.

[0042] The model extracts image features through a backbone network (such as ResNet-50), generates candidate regions through a region proposal network, completes component category identification and bounding box localization in the target detection head, and generates pixel-level binary masks (such as mask regions to distinguish transformer windings from oil tanks) in the instance segmentation head. Finally, it outputs the identification results containing component category, bounding box coordinates, and segmentation mask.

[0043] The steps for training a Mask R-CNN model are as follows: S11a. Obtain standard image data of the device from the standard image library, and perform pixel-level annotation on the power equipment and key components in the image to generate annotation information including component category, bounding box and instance segmentation mask; Standard equipment images are retrieved from the equipment standard image library. Professional annotators use annotation tools to perform pixel-level annotations on the power equipment and key components (such as circuit breaker contacts and insulating rods) in the images, generating annotation information that includes component category, bounding box coordinates (x1, y1, x2, y2), and instance segmentation mask (pixel-level mask matrix). After annotation, the images and annotation information are divided into training and validation sets in an 8:2 ratio, and the data format is converted to the COCO format supported by the PyTorch framework.

[0044] S12a. Input standard image data into the backbone network of the initialized Mask R-CNN model to extract multi-scale image feature maps; The standard images from the training set are input into the initialized Mask R-CNN model. Multi-scale image feature maps (such as 8×8, 16×16, and 32×32 resolution feature maps) are extracted through the backbone network (ResNet-50+FPN). The feature extraction process utilizes the computing power of the GPU server to ensure that the feature maps can cover the feature information of large components (such as switch cabinets) and small components (such as wiring terminals) of the equipment.

[0045] S13a. Using the image feature map extracted by the backbone network, candidate target regions are generated through the region proposal network of the Mask R-CNN model. Based on the multi-scale feature map output by the backbone network, candidate target regions are generated through the Region Proposal Network (RPN) of Mask R-CNN. The RPN anchor point is set to adapt to the size of the power equipment components (e.g., 16×16 for small components and 128×128 for large components), and candidate regions with confidence scores higher than 0.7 are selected to reduce the amount of subsequent computation.

[0046] S14a. After processing the candidate target regions through the RoIAlign layer of the Mask R-CNN model, input them into the object detection and instance segmentation head, and perform the following operations in parallel: For each candidate target region, perform multi-classification to accurately identify the component type and fine-tune its bounding box position; A binary mask is generated for each candidate target region to achieve pixel-level component instance segmentation; The candidate target regions are input into the RoIAlign layer, uniformly scaled to a 7×7 resolution feature map, and then passed into the object detection and instance segmentation head: The detection head performs multi-classification of candidate regions through a fully connected layer (such as distinguishing between "transformer core, winding, and bushing"), and uses the cross-entropy loss function to optimize the classification accuracy; at the same time, the bounding box regression branch is used to fine-tune the coordinates of candidate regions so that the bounding box fits the actual position of the component better.

[0047] The segmentation head generates a 28×28 resolution binary mask through a convolutional layer. The area with a mask value of 1 corresponds to the target part, and the area with a value of 0 corresponds to the background, thus achieving pixel-level segmentation of the part.

[0048] S15a. During training, the model parameters are optimized by minimizing the total loss function of the multi-task task. The loss function is a weighted sum of the classification loss, the bounding box regression loss, and the mask segmentation loss. The model parameters are optimized using a multi-task total loss function. The loss function consists of a weighted sum of classification loss (Cross-Entropy Loss), bounding box regression loss (Smooth L1 Loss), and mask segmentation loss (Binary Cross-Entropy Loss). The weight coefficients are set according to the priority of power equipment identification (e.g., classification loss weight 0.3, bounding box regression loss weight 0.4, and mask segmentation loss weight 0.3). The model is trained iteratively for 50 rounds using the gradient descent algorithm. After each round of training, the accuracy is evaluated on the validation set to ensure model convergence.

[0049] Multi-task total loss function in S15a Classification loss Bounding box regression loss With mask segmentation loss The weighted summation is calculated using the following steps: The cross-entropy loss function is used to calculate the classification loss by comparing the model's predicted classification results for candidate target regions with the actual component category labels. The formula is as follows:

[0050] N The number of samples in a training batch. C This represents the total number of categories of electrical equipment components. It is an indicator function, when the sample i The true category is c The value is 1 if it is true, and 0 otherwise. The model predicts the sample. i Category c The probability of; The Smooth L1 loss function is used to calculate the difference between the model-predicted bounding box coordinates and the true bounding box coordinates, resulting in the bounding box regression loss, as shown in the following formula:

[0051]

[0052] in, The model predicts the first i The coordinate parameters of a bounding box. These are the coordinate parameters of the corresponding true bounding box. j Including the center point coordinates of the bounding box ( x , y ) as well as width and height ( w , h ); The binary cross-entropy loss function is used to compare the predicted binary mask for each candidate target region with the actual pixel-level segmentation mask to obtain the mask segmentation loss, as shown in the following formula:

[0053] in, M The total number of pixels in each mask. For the first i The true mask of the sample in the th... k The value of each pixel. The mask predicted by the model in the 1st... k The probability value of each pixel; Classification loss Bounding box regression loss With mask segmentation loss The total loss is obtained by weighting and summing the results according to the preset weighting coefficients. The formula is as follows:

[0054] For classifying losses, For bounding box regression loss, For mask segmentation loss, These are the weight coefficients for classification loss, bounding box regression loss, and mask segmentation loss, respectively.

[0055] S16a. Export the trained model as an ONNX format file and deploy it to the AI ​​comparison model library for real-time recognition. After training, export the Mask R-CNN model as an ONNX format file, and then deploy the ONNX model to the AI ​​comparison model library according to the device type. This connects with the WMS system service registration / discovery module, supporting real-time calls via HTTP / HTTPS interfaces during job execution.

[0056] When training the Mask R-CNN model, joint feature learning is performed by fusing text data, specifically including the following steps: S11b: Convert the equipment's material code and text data into a text feature vector through a text embedding layer; S12b. The text feature vector is fused with the image feature map extracted by the backbone network to generate a multimodal fusion feature map that incorporates semantic context information. S13b. Using the multimodal fusion feature map, perform the training process from S13a to S15a. During the training process, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are optimized in an integrated manner through the backpropagation algorithm, and finally a device recognition model that can understand the semantic information of the device and fuses multimodal features is generated.

[0057] The multimodal fusion feature map is incorporated into the training process from S13a to S15a, which involves generating candidate target regions, performing target detection and instance segmentation, and calculating the total loss for multiple tasks. During training, the parameters of the backbone network, text embedding layer, region proposal network, and target detection and instance segmentation heads of Mask R-CNN are simultaneously optimized using the backpropagation algorithm. The optimizer used is AdamW, with a learning rate set to 1e-4, and 60 iterations of training are conducted to ensure that textual semantic information can help improve the accuracy of component recognition. Finally, a device recognition model that fuses multimodal features is generated and deployed to the AI ​​comparison model library.

[0058] It should be noted that the step in S13b, which uses the backpropagation algorithm to simultaneously optimize the parameters of the backbone network, text embedding layer, region proposal network, object detection, and instance segmentation head, includes: S13b.1 Calculate the total loss L The gradient relative to the final output layer of the model; S13b.2. Propagate the gradient backward along the model architecture from top to bottom, passing through the following paths in sequence: Prediction head path: The gradient flows through the target detection and instance segmentation head, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Region Proposal Network Path: The gradient flows through the region proposal network, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Multimodal fusion path: Gradients from the prediction head path and the region proposal network path converge at the multimodal fusion feature map and continue to propagate backward; Backbone network path: The merged gradients flow through the backbone network of the Mask R-CNN model to calculate the gradients of the image feature extraction parameters; Text embedding layer path: The merged gradients flow through the text embedding layer simultaneously, and the gradients of the text feature extraction and mapping parameters are calculated. S13b.3. Using the optimizer algorithm, based on all the gradients calculated in S13b.2, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are updated in a one-time, synchronous manner.

[0059] In some embodiments, step S2 includes: S21. At the material entry and exit or inventory site, real-time images of the electrical equipment to be identified are captured by a camera and uploaded to the system server. During the receiving and acceptance of purchase orders, operators use a mobile app to scan the equipment material barcodes, triggering the camera to automatically capture three images of the equipment from different angles (front, key components, and nameplate). When materials are issued and the equipment is taken out of the warehouse, images of the equipment's current status are captured to ensure consistency with its condition upon receipt. During inventory checks, operators follow the inventory process, using the app to photograph each piece of equipment on the shelves. These images are transmitted to the system server (OSS storage in a temporary directory) with encrypted data to prevent data leakage.

[0060] S22. After receiving the real-time image, the system server calls the AI ​​model corresponding to the target device in the AI ​​comparison model library to perform forward inference on the real-time image and output the recognition result; the recognition result includes the category, bounding box position and pixel-level segmentation mask of all power equipment components identified in the image; After receiving the image, the system server uses the routing and forwarding function of the service access layer to call the matching multimodal Mask R-CNN model from the AI ​​comparison model library according to the equipment material code. The model performs forward inference on the real-time image and outputs the recognition result, which includes the category of each component, bounding box coordinates, and pixel-level segmentation mask. The inference result is stored in the RDS database.

[0061] S23. Based on the business document or user selection, retrieve the standard component list and standard images corresponding to the equipment material codes from the standard image library; Based on the material code in the business document, the system retrieves the standard component list and standard image of the corresponding equipment from the equipment standard drawing library. The standard data is linked with the material batch attribute management module of the WMS system to ensure information consistency.

[0062] S24. Compare the identification results with the standard component list and perform the following analysis: Verify that the identified component categories fully cover all required components in the standard component list; Based on the bounding box position and the segmentation mask, analyze whether each component exists, whether its position is correct, and whether its shape is complete. S25. Generate a structured comparison report, the report including at least: Correctly identify components that conform to the standard list; Components that exist in the standard list but are not identified in the live images; Components that are identified but have abnormal shapes or do not conform to the standard list; The identified component bounding boxes and masks are overlaid on the real-time image and displayed side-by-side with the standard image.

[0063] The system automatically generates a comparison report, which contains three core parts: First, a "List of Compliant Components," listing correctly identified components that conform to the standards; second, a "List of Anomalies," marking missing components (such as "missing sleeves") and components with abnormal shapes (such as "deformed fuel tanks"); and third, a "Visual Comparison Chart," which overlays the component bounding boxes and masks onto the real-time image and displays them side-by-side with the standard image on the mobile app and WMS system front-end interface, allowing operators to intuitively view the differences. The report can be exported as a PDF file for archiving.

[0064] In some embodiments, step S3 includes: S31. Automatically match and load the corresponding verification rules based on the current job type and equipment type; S32. Receive the comparison result, and based on the loaded verification rules, automatically trigger one or more of the following verifications: Integrity verification: If the list of missing items in the comparison results is not empty, it is determined that a component is missing; Consistency verification: The list of anomalies in the comparison results is compared with the tolerance standard in the verification rules. When the degree of anomaly exceeds the preset threshold, it is determined that the components are inconsistent. Compliance verification: Verify whether the identified component types and quantities comply with current business procedures; S33. Based on the verification results, execute the corresponding real-time alarm strategy: Level 1 Alarm: For situations requiring manual confirmation, the alert will be highlighted and accompanied by text prompts on the mobile terminal and application front-end interface; Level 2 Alarm: For confirmed missing or inconsistent components, an audible alert will be issued while triggering a warning on the front-end interface, and subsequent business operations will be automatically suspended. Level 3 alarm: For anomalies that affect operational safety, an alarm work order will be automatically generated and pushed to the backend management system and the designated person in charge while triggering Level 1 and Level 2 warnings; S34. Regardless of whether an alarm is triggered, the final result of this identification, the verification conclusion, and the operator's handling opinion will be stored in the database as a feedback record.

[0065] In some embodiments, step S4 includes: S41. The system automatically filters feedback records that have been verified and confirmed on-site; records that are determined to be correctly identified are used as positive samples, and their images and annotation information are directly stored in the incremental training sample library; records that trigger alarms, or those with identification errors or missing information, are labeled and corrected, and then stored as negative samples in the incremental training sample library. S42. Automatically trigger the model incremental training task according to the preset period, or when the number of new samples in the incremental training sample library reaches the set threshold. S43. When performing incremental training tasks, the specific tasks include: Load the model from the current online service as pre-trained weights from the AI ​​comparison model library; The new samples in the incremental training sample library are mixed with a portion of the samples in the original training dataset to form the dataset used for this incremental training. The model is trained for a limited number of rounds using a learning rate lower than that used in the initial training, in order to fine-tune the model parameters. S44. Use a fixed validation set to evaluate the performance of the new model after incremental training. If the overall performance of the new model is not lower than that of the original model, a new version number will be assigned to the new model and it will be deployed to the test environment in the AI ​​comparison model library.

[0066] like Figure 2 As shown, this embodiment of the invention also provides a power equipment intelligent identification and operation and maintenance management system based on multimodal feedback iteration, applied to material entry and exit or inventory operation sites. The system includes: The standard image and model library construction module is used to collect material codes, text data and multi-angle standard image data of power equipment to establish a standard image library of equipment; and based on the standard image library, to train and generate AI models for equipment component identification and comparison, and to build an AI comparison model library. The on-site operation and real-time comparison module is used to collect real-time images of the equipment to be identified through a mobile terminal during material entry and exit or inventory operations; it calls the AI ​​model in the AI ​​comparison model library to identify the real-time images, compares the identified equipment components with the standard information in the standard image library, and outputs the comparison results. The intelligent verification and alarm module is used to verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, a real-time alarm will be issued. The feedback iteration and model optimization module is used to take the verified images and annotation information in the on-site recognition results as new training samples, periodically trigger incremental training of the AI ​​model to optimize the recognition performance of the model in specific work scenarios, and update the optimized model to the AI ​​comparison model library.

[0067] In some embodiments, the standard atlas and model library construction module is specifically used to train a Mask R-CNN model to achieve target detection and instance segmentation of various components in power equipment images. During training, the standard atlas and model library construction module is further used to: Standard image data of the device is obtained from the standard image library, and the power equipment and key components in the image are annotated at the pixel level to generate annotation information including component category, bounding box and instance segmentation mask; Standard image data is input into the backbone network of an initialized Mask R-CNN model to extract multi-scale image feature maps; Using the image feature maps extracted by the backbone network, candidate target regions are generated through the region proposal network of the Mask R-CNN model; After the candidate target regions are processed through the RoIAlign layer of the Mask R-CNN model, they are input into the object detection and instance segmentation head, which performs the following operations in parallel: performs multi-classification on each candidate target region to accurately identify the part type and fine-tunes its bounding box position; generates a binary mask for each candidate target region to achieve pixel-level part instance segmentation; During training, the model parameters are optimized by minimizing the total multi-task loss function, which is a weighted sum of classification loss, bounding box regression loss, and mask segmentation loss. The trained model is exported as an ONNX format file and deployed to the AI ​​comparison model library.

[0068] In some embodiments, the standard atlas and model library construction module is also used to fuse text data for joint feature learning during training, specifically including: The equipment's material codes and text data are converted into text feature vectors through a text embedding layer. The text feature vector is fused with the image feature map extracted by the backbone network to generate a multimodal fusion feature map that incorporates semantic context information; Using the multimodal fusion feature map, subsequent region proposal, target recognition, and segmentation training processes are performed. During training, the parameters of the backbone network, text embedding layer, region proposal network, target detection, and instance segmentation head are simultaneously optimized through backpropagation algorithm, ultimately generating a device recognition model that can understand device semantic information and integrates multimodal features.

[0069] In some embodiments, the system employs a Spring Cloud-based microservice architecture and includes: The mobile front-end module, developed based on the Vue.js framework and deployed on mobile terminals, is used to implement image acquisition, result display, and human-computer interaction. The backend service module is used for scheduling AI models, processing business logic, data storage and retrieval, and service discovery. The model service module is dedicated to deploying and calling models in the AI ​​comparison model library, and provides an API interface for incremental model training. The data storage module adopts a storage-compute separation and distributed storage architecture, using a relational database to store standard map and text data, and a distributed object storage service to store image data and model files.

[0070] In some embodiments, the on-site operation and real-time comparison module is specifically used for: At the material entry and exit or inventory site, real-time images of the electrical equipment to be identified are captured by cameras and uploaded to the system server; Call the AI ​​model corresponding to the target device in the AI ​​comparison model library to perform forward inference on the real-time image and output the recognition results including part category, bounding box position and pixel-level segmentation mask; Based on business documents or user selection, retrieve the standard component list and standard images corresponding to the equipment material codes from the standard image library; The identification results are compared with the standard component list to verify the completeness, location and integrity of the components, and a structured comparison report is generated.

[0071] In some embodiments, the intelligent verification and alarm module is specifically used for: Based on the current job type and equipment type, automatically match and load the corresponding verification rules; Based on the aforementioned verification rules, integrity verification, consistency verification, and compliance verification are automatically triggered. Based on the verification results, implement a tiered real-time alarm strategy, including front-end notifications, sound warnings and suspension of business processes, and generation and push of alarm work orders; The final result, verification conclusion, and processing opinion of each identification are stored in the database as a feedback record.

[0072] In some embodiments, the feedback iteration and model optimization module is specifically used for: The system automatically filters feedback records that have been verified on-site, uses correctly identified records as positive samples, and uses corrected incorrectly identified records as difficult negative samples, and stores them in the incremental training sample library. Automatically trigger incremental training tasks for the model according to a preset period or sample size threshold; When performing incremental training, load the online model weights, mix the old and new datasets, and fine-tune with a low learning rate; The performance of the new model after incremental training is verified. If the performance is not lower than that of the original model, a new version number is assigned to it and it is hot-updated to the AI ​​comparison model library in the production environment.

[0073] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent identification and operation and maintenance management of power equipment based on multimodal feedback iteration, characterized in that, Applied to material inbound / outbound or inventory operations, it includes the following steps: S1. Collect material codes, text data, and multi-angle standard image data of power equipment to establish a standard image library for equipment; based on the standard image library, train and generate an AI model for equipment component identification and comparison, and construct an AI comparison model library; S2. During material entry / exit or inventory operations, real-time images of the equipment to be identified are collected via a mobile terminal; the AI ​​model in the AI ​​comparison model library is called to identify the real-time images, and the identified equipment components are compared with the standard information in the standard image library, and the comparison results are output. S3. Verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, issue a real-time alarm. S4. The verified images and annotation information in the on-site recognition results are used as new training samples. The incremental training of the AI ​​model is triggered periodically to optimize the recognition performance of the model in specific work scenarios, and the optimized model is updated to the AI ​​comparison model library.

2. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 1, characterized in that, In S1, the AI ​​model for equipment component identification and comparison is trained by using the Mask R-CNN model to achieve target detection and instance segmentation of each component in the power equipment image.

3. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 2, characterized in that, The steps for training a Mask R-CNN model are as follows: S11a. Obtain standard image data of the device from the standard image library, and perform pixel-level annotation on the power equipment and key components in the image to generate annotation information including component category, bounding box and instance segmentation mask; S12a. Input standard image data into the backbone network of the initialized Mask R-CNN model to extract multi-scale image feature maps; S13a. Using the image feature map extracted by the backbone network, candidate target regions are generated through the region proposal network of the Mask R-CNN model. S14a. After processing the candidate target regions through the RoIAlign layer of the Mask R-CNN model, input them into the object detection and instance segmentation head, and perform the following operations in parallel: For each candidate target region, perform multi-classification to accurately identify the component type and fine-tune its bounding box position; A binary mask is generated for each candidate target region to achieve pixel-level component instance segmentation; S15a. During training, the model parameters are optimized by minimizing the total loss function of the multi-task task. The loss function is a weighted sum of the classification loss, the bounding box regression loss, and the mask segmentation loss. S16a. Export the trained model as an ONNX format file and deploy it to the AI ​​comparison model library for real-time recognition and retrieval.

4. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 3, characterized in that, When training the Mask R-CNN model, joint feature learning is performed by fusing text data, specifically including the following steps: S11b: Convert the equipment's material code and text data into a text feature vector through a text embedding layer; S12b. The text feature vector is fused with the image feature map extracted by the backbone network to generate a multimodal fusion feature map that incorporates semantic context information. S13b. Using the multimodal fusion feature map, perform the training process from S13a to S15a. During the training process, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are optimized in an integrated manner through the backpropagation algorithm, and finally a device recognition model that can understand the semantic information of the device and fuses multimodal features is generated.

5. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 4, characterized in that, Multi-task total loss function in S15a Classification loss Bounding box regression loss With mask segmentation loss The weighted summation is calculated using the following formula: For classifying losses, For bounding box regression loss, For mask segmentation loss, These are the weight coefficients for classification loss, bounding box regression loss, and mask segmentation loss, respectively.

6. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 5, characterized in that, The step in S13b that integrates the optimization of parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head through backpropagation algorithm includes: S13b.1 Calculate the total loss L The gradient relative to the final output layer of the model; S13b.

2. Propagate the gradient backward along the model architecture from top to bottom, passing through the following paths in sequence: Prediction head path: The gradient flows through the target detection and instance segmentation head, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Region Proposal Network Path: The gradient flows through the region proposal network, calculates the gradient of the parameters of that part, and continues to propagate back to its input. Multimodal fusion path: Gradients from the prediction head path and the region proposal network path converge at the multimodal fusion feature map and continue to propagate backward; Backbone network path: The merged gradients flow through the backbone network of the Mask R-CNN model to calculate the gradients of the image feature extraction parameters; Text embedding layer path: The merged gradients flow through the text embedding layer simultaneously, and the gradients of the text feature extraction and mapping parameters are calculated. S13b.

3. Using the optimizer algorithm, based on all the gradients calculated in S13b.2, the parameters of the backbone network, text embedding layer, region proposal network, object detection and instance segmentation head are updated in a one-time, synchronous manner.

7. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 6, characterized in that, The steps in S2 include: S21. At the material entry and exit or inventory site, real-time images of the electrical equipment to be identified are captured by a camera and uploaded to the system server. S22. After receiving the real-time image, the system server calls the AI ​​model corresponding to the target device in the AI ​​comparison model library to perform forward inference on the real-time image and output the recognition result; the recognition result includes the category, bounding box position and pixel-level segmentation mask of all power equipment components identified in the image; S23. Based on the business document or user selection, retrieve the standard component list and standard images corresponding to the equipment material codes from the standard image library; S24. Compare the identification results with the standard component list and perform the following analysis: Verify that the identified component categories fully cover all required components in the standard component list; Based on the bounding box position and the segmentation mask, analyze whether each component exists, whether its position is correct, and whether its shape is complete. S25. Generate a structured comparison report, the report including at least: Correctly identify components that conform to the standard list; Components that exist in the standard list but are not identified in the live images; Components that are identified but have abnormal shapes or do not conform to the standard list; The identified component bounding boxes and masks are overlaid on the real-time image and displayed side-by-side with the standard image.

8. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 7, characterized in that, The steps in S3 include: S31. Automatically match and load the corresponding verification rules based on the current job type and equipment type; S32. Receive the comparison result, and based on the loaded verification rules, automatically trigger one or more of the following verifications: Integrity verification: If the list of missing items in the comparison results is not empty, it is determined that a component is missing; Consistency verification: The list of anomalies in the comparison results is compared with the tolerance standard in the verification rules. When the degree of anomaly exceeds the preset threshold, it is determined that the components are inconsistent. Compliance verification: Verify whether the identified component types and quantities comply with current business procedures; S33. Based on the verification results, execute the corresponding real-time alarm strategy: Level 1 Alarm: For situations requiring manual confirmation, the alert will be highlighted and accompanied by text prompts on the mobile terminal and application front-end interface; Level 2 Alarm: For confirmed missing or inconsistent components, an audible alert will be issued while triggering a warning on the front-end interface, and subsequent business operations will be automatically suspended. Level 3 alarm: For anomalies that affect operational safety, an alarm work order will be automatically generated and pushed to the backend management system and the designated person in charge while triggering Level 1 and Level 2 warnings; S34. Regardless of whether an alarm is triggered, the final result of this identification, the verification conclusion, and the operator's handling opinion will be stored in the database as a feedback record.

9. The intelligent identification and operation and maintenance management method for power equipment based on multimodal feedback iteration according to claim 8, characterized in that, The steps in S4 include: S41. The system automatically filters feedback records that have been verified and confirmed on-site; records that are determined to be correctly identified are used as positive samples, and their images and annotation information are directly stored in the incremental training sample library; records that trigger alarms, or those with identification errors or missing information, are labeled and corrected, and then stored as negative samples in the incremental training sample library. S42. Automatically trigger the model incremental training task according to the preset period, or when the number of new samples in the incremental training sample library reaches the set threshold. S43. When performing incremental training tasks, the specific tasks include: Load the model from the current online service as pre-trained weights from the AI ​​comparison model library; The new samples in the incremental training sample library are mixed with a portion of the samples in the original training dataset to form the dataset used for this incremental training. The model is trained using a learning rate lower than that used during initial training to fine-tune the model parameters; S44. Use a fixed validation set to evaluate the performance of the new model after incremental training. If the overall performance of the new model is not lower than that of the original model, a new version number will be assigned to the new model and it will be deployed to the test environment in the AI ​​comparison model library.

10. A power equipment intelligent identification and operation and maintenance management system based on multimodal feedback iteration, characterized in that, The system, applied to material handling or inventory operations, includes: The standard image and model library construction module is used to collect material codes, text data and multi-angle standard image data of power equipment to establish a standard image library of equipment; and based on the standard image library, to train and generate AI models for equipment component identification and comparison, and to build an AI comparison model library. The on-site operation and real-time comparison module is used to collect real-time images of the equipment to be identified through a mobile terminal during material entry and exit or inventory operations; it calls the AI ​​model in the AI ​​comparison model library to identify the real-time images, compares the identified equipment components with the standard information in the standard image library, and outputs the comparison results. The intelligent verification and alarm module is used to verify the comparison results. If the identified component is inconsistent with the standard information or a component is missing, a real-time alarm will be issued. The feedback iteration and model optimization module is used to take the verified images and annotation information in the on-site recognition results as new training samples, periodically trigger incremental training of the AI ​​model to optimize the recognition performance of the model in specific work scenarios, and update the optimized model to the AI ​​comparison model library.