Multi-mode AI image recognition method and system applied to arrival goods acceptance of power equipment

Abstract

The invention relates to the technical field of image recognition, and discloses a multi-modal AI image recognition method and system applied to arrival goods acceptance of power equipment, and the method comprises the steps: obtaining multi-modal data of target equipment, and carrying out the standardization of the multi-modal data, and obtaining the standardized data; according to a preset acceptance standard, abnormal contours and thermal anomaly features in the data are extracted; performing cross-modal correlation analysis on the two features to obtain composite defect features; the composite defect feature confidence is evaluated, and a high-confidence evaluation report is generated; and combining acceptance standard matching report compliance to form a structured acceptance report. According to the scheme, the acceptance accuracy and compliance are improved, and the safety risk is reduced. According to the scheme, the problems of incomplete single-mode detection, unassociated multi-mode data, no confidence evaluation of results, difficulty in compliance matching of reports, easiness in missing judgment and existence of potential safety hazards in existing acceptance check are solved.

Description

Multimodal AI Image Recognition Method and System for Power Equipment Delivery Acceptance Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a multimodal AI image recognition method and system for the acceptance of incoming power equipment. Background Technology

[0002] Image recognition technology is now being used to some extent in the acceptance of incoming power equipment. Single-modal data is often used for defect detection. By using preset image comparison algorithms or simple feature extraction models, it is determined whether there is obvious damage to the appearance of the equipment. In some scenarios, human experience is combined to verify the recognition results, which helps to complete the acceptance process. To a certain extent, it has replaced the traditional method of purely manual visual inspection and improved the efficiency of basic acceptance.

[0003] However, existing technologies can only conduct inspections based on single-modal data, and cannot simultaneously acquire equipment contour and thermal distribution information, making it difficult to comprehensively identify both explicit and implicit defects. Furthermore, the lack of standardized processing and cross-modal correlation analysis of multimodal data makes it impossible to establish the correlation between abnormal contour features and thermal anomaly features, easily leading to the omission of compound defects. At the same time, the lack of a scientific confidence assessment mechanism means that the accuracy of the identification results lacks quantitative support, and the final acceptance report is difficult to accurately match with equipment acceptance standards, increasing the cost of manual verification and potentially creating hidden dangers for equipment operation safety due to judgment bias. Summary of the Invention

[0004] This invention provides a multimodal AI image recognition method and system for power equipment delivery acceptance. Its main purpose is to solve the problems in the existing power equipment delivery acceptance process, such as incomplete single-modal detection, lack of correlation between multimodal data, lack of confidence assessment of results, difficulty in compliance matching of reports, easy omission of defects, and potential safety hazards.

[0005] To achieve the above objectives, the present invention provides a multimodal AI image recognition method for power equipment delivery acceptance, comprising: S1: acquiring multimodal data of the target equipment and standardizing the multimodal data to obtain standardized data of the target equipment; S2: extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards; S3: performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain composite defect features of the target equipment; S4: performing confidence assessment on the composite defect features to obtain a high-confidence assessment report of the target equipment; S5: performing compliance matching on the high-confidence assessment report based on the equipment acceptance standards to obtain a structured acceptance report of the target equipment.

[0006] Preferably, acquiring the multimodal data of the target device includes: acquiring a polarization image sequence of the target device based on a preset optical sensor; acquiring an infrared thermal image sequence of the target device based on a preset thermodynamic sensor; and registering the polarization image sequence and the infrared thermal image sequence to obtain the multimodal data of the target device.

[0007] Preferably, the step of standardizing the multimodal data to obtain standardized data of the multimodal data includes: performing grayscale normalization on the multimodal data to obtain grayscale standard data of the target device; and performing background suppression on the grayscale standard data to obtain standardized data of the target device.

[0008] Preferably, the step of extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance criteria includes: performing deformation mapping on the standard contour of the target equipment based on the polarization image sequence to obtain candidate abnormal features of the target equipment; performing dynamic verification on the candidate abnormal features to obtain abnormal contour features of the target equipment; performing thermal flow analysis on the abnormal region of the target equipment based on the infrared thermal image sequence and the main load-bearing direction of the target equipment to obtain preliminary thermal anomaly features of the target equipment; and applying physical constraints to the preliminary thermal anomaly features to obtain the thermal anomaly features of the target equipment.

[0009] Preferably, the step of performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain the composite defect features of the target device includes: performing cross-modal alignment on the target device based on the abnormal contour features and the thermal anomaly features to obtain the spatial defect features of the target device; performing physical coupling analysis on the target device based on the spatial defect features to obtain the failure defect features of the target device; and performing dynamic verification on the target device based on the failure defect features and the real-time operating parameters of the target device to obtain the composite defect features of the target device.

[0010] Preferably, the step of performing cross-modal alignment of the target device based on the abnormal contour features and the thermal anomaly features to obtain the spatially associated defect features of the target device includes: performing geometric reference matching on the target device based on the abnormal contour features and the thermal anomaly features to obtain the modal registration reference points of the target device; performing deformation compensation on the target device based on the modal registration reference points to obtain the precise registration features of the target device; and performing spatial overlay verification on the abnormal contour features and the thermal anomaly features based on the precise registration feature mapping to obtain the spatially associated defect features of the target device.

[0011] Preferably, the step of assessing the confidence level of the composite defect features to obtain a high-confidence assessment report for the target device includes: performing multi-dimensional parameter coupling analysis on the composite defect features based on a preset dynamic benchmark library to obtain a physical consistency score for the composite defect features; verifying the defect evolution trend of the target device based on the physical consistency score to obtain a stability level for the target device; and determining the graded confidence level of the target device based on the stability level to obtain a high-confidence assessment report for the target device.

[0012] Preferably, the step of performing multidimensional parameter coupling analysis on the composite defect features based on a preset dynamic benchmark library to obtain the physical consistency score of the composite defect features includes: performing cross-modal alignment matching on the composite defect features based on the dynamic benchmark library to obtain the modal correlation matrix of the composite defect features; performing parameter coupling calculation on the composite defect features based on the modal correlation matrix to obtain the physical coupling coefficient of the composite defect features; and performing normalized score integration on the composite defect features based on the physical coupling coefficient to obtain the physical consistency score of the composite defect features.

[0013] Preferably, the step of performing compliance matching on the high-confidence assessment report based on the equipment acceptance criteria to obtain a structured acceptance report for the target equipment includes: performing a compliance traversal on the high-confidence assessment report based on the equipment acceptance criteria to obtain a violation determination result of the high-confidence assessment report; and archiving the violation determination result as a structured acceptance report for the target equipment.

[0014] To address the aforementioned issues, this invention also provides a multimodal AI image recognition system for power equipment delivery acceptance, comprising: an acquisition module for acquiring multimodal data of a target device and standardizing the multimodal data to obtain standardized data of the target device; an extraction module for extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards; a correlation analysis module for performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain composite defect features of the target device; an evaluation module for evaluating the confidence level of the composite defect features to obtain a high-confidence evaluation report of the target device; and an acceptance module for performing compliance matching on the high-confidence evaluation report based on the equipment acceptance standards to obtain a structured acceptance report of the target device.

[0015] Beneficial Effects: This solution standardizes multimodal data through grayscale normalization and background suppression, eliminating sensor differences and environmental interference. Based on equipment acceptance standards, it extracts accurate abnormal contour features through dynamic verification, filters reliable thermal anomaly features through physical constraints, and finally constructs composite defect features through cross-modal alignment and physical coupling analysis. This can comprehensively capture composite defects that are easily missed, such as cabinet dents accompanied by local high temperatures, significantly improving the comprehensiveness and accuracy of defect identification and avoiding potential safety hazards in equipment operation due to the limitations of a single mode.

[0016] Meanwhile, relying on the dynamic benchmark library, multi-dimensional parameter coupling analysis is carried out. The physical coupling coefficient is calculated through the modal correlation matrix and converted into a physical consistency score. The stability level is verified and determined in combination with the defect evolution trend. Finally, a high-confidence assessment report is generated, which provides quantitative support for defect judgment and avoids the subjective bias of traditional manual review. Attached Figure Description

[0017] Figure 1 is a flowchart illustrating a multimodal AI image recognition method for power equipment arrival acceptance provided in an embodiment of the present invention; Figure 2 is a functional block diagram of a multimodal AI image recognition system for power equipment arrival acceptance provided in an embodiment of the present invention; In the figures: 100, multimodal AI image recognition system for power equipment arrival acceptance; 101, acquisition module; 102, extraction module; 103, correlation analysis module; 104, evaluation module; 105, acceptance module.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This application provides a multimodal AI image recognition method and system for power equipment arrival and acceptance. The executing entity of the multimodal AI image recognition method and system for power equipment arrival and acceptance includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the multimodal AI image recognition method and system for power equipment arrival and acceptance can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks, and big data and artificial intelligence platforms.

[0021] Referring to Figure 1, a flowchart illustrating a multimodal AI image recognition method for power equipment arrival acceptance is provided according to an embodiment of the present invention. In this embodiment, the multimodal AI image recognition method for power equipment arrival acceptance includes: S1: acquiring multimodal data of the target equipment and standardizing the multimodal data to obtain standardized data of the target equipment; in this embodiment, acquiring the multimodal data of the target equipment includes: acquiring a polarization image sequence of the target equipment based on a preset optical sensor; acquiring an infrared thermal image sequence of the target equipment based on a preset thermodynamic sensor; registering the polarization image sequence and the infrared thermal image sequence to obtain the multimodal data of the target equipment.

[0022] Specifically, target equipment refers to power-related equipment to be tested in the power equipment delivery and acceptance scenario, such as transformers, switchgear, cable terminals, etc., which need to be confirmed through acceptance to determine whether they meet quality standards. It is the core object of multimodal data acquisition and analysis.

[0023] Multimodal data includes two types of data: polarization image sequences and infrared thermographic sequences of the target device. The polarization image sequences are used to reflect the device's appearance outline, surface texture, and visible structural features, while the infrared thermographic sequences are used to reflect the heat distribution on the device's surface and inside, and can capture hidden thermal anomalies. After registration, the two types of data together constitute multimodal data, providing comprehensive information support for subsequent defect detection.

[0024] The preset optical sensor is an optical imaging device with pre-set model, parameters and acquisition position. It has the function of capturing polarized light information. Its parameters need to be adapted according to the size, material and acceptance environment of the power equipment to ensure that the acquired polarized image sequence can clearly present the outline details of the equipment. For example, for switch cabinets made of metal, the polarization angle of the sensor needs to be adjusted to reduce the impact of surface reflection on image quality.

[0025] A polarization image sequence is a collection of images containing polarization information of different angles of a target device, continuously acquired by a preset optical sensor at a set frequency. It can record whether there are obvious contour abnormalities such as deformation, scratches, or damage on the surface of the device. The acquisition frequency needs to be determined according to the size of the device.

[0026] The preset thermodynamic sensor is a pre-configured detection device with infrared thermal imaging capabilities. Its parameters must match the normal operating temperature range of the power equipment and the acceptance environment temperature. For example, for cable terminals with a normal operating temperature of -20℃ to 150℃, the sensor temperature measurement range must be set to -30℃ to 200℃, and the thermal sensitivity must be no less than 0.05℃ to accurately capture minute thermal anomalies in the equipment. In addition, the sensor installation location must avoid interference from heat sources in the acceptance site to ensure the accuracy of the collected data.

[0027] Infrared thermal imaging sequences are image sequences that are continuously acquired by preset thermodynamic sensors and reflect the surface temperature distribution of the target equipment. They can show local temperature anomalies caused by internal faults in the equipment. The acquisition time needs to be determined according to the thermal stability characteristics of the equipment. For transformers with slow heat dissipation, the acquisition time should be no less than 5 minutes to obtain stable data on the thermal distribution of the equipment.

[0028] Furthermore, firstly, the preset optical sensor is activated to perform an all-round scan of the target power equipment according to the pre-set acquisition frequency, polarization angle and other parameters, continuously acquiring polarization images of the equipment at different angles to form a polarization image sequence; at the same time, the preset thermodynamic sensor is activated to perform continuous thermal imaging acquisition of the target equipment based on the temperature measurement range, thermal sensitivity and other parameters adapted to the characteristics of the equipment, at an acquisition position that avoids heat source interference in the site, to obtain an infrared thermal image sequence reflecting the temperature distribution of the equipment.

[0029] Subsequently, an image registration algorithm is used to spatially align the acquired polarization image sequence with the infrared thermal image sequence, eliminating spatial deviations caused by differences in acquisition angle and sensor position between the two types of images. This ensures that the device outline position in the polarization image corresponds one-to-one with the device thermal distribution position in the infrared thermal image, ultimately obtaining multimodal data of the target device that can be used for subsequent analysis.

[0030] Next, the multimodal data is subjected to grayscale normalization to map the grayscale values ​​of different modal data to the same range, eliminating data scale differences. Then, the background suppression algorithm is used to remove background areas in the image that are not related to the target device, highlighting the main features of the device, thereby obtaining standardized data. This prepares the data for subsequent extraction of abnormal contour features and thermal anomaly features based on preset equipment acceptance standards.

[0031] In this embodiment, the standardization process of the multimodal data to obtain standardized data of the multimodal data includes: performing grayscale normalization on the multimodal data to obtain grayscale standard data of the target device; and performing background suppression on the grayscale standard data to obtain standardized data of the target device.

[0032] Specifically, grayscale standard data is intermediate data obtained after grayscale normalization of multimodal data. Its core is to uniformly map the grayscale values ​​of different pixels in polarized image sequences and infrared thermal image sequences to a preset standard grayscale range, eliminating grayscale value fluctuations caused by differences in sensor sensitivity and changes in ambient light during acquisition, so that the data has a unified grayscale reference benchmark.

[0033] Standardized data is the final data obtained after background suppression processing of grayscale standard data. It mainly contains the effective feature areas of the target device, removes irrelevant background elements in the acceptance scene, and uses algorithms to suppress the grayscale values ​​of the background area to the lowest level, highlighting the outline and thermal distribution characteristics of the device body, and reducing interference for subsequent abnormal feature extraction.

[0034] Furthermore, first determine the grayscale adjustment benchmark under the acceptance scenario, and combine the grayscale characteristics of common materials of power equipment. Then, through linear or nonlinear transformation, map the grayscale values ​​of all pixels in the multimodal data to the standard range of 0-255. During the process, it is necessary to correct the grayscale deviation caused by uneven lighting at the acceptance site or differences in sensor parameters. Finally, obtain grayscale standard data with uniform grayscale value distribution and unified reference standard.

[0035] Next, the background area in the acceptance scene is identified by image analysis algorithm. The identification model of the equipment area is established by combining the shape, size and structural features of the power equipment. Then, threshold segmentation or background subtraction is used to suppress the gray value of the background area that does not belong to the equipment body in the gray standard data to 0, and only the gray feature of the equipment area is retained. This eliminates the interference of background elements on subsequent defect detection, and finally obtains standardized data that only contains the effective features of the equipment.

[0036] S2: Extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards; In this embodiment, extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards includes: performing deformation mapping on the standard contour of the target equipment based on the polarization image sequence to obtain abnormal candidate features of the target equipment; performing dynamic verification on the abnormal candidate features to obtain abnormal contour features of the target equipment; performing thermal flow analysis on the abnormal region of the target equipment based on the infrared thermal image sequence and the main load-bearing direction of the target equipment to obtain preliminary thermal anomaly features of the target equipment; and applying physical constraints to the preliminary thermal anomaly features to obtain the thermal anomaly features of the target equipment.

[0037] Specifically, the standard profile of the target equipment is the profile data of the equipment under normal condition, determined according to the preset power equipment acceptance standards. It includes the reference information such as the size, shape, and relative position of each component of the equipment. It serves as a reference for judging whether the equipment has any profile abnormalities, such as the standard length, width, and height dimensions of the switch cabinet and the standard width of the gap between the cabinet door and the cabinet body.

[0038] Anomaly candidate features are preliminary screening of regions that may have contour anomalies after deforming the standard contour of the target device based on polarization image sequences. These features may be preliminary manifestations of obvious defects such as deformation, scratches, and damage on the device surface, but have not yet been verified to confirm whether they are real anomalies.

[0039] Dynamic verification is a set of dynamic indicators and algorithm parameters used to verify the authenticity of abnormal candidate features. These parameters include physical properties such as the elastic modulus and yield strength of the equipment material, as well as dynamic analysis parameters such as vibration frequency and stress distribution. By simulating the performance of abnormal candidate features under actual stress or operating conditions, it is determined whether they conform to the dynamic characteristics of real defects.

[0040] Abnormal contour features are the actual contour anomalies of the target equipment confirmed after dynamic verification of abnormal candidate features. They can accurately reflect the obvious defects in the appearance of the equipment, such as dents in the cabinet, bending of components, and misalignment of connection parts, and are an important basis for subsequent defect analysis.

[0041] The main load-bearing direction of the target equipment is determined based on the structural design and working stress conditions of the power equipment. For example, transformers mainly bear the vertical gravity and the horizontal vibration and impact forces, while switch cabinets mainly bear the horizontal force when the cabinet door is opened and closed, as well as the gravity of the internal components. This parameter is used to accurately locate the critical areas where the equipment may experience thermal anomalies under stress conditions.

[0042] The abnormal areas of the target equipment are those in the infrared thermographic sequence where the temperature distribution deviates from the normal thermal distribution range of the equipment. These areas may be thermal anomalies caused by internal equipment faults and are the key targets for thermal flow analysis.

[0043] The preliminary characteristics of thermal anomalies are obtained by analyzing the thermal flow of the abnormal area based on infrared thermographic sequences and the main load-bearing direction of the target equipment. These characteristics include the temperature value, temperature gradient, and heat flow direction of the abnormal area. However, these characteristics have not been verified by physical constraints and may contain certain errors or interference factors.

[0044] Physical constraints are constraints set based on the physical characteristics and working principle of power equipment to verify the authenticity of preliminary thermal anomaly characteristics. These constraints include the maximum allowable temperature when the equipment is working normally, the normal temperature difference between different components, and the reasonable range of heat flow distribution. For example, the normal operating temperature of transformer windings should not exceed 105℃, and the temperature difference between different components in the same switch cabinet should not exceed 10℃.

[0045] Thermal anomaly characteristics are the actual thermal anomaly characteristics of the target equipment confirmed after physical constraint verification of the preliminary thermal anomaly characteristics. They can accurately reflect the hidden defects that may exist inside the equipment, such as local high temperature caused by poor line contact, uneven heat distribution caused by insulation aging, etc., and provide thermal basis for subsequent composite defect characteristic analysis.

[0046] Furthermore, the registered polarization image sequence of the target power equipment is retrieved first, and the standard contour data of the equipment determined according to the preset acceptance criteria is obtained simultaneously. Then, the image deformation mapping algorithm is used to compare and analyze the actual contour of the equipment in the polarization image with the standard contour point by point, and calculate parameters such as position deviation and shape difference of each point. Finally, according to the preset deviation threshold, the area with deviation exceeding the threshold is filtered out and its contour features are marked as abnormal candidate features.

[0047] Collect physical property parameters such as the elastic modulus and yield strength of the equipment material, as well as dynamic parameters such as vibration frequency and stress distribution during operation, and construct a dynamic verification model; input the candidate abnormal features into the model and simulate their dynamic response under normal equipment stress operation; compare the simulation results with the real defect dynamic feature library, and confirm the abnormal contour features if they match, and remove them if they do not match, and finally obtain the abnormal contour features.

[0048] Obtain registered infrared thermal image sequences and determine the main load-bearing direction by combining the equipment structure and stress; use analysis software to identify temperature anomaly areas in the thermal images and record the information; construct a heat flow analysis model based on the equipment's thermophysical characteristic parameters and environmental thermal parameters, input the temperature data of the anomaly area and the main load-bearing direction, calculate the heat flow parameters and analyze them; extract information such as temperature values ​​and temperature gradients, and integrate them into preliminary thermal anomaly characteristics.

[0049] Based on the physical characteristics and working principle of the equipment, physical constraints such as normal operating temperature and component temperature difference are set; the preliminary characteristics of thermal anomalies are compared and verified one by one with the constraints; if they are met, the thermal anomaly characteristics are confirmed; if they are not met, interference is eliminated, corrections and optimizations are made until the thermal anomaly characteristics that meet the conditions are obtained.

[0050] S3: Perform cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain the composite defect features of the target device; In this embodiment, performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain the composite defect features of the target device includes: performing cross-modal alignment on the target device based on the abnormal contour features and the thermal anomaly features to obtain the spatial defect features of the target device; performing physical coupling analysis on the target device based on the spatial defect features to obtain the failure defect features of the target device; and performing dynamic verification on the target device based on the failure defect features and the real-time operating parameters of the target device to obtain the composite defect features of the target device.

[0051] Specifically, spatial defect features are features obtained by cross-modal alignment of abnormal contour features and thermal anomaly features. These features can be correlated in spatial location to clarify the specific distribution and correlation of equipment defects in the spatial dimension.

[0052] Failure defect characteristics are obtained through physical coupling analysis of spatial defect characteristics. They can reflect the characteristics of the impact of defects on the physical function of equipment, and reflect the degree and type of equipment failure that defects may cause, such as the risk of equipment failure due to uneven stress caused by component bending, and the potential for insulation damage caused by local high temperature.

[0053] Physical coupling analysis includes the physical property parameters of each component of the equipment and the physical field parameters when the equipment is working normally. These parameters are obtained by consulting the equipment design manual, industry standards and previous experimental test data.

[0054] Real-time operating parameters are data collected in real time by sensors during the acceptance process of power equipment, simulating the normal working state of the equipment. These data include operating voltage, operating current, real-time temperature of key parts of the equipment, and operating vibration frequency. The collection frequency is determined according to the type of equipment.

[0055] Composite defect features are characteristics that reflect equipment defects obtained through dynamic verification. They integrate failure defect features with defect-related information reflected in real-time operating parameters, and can comprehensively and accurately present the defect status of the equipment, providing a basis for subsequent confidence assessment.

[0056] Furthermore, the abnormal contour features and thermal anomaly features of the target equipment are first acquired. A geometric reference matching algorithm is used to determine the modal registration reference points by taking structural reference points such as the equipment fixing holes and component connection boundaries as references. Then, based on the reference points, feature deformation caused by differences in acquisition angle and sensor position is compensated to obtain accurate registration features. Finally, based on the accurate registration feature mapping, the two features are spatially superimposed and verified to obtain spatial defect features.

[0057] At the acceptance site where the equipment is statically placed and the ambient temperature is between -5℃ and 40℃, the spatial defect characteristics and physical coupling analysis parameters such as the thermal conductivity of the equipment material and the normal operating temperature field distribution are retrieved to construct an analysis model. The spatial defect characteristics are substituted into the model to simulate the process of the defects acting in the physical field. The interaction between the defects and the physical properties of the equipment and the physical field is analyzed to screen out the characteristics that affect the physical function of the equipment and may lead to failure, thus obtaining the failure defect characteristics.

[0058] In the test area equipped with temporary power supply, parameter acquisition and safety protection facilities, the failure and defect characteristics and real-time operating parameters such as voltage, current and temperature of key parts collected during the operation of the simulated equipment are imported into the dynamic verification model; the low load and rated load conditions are simulated to monitor the changes in failure and defect characteristics and analyze the correlation between parameters and characteristics; based on the verification data and the defect change pattern, the failure and defect characteristics are corrected and improved and integrated to form composite defect characteristics.

[0059] In this embodiment, the step of performing cross-modal alignment of the target device based on the abnormal contour features and the thermal anomaly features to obtain the spatially associated defect features of the target device includes: performing geometric reference matching on the target device based on the abnormal contour features and the thermal anomaly features to obtain the modal registration reference points of the target device; performing deformation compensation on the target device based on the modal registration reference points to obtain the precise registration features of the target device; and performing spatial overlay verification on the abnormal contour features and the thermal anomaly features based on the precise registration feature mapping to obtain the spatially associated defect features of the target device.

[0060] Specifically, modal registration reference points are spatial reference points used to align abnormal contour features and thermal abnormal features. These points must be selected from structurally stable, easily identifiable locations on the power equipment that are clearly visible in both modal images, such as the center of equipment mounting holes, the intersection of cabinet corners, or the edge of component connecting flanges. The coordinate parameters of these reference points must conform to the standard dimensions in the power equipment design drawings to ensure spatial accuracy of cross-modal alignment.

[0061] Precise registration features are a set of features that are precisely aligned in space after deformation compensation of abnormal contour features and thermal anomaly features. They include defect region information related to contour-thermal correlation, such as the coordinates of the local temperature anomaly region corresponding to a certain concave defect being completely matched, and the shape and range parameters of the defect remaining consistent in both modes.

[0062] Spatial correlation defect features are comprehensive defect features obtained by merging abnormal contour features and thermal anomaly features after spatial overlay verification. They not only include independent parameters of the two types of features, but also correlation parameters, which can directly reflect the explicit-implicit correlation of defects and provide core data for subsequent composite defect analysis.

[0063] Furthermore, retrieve the abnormal contour features, thermal anomaly features, and standard parameters from the design drawings of the target equipment to unify the coordinate system; use the edge / corner detection algorithm to initially screen candidate reference points such as mounting holes and cabinet corners from the two types of features, and eliminate blurred points in thermal imaging; compare the deviation between the candidate points and the standard coordinates, and determine 3 to 5 modal registration reference points after spacing verification.

[0064] Calculate the spatial deviation of non-reference corresponding defect points. If it is greater than 1mm, analyze the error source. When there is a viewing angle deviation, calculate the compensation matrix based on the sensor installation angle and distance to correct the thermal anomaly coordinates. When there is environmental deformation, combine temperature changes with material thermal expansion and elastic deformation coefficient to correct the contour coordinates. Verify the alignment deviation of the reference points and the overlap of the defect areas after compensation, and integrate to generate accurate registration features.

[0065] A spatial mapping model is established based on the precise registration feature contour and thermal correlation, and an outlier is excluded by setting a precision threshold of 0.1 pixels. The two types of features are overlaid in the system interface to generate an image while maintaining structural integrity. The overlay parameters are checked, and if they do not meet the standards, they are adjusted backtracked. After meeting the standards, the contour-thermal independence and correlation parameters are extracted to generate spatial correlation defect features.

[0066] S4: Assess the confidence level of the composite defect features to obtain a high-confidence assessment report for the target device. In this embodiment, assessing the confidence level of the composite defect features to obtain a high-confidence assessment report for the target device includes: performing multi-dimensional parameter coupling analysis on the composite defect features based on a preset dynamic benchmark library to obtain a physical consistency score for the composite defect features; verifying the defect evolution trend of the target device based on the physical consistency score to obtain a stability level for the target device; and determining the graded confidence level of the target device based on the stability level to obtain a high-confidence assessment report for the target device.

[0067] Specifically, the pre-built dynamic benchmark library is a pre-constructed and continuously updated benchmark database used for defect feature assessment during the acceptance testing of power equipment. It contains multimodal data, physical parameters, and defect evolution data of different types of power equipment under normal conditions and various typical defect conditions. Its data sources include power equipment industry standards and specifications, equipment parameter manuals provided by equipment manufacturers, and data on qualified and unqualified equipment accumulated during historical acceptance processes. It will be dynamically supplemented and updated according to new equipment types, new defect cases, and upgrades in acceptance technology to ensure that an accurate and comprehensive reference benchmark is provided for the analysis of complex defect features.

[0068] The physical consistency score is a quantitative indicator based on the results of multidimensional parameter coupling analysis. It quantifies the degree of consistency between the physical attributes of composite defect features and the normal and typical defect features of similar equipment in a pre-set dynamic benchmark library. The score typically ranges from 0 to 100. A higher score indicates a greater consistency between the physical attributes of the composite defect features and the reasonable defect features in the benchmark library, and a higher reliability of the defect features. A lower score indicates that the composite defect features may contain errors or interference factors, requiring further verification.

[0069] Stability level is an indicator that classifies the likelihood of defects in a target device remaining in their current state or worsening during operation, based on the verification results of defect evolution trends. It is usually divided into four levels: stable, relatively stable, unstable, and extremely unstable, providing a basis for subsequent confidence level determination.

[0070] The graded confidence level determination is a process of determining the credibility of the composite defect characteristics and defect risk assessment results of the target equipment based on its stability level and pre-set confidence level determination criteria. During the determination process, fine-tuning is also made based on the type and importance of the target equipment to ensure that the confidence level determination results better align with actual acceptance requirements.

[0071] A high-confidence assessment report is a comprehensive report that includes basic information about the target equipment, a detailed description of the composite defect characteristics, the process and results of multi-dimensional parameter coupling analysis, physical consistency score, verification process and stability level of defect evolution trend, graded confidence level determination results, and defect handling recommendations. The report must ensure data accuracy and logical clarity, providing a highly credible assessment basis for subsequent compliance matching based on equipment acceptance standards. For example, the report might specify that the composite defect characteristics of a switchgear are cabinet door deformation accompanied by localized high temperatures, a physical consistency score of 85, a stability level of relatively stable, a confidence level of 82%, and recommend thermal imaging monitoring of the area every 3 months after installation.

[0072] Furthermore, firstly, a preset dynamic benchmark library is retrieved and a subset matching the target equipment type is selected. Multi-dimensional parameters such as the three-dimensional spatial coordinates, temperature distribution, and material detection of the composite defect characteristics of the target equipment are extracted. These parameters are compared with the corresponding parameters of the benchmark subset to calculate the correlation coefficient. The weighted sum is then used to obtain the comprehensive value of the coupling degree, which is then converted into a physical consistency score according to a preset mapping relationship.

[0073] Next, combining the equipment type and expected usage environment, historical data on defect evolution of similar equipment are retrieved from the benchmark library, a prediction model is constructed and relevant parameters are input, defect changes in different time periods are simulated, the probability of defect evolution is statistically analyzed, and the stability level is determined according to preset rules.

[0074] Finally, the corresponding basic confidence interval is retrieved, and the adjustment coefficient is determined in combination with the equipment importance level. After fine-tuning the interval, the final confidence level is determined in combination with the physical consistency score. Various equipment information is collected, integrated according to the template, defect handling suggestions are added, and a high-confidence assessment report is formed.

[0075] In this embodiment, the step of performing multidimensional parameter coupling analysis on the composite defect features based on a preset dynamic benchmark library to obtain the physical consistency score of the composite defect features includes: performing cross-modal alignment matching on the composite defect features based on the dynamic benchmark library to obtain the modal correlation matrix of the composite defect features; performing parameter coupling calculation on the composite defect features based on the modal correlation matrix to obtain the physical coupling coefficient of the composite defect features; and performing normalized score integration on the composite defect features based on the physical coupling coefficient to obtain the physical consistency score of the composite defect features.

[0076] Specifically, the modal correlation matrix is ​​matrix data generated after cross-modal alignment and matching of composite defect features based on a dynamic benchmark library. The rows and columns of the matrix correspond to the feature dimensions of different modalities in the composite defect features, and the element values ​​in the matrix represent the correlation strength between two corresponding feature dimensions of different modalities. The correlation strength typically ranges from 0 to 1. The closer the value is to 1, the stronger the correlation between the two corresponding feature dimensions of the modality, and the better it reflects the consistency of the defect under different modalities; the closer the value is to 0, the weaker the correlation, and the more likely there may be feature errors or interference factors.

[0077] The physical coupling coefficient is a quantitative index obtained by calculating the parameter coupling of composite defect features based on the modal correlation matrix. It is used to characterize the degree of interaction and synergistic consistency between different modal parameters in composite defect features at the physical level. Its calculation takes into account the physical characteristics of the power equipment, and the value range is generally 0-1. The higher the coefficient value, the more significant the physical coupling relationship between different modal parameters and the stronger the physical rationality of the defect feature; the lower the coefficient value, the weaker the physical coupling relationship, and the more necessary it is to further verify the authenticity of the defect feature.

[0078] Furthermore, the preset dynamic benchmark library calling program is first launched to retrieve benchmark data that matches the target equipment type, and the acquired composite defect features of the target equipment are imported at the same time. Then, a cross-modal alignment matching algorithm is adopted, using the standard structural features of the equipment in the dynamic benchmark library as a reference, to align and match the contour modal data and thermal modal data in the composite defect features in terms of spatial position and feature dimension. By calculating the similarity and correlation between different modal features, a modal correlation matrix is ​​generated.

[0079] Then, based on the modal correlation matrix and combined with the physical characteristic parameters of the target device, the physical interaction relationship of each modal parameter in the composite defect feature is quantitatively calculated using the parameter coupling calculation model to obtain the physical coupling coefficient.

[0080] Finally, referring to the standard range of physical coupling coefficients of similar equipment in the dynamic benchmark library, a normalization algorithm is used to convert the physical coupling coefficients into intuitive scores of 0-100 points, completing the normalization score integration of the composite defect characteristics of the target equipment, and finally obtaining the physical consistency score, which provides the core quantitative basis for the subsequent confidence assessment of the target equipment.

[0081] S5: Based on the equipment acceptance criteria, perform compliance matching on the high-confidence assessment report to obtain the structured acceptance report of the target equipment.

[0082] In this embodiment, the step of performing compliance matching on the high-confidence assessment report based on the equipment acceptance criteria to obtain a structured acceptance report for the target equipment includes: performing a compliance traversal on the high-confidence assessment report based on the equipment acceptance criteria to obtain a violation determination result of the high-confidence assessment report; and archiving the violation determination result as a structured acceptance report for the target equipment.

[0083] Specifically, the equipment acceptance standard is a compliance judgment benchmark established in the power equipment delivery acceptance process based on national power industry standards, equipment manufacturer technical requirements, and project customization needs. It covers specific indicators such as the range of errors in the equipment's appearance outline, the normal threshold of heat distribution, and the installation accuracy of components. For example, the depth of the dent on the transformer cabinet should not exceed 2mm, and the normal operating temperature difference of the internal components of the switch cabinet should be controlled within 10℃. This standard provides a clear judgment basis for the compliance review of the high-confidence assessment report.

[0084] The result of the violation determination is determined by comparing the defect information in the high-confidence assessment report with the equipment acceptance standards one by one through the compliance process. The result includes the specific item of the violation, the corresponding standard clause, the actual deviation value and the violation level. For example, if the thermal anomaly temperature of a certain component in the report exceeds the standard threshold by 5°C, it is determined to be a serious violation. This result is a core component of the structured acceptance report.

[0085] A structured acceptance report is a standardized report formed by archiving the results of violation judgments in a preset format. The format conforms to the power industry acceptance document specifications and can be directly used for acceptance conclusion approval, equipment quality traceability, and subsequent rectification tracking. For example, the report will clearly indicate the acceptance standard clause number and rectification completion deadline for each violation defect.

[0086] Furthermore, firstly, the preset equipment acceptance standards matching the target equipment model are retrieved, and at the same time, the generated high-confidence assessment report of the target equipment is obtained.

[0087] Next, the compliance traversal process is initiated. According to each indicator in the equipment acceptance standard, the defect information in the high-confidence assessment report is extracted one by one for comparison and analysis. It is determined whether each defect exceeds the standard allowable range. The defect items that do not meet the standard, the corresponding standard clauses, and the actual deviation values ​​are recorded. The violation level is divided according to the degree of deviation to form a violation judgment result.

[0088] Finally, following the standard structure of power industry acceptance documents, the violation judgment results are integrated and archived with the target equipment's basic information, acceptance testing time, and testing personnel information to generate a structured acceptance report containing modules for basic equipment information, a list of violations and defects, judgment basis, and rectification suggestions. This ensures that the report is complete and formatted correctly, and can be directly used for subsequent approval and quality traceability work in the acceptance process.

[0089] Figure 2 shows a functional block diagram of a multimodal AI image recognition system for power equipment arrival acceptance provided by an embodiment of the present invention.

[0090] The multimodal AI image recognition system 100 for power equipment arrival acceptance described in this invention can be installed in an electronic device. Depending on the functions implemented, the multimodal AI image recognition system 100 for power equipment arrival acceptance may include an acquisition module 101, an extraction module 102, a correlation analysis module 103, an evaluation module 104, and an acceptance module 105. The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0091] In this embodiment, the functions of each module / unit are as follows: Acquisition module 101: used to acquire multimodal data of the target device and perform standardization processing on the multimodal data to obtain standardized data of the target device; Extraction module 102: used to extract abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards; Correlation analysis module 103: used to perform cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain composite defect features of the target device; Evaluation module 104: used to evaluate the confidence level of the composite defect features to obtain a high-confidence evaluation report of the target device; Acceptance module 105: used to perform compliance matching on the high-confidence evaluation report based on the equipment acceptance standards to obtain a structured acceptance report of the target device.

[0092] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0093] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0095] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0096] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multimodal AI image recognition method applied to the acceptance of power equipment delivery, characterized in that, The method includes: S1: acquiring multimodal data of the target device and standardizing the multimodal data to obtain standardized data of the target device; S2: extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance criteria; S3: performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain composite defect features of the target device; S4: assessing the confidence level of the composite defect features to obtain a high-confidence assessment report of the target device; S5: performing compliance matching on the high-confidence assessment report based on the equipment acceptance criteria to obtain a structured acceptance report of the target device.

2. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 1, characterized in that, The acquisition of multimodal data of the target device includes: acquiring a polarization image sequence of the target device based on a preset optical sensor; acquiring an infrared thermal image sequence of the target device based on a preset thermodynamic sensor; and registering the polarization image sequence and the infrared thermal image sequence to obtain the multimodal data of the target device.

3. The multimodal AI image recognition method for power equipment arrival acceptance as described in claim 1, characterized in that, The multimodal data is then standardized to obtain standardized data, including: performing grayscale normalization on the multimodal data to obtain grayscale standard data of the target device; and performing background suppression on the grayscale standard data to obtain standardized data of the target device.

4. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 2, characterized in that, Extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance criteria includes: performing deformation mapping on the standard contour of the target equipment based on the polarization image sequence to obtain candidate abnormal features of the target equipment; performing dynamic verification on the candidate abnormal features to obtain abnormal contour features of the target equipment; performing heat flow analysis on the abnormal region of the target equipment based on the infrared thermal image sequence and the main load-bearing direction of the target equipment to obtain preliminary thermal anomaly features of the target equipment; and applying physical constraints to the preliminary thermal anomaly features to obtain the thermal anomaly features of the target equipment.

5. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 1, characterized in that, Cross-modal correlation analysis is performed on the abnormal contour features and the thermal anomaly features to obtain the composite defect features of the target device, including: cross-modal alignment of the target device based on the abnormal contour features and the thermal anomaly features to obtain the spatial defect features of the target device; physical coupling analysis of the target device based on the spatial defect features to obtain the failure defect features of the target device; and dynamic verification of the target device based on the failure defect features and the real-time operating parameters of the target device to obtain the composite defect features of the target device.

6. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 5, characterized in that, Based on the abnormal contour features and the thermal anomaly features, cross-modal alignment of the target device is performed to obtain the spatially associated defect features of the target device, including: performing geometric reference matching of the target device based on the abnormal contour features and the thermal anomaly features to obtain the modal registration reference points of the target device; performing deformation compensation of the target device based on the modal registration reference points to obtain the precise registration features of the target device; and performing spatial overlay verification of the abnormal contour features and the thermal anomaly features based on the precise registration feature mapping to obtain the spatially associated defect features of the target device.

7. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 1, characterized in that, The confidence assessment of the composite defect features to obtain a high-confidence assessment report for the target device includes: performing multi-dimensional parameter coupling analysis on the composite defect features based on a preset dynamic benchmark library to obtain a physical consistency score for the composite defect features; verifying the defect evolution trend of the target device based on the physical consistency score to obtain a stability level for the target device; and determining the graded confidence level of the target device based on the stability level to obtain a high-confidence assessment report for the target device.

8. The multimodal AI image recognition method for power equipment delivery acceptance as described in claim 7, characterized in that, Based on a preset dynamic benchmark library, a multidimensional parameter coupling degree analysis is performed on the composite defect features to obtain a physical consistency score for the composite defect features. This includes: performing cross-modal alignment matching on the composite defect features based on the dynamic benchmark library to obtain a modal correlation matrix of the composite defect features; performing parameter coupling calculation on the composite defect features based on the modal correlation matrix to obtain a physical coupling coefficient of the composite defect features; and performing normalized score integration on the composite defect features based on the physical coupling coefficient to obtain a physical consistency score for the composite defect features.

9. The multimodal AI image recognition method for power equipment arrival acceptance as described in claim 1, characterized in that, Based on the equipment acceptance criteria, the high-confidence assessment report is matched for compliance to obtain a structured acceptance report for the target equipment. This includes: performing a compliance traversal on the high-confidence assessment report based on the equipment acceptance criteria to obtain a violation determination result for the high-confidence assessment report; and archiving the violation determination result as a structured acceptance report for the target equipment.

10. A multimodal AI image recognition system applied to the acceptance of power equipment delivery, characterized in that, The system includes: an acquisition module (101) for acquiring multimodal data of a target device and standardizing the multimodal data to obtain standardized data of the target device; an extraction module (102) for extracting abnormal contour features and thermal anomaly features from the standardized data based on preset equipment acceptance standards; a correlation analysis module (103) for performing cross-modal correlation analysis on the abnormal contour features and the thermal anomaly features to obtain composite defect features of the target device; an evaluation module (104) for evaluating the confidence level of the composite defect features to obtain a high-confidence evaluation report of the target device; and an acceptance module (105) for performing compliance matching on the high-confidence evaluation report based on the equipment acceptance standards to obtain a structured acceptance report of the target device.

Images