Intelligent power meter sorting and fault diagnosis method based on multi-modal data fusion

Abstract

The application relates to the technical field of image recognition, and particularly discloses a multi-modal data fusion intelligent electric meter sorting and fault diagnosis method, which comprises the following steps: acquiring basic information of an intelligent electric meter to be processed, collecting real-time environment data and historical environment data of the intelligent electric meter, assigning a unique identifier to each electric meter, and associating and binding the information and the data; collecting image data, electric performance data and communication history data of the intelligent electric meter; and calculating an aging coefficient based on the historical environment data. The application successfully solves the limitations in the traditional intelligent electric meter fault diagnosis method by introducing a multi-modal data fusion technology; by comprehensively utilizing image data, electric performance data, communication history data and environment information, the application can comprehensively analyze the running state and potential faults of the electric meter from multiple dimensions, and effectively improves the accuracy and reliability of fault diagnosis.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, in particular to a multi-modal data fusion intelligent electric meter sorting and fault diagnosis method. BACKGROUND

[0002] Intelligent electric meters are widely used in various power systems, responsible for accurate electric energy metering, data transmission, and fault diagnosis, etc. However, as the use time extends, intelligent electric meters may have problems such as aging and performance degradation, which may affect their accuracy and reliability. Currently, the fault diagnosis of intelligent electric meters mainly relies on a single data modality, such as electrical performance data or communication data, and the limitations of this traditional method greatly affect the accuracy of fault diagnosis.

[0003] In addition, existing intelligent electric meter fault diagnosis methods rely on manual detection or single sensor-based data processing, and cannot fully utilize the complementary advantages of multiple data sources. With the development of the Internet of Things and sensor technology, multi-modal data fusion (such as image, communication history, and electrical performance data) has become a trend, which can improve the accuracy and efficiency of fault diagnosis. However, how to effectively extract useful features from multiple data sources, perform reasonable data fusion, and solve data conflict problems is still a major problem in current technology. SUMMARY

[0004] The present application aims to at least partially solve one of the technical problems in the related art. To this end, the purpose of the present application is to propose a multi-modal data fusion intelligent electric meter sorting and fault diagnosis method to improve the accuracy of intelligent electric meter fault identification and sorting.

[0005] To achieve the above-mentioned purpose, the first aspect of the present application proposes a multi-modal data fusion intelligent electric meter sorting and fault diagnosis method, comprising the following steps:

[0006] S1, obtaining the basic information of the intelligent electric meter to be processed, collecting the real-time environment data and historical environment data thereof, assigning a unique identifier to each electric meter, and associating and binding the information and data;

[0007] S2, collecting image data, electrical performance data, and communication history data of the intelligent electric meter; calculating an aging coefficient based on the historical environment data , and dynamically adjusting the detection parameters according to the aging coefficient to process the electrical performance data; extracting features from the three types of data respectively to generate image aging feature vectors , electrical performance feature vectors , and communication history feature vectors ;

[0008] S3, combine the real-time environment data with the aging coefficient , the , , Dynamic weighted fusion is performed, and multi-modal data conflicts generated in the fusion process are resolved to generate a final comprehensive feature vector ;

[0009] S4, similarity calculation is performed between the final comprehensive feature vector and a standard feature vector preset in a knowledge base to determine a fault type, and the aging coefficient is combined to grade the fault severity;

[0010] S5, according to the fault grade, a corresponding automatic sorting operation is performed, and based on the sorting result feedback, key parameters in the diagnosis process are closed-loop optimized.

[0011] To achieve the above purpose, a second aspect embodiment of the present application provides an intelligent electric meter sorting and fault diagnosis system based on multi-modal data fusion, which comprises:

[0012] A data acquisition module is configured to acquire basic information of an intelligent electric meter to be processed, real-time environment data, historical environment data, electrical performance data, image data, and communication history data.

[0013] A data processing module is configured to calculate an aging coefficient according to the historical environment data, and dynamically adjust detection parameters of the electrical performance data according to the coefficient, and then process the electrical performance data.

[0014] A feature extraction module is configured to extract an image aging feature vector, an electrical performance feature vector, and a communication history feature vector from the image data, the electrical performance data, and the communication history data, respectively.

[0015] A data fusion module is configured to combine the real-time environment data with the aging coefficient, dynamically weightedly fuse various types of data, and resolve multi-modal data conflicts in the fusion process to generate a final comprehensive feature vector.

[0016] A fault diagnosis module is configured to perform similarity calculation between the final comprehensive feature vector and a standard feature vector preset in a knowledge base, and then determine a fault type of the electric meter, and grade the fault severity according to the fault type and the aging coefficient.

[0017] An automatic sorting module is configured to perform an automatic sorting operation according to the fault severity grade, and through feedback, to perform closed-loop optimization on key parameters in the diagnosis process.

[0018] To achieve the above object, the third aspect of the present application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory, wherein the computer program is executed by the processor to realize the multi-modal data fusion based smart meter sorting and fault diagnosis method.

[0019] Compared with the prior art, the present application has the following advantages:

[0020] The multi-modal data fusion based smart meter sorting and fault diagnosis method of the present application successfully solves the limitations in the traditional smart meter fault diagnosis method by introducing multi-modal data fusion technology; by comprehensively utilizing image data, electrical performance data, communication history data and environmental information, the present application can comprehensively analyze the running state and potential faults of the meter from multiple dimensions, effectively improving the accuracy and reliability of fault diagnosis; especially in the process of dynamically adjusting detection parameters and fusing multiple data features, the present application not only improves the recognition accuracy of fault types, but also can reasonably classify the fault severity, thereby realizing the automatic sorting of smart meters and the closed-loop optimization of key parameters. BRIEF DESCRIPTION OF DRAWINGS

[0021] The disclosure of the present application will be described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0022] Figure 1 is a flowchart of the multi-modal data fusion based smart meter sorting and fault diagnosis method provided by the present application;

[0023] Figure 2 is a comparison diagram of the effect of smart meter image bright spot detection and elimination in the multi-modal data fusion based smart meter sorting and fault diagnosis method provided by the present application;

[0024] Figure 3 is a surface diagram of the relationship between the aging coefficient K and the environmental factors in the multi-modal data fusion based smart meter sorting and fault diagnosis method provided by the present application;

[0025] Figure 4 is a response diagram of the shallow and deep feature extraction capability of the CNN network in the multi-modal data fusion based smart meter sorting and fault diagnosis method provided by the present application in the processing of smart meter images;

[0026] Figure 5 is a curve change diagram of the multi-modal feature fusion weight with the aging coefficient in the multi-modal data fusion based smart meter sorting and fault diagnosis method provided by the present application;

[0027] Figure 6is a schematic diagram of the relationship between the similarity score distribution and the determination threshold in the smart meter sorting and fault diagnosis method of the multi-modal data fusion provided by the application.

[0028] Figure 7 is a schematic diagram of the comparison of fault diagnosis accuracy before and after closed-loop optimization in the smart meter sorting and fault diagnosis method of the multi-modal data fusion provided by the application.

[0029] Figure 8 is a schematic diagram of the framework structure of the smart meter sorting and fault diagnosis system of the multi-modal data fusion provided by the application.

[0030] Figure 9 is a schematic diagram of the structure of the electronic device provided by the application. DETAILED DESCRIPTION

[0031] The embodiments of the application will be described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the application, and cannot be understood as a limitation of the application.

[0032] The multi-modal data fusion smart meter sorting and fault diagnosis method, system and electronic device of the embodiments of the application will be described below with reference to the accompanying drawings.

[0033] Embodiment one:

[0034] As shown in Figure 1 A multi-modal data fusion smart meter sorting and fault diagnosis method, comprising the following steps:

[0035] S1, obtaining the basic information of the smart meter to be processed, collecting the real-time environment data and historical environment data thereof, assigning a unique identifier to each meter, and associating and binding the information and data. The core goal is to build an initial data set of meter unique identity profile + full-dimensional environment data, which provides accurate basis for subsequent aging coefficient calculation, detection parameter adjustment and fault diagnosis tracing. Specifically, it includes the following contents:

[0036] 1. The basic information acquisition of the smart meter to be processed includes the range of information collection and the way of information collection.

[0037] (1) Information collection range; for the old smart meter returned, two types of core basic information need to be collected, covering identity identification and technical parameters, as follows:

[0038] Identity information: unique device number of the meter, such as asset number allocated by the power department, batch number, manufacturer, installation and operation time, which is used to assist in judging the service life;

[0039] Technical parameter information: meter model, such as DT862 three-phase four-wire meter, DDZY102 single-phase intelligent meter, rated voltage / current, such as 220V / 10(40)A, measurement accuracy level, such as 1 level, 0.5S level, communication interface type, such as RS485, LoRa.

[0040] (2) Information collection method: adopt double-checking mode of automatic identification + manual review to ensure information accuracy, specific operation:

[0041] Automatic identification: scan the two-dimensional code / bar code on the surface of the meter (printed at the factory or attached by the power department later) through the industrial code scanning gun, directly read the structured information such as device number, model, batch number, etc., and the data is automatically transmitted to the meter basic information library of the local server;

[0042] Manual review: for old meters that cannot be identified by scanning code, such as two-dimensional code wear, take photos of the meter nameplate (clear text area) through high-definition industrial camera, use OCR technology to extract information, and check key parameters such as rated current and accuracy level by detection personnel, to avoid OCR misidentification, such as misjudging 10(40)A as 10(4)A.

[0043] 2, Real-time environmental data collection includes two aspects of collection parameters and device requirements and data validity verification;

[0044] (1) Collection parameters and device requirements: focus on key environmental factors that affect meter test results and aging state, collect the following parameters in the table, and the device accuracy should meet the industrial level detection standard:

[0045]

[0046] (2) Data validity verification: after collecting the above parameters, automatically perform two verifications:

[0047] Range verification: if the collected value exceeds the device range (such as temperature > 85℃ or <-40℃), it is determined as invalid data, triggering the sensor to re-collect. At most, retry 3 times, still invalid, then alarm to replace the sensor;

[0048] Fluctuation verification: if the temperature difference of 3 consecutive collections is >2℃ or the humidity difference is >5%RH, it is determined as environmental fluctuation anomaly, such as air outlet near the detection station, the system automatically prompts to adjust the detection environment, such as closing the air conditioner or moving the sensor position, and re-collecting after the fluctuation is stable.

[0049] 3. Historical environmental data collection includes information collection range and collection method and traceability logic;

[0050] (1) Information collection range; retrieve the storage environment data of the meter after disassembly from the meter full life cycle management system of the power department, which includes three categories:

[0051] Total storage time: the total time from the meter disassembly from the field to entering the detection station, in units of months;

[0052] Monthly average humidity: the average relative humidity during storage each month, in %RH. For example, if a meter is stored for 3 months, the humidity each month is 55%, 60%, and 58%, the monthly average humidity is 57.7%;

[0053] Extreme environment times: the number of days during storage that meet the conditions of environmental temperature > 40℃ or relative humidity > 80% (single extreme environment lasting ≥ 24 hours is counted as 1, less than 24 hours is not counted), which is used for subsequent quantification of aging degree.

[0054] (2) Collection method and traceability logic includes: data association: the unique device number of the meter obtained through the above steps is used as an index to match the corresponding storage record in the meter full life cycle management system, which records the storage warehouse, warehouse temperature and humidity monitoring data of each meter after disassembly.

[0055] Data completion: if the storage data is missing in the system for a certain period of time (such as warehouse sensor failure), the average environmental data of other meters in the same warehouse during the same period is used to complete the data, ensuring the integrity of the historical data.

[0056] 4. Unique identifier ID allocation and multi-dimensional data association binding;

[0057] (1) The generation rule of the unique identifier ID can be: using the UUID format of timestamp + meter model abbreviation + random sequence to generate a unique ID, example: 20240515-DT862-8F3A7C, where: 20240515 is the detection date; DT862 is the meter model abbreviation; 8F3A7C is a 6-bit random hexadecimal sequence, which ensures that the IDs of the same batch and same model meters are not repeated.

[0058] (2) Data association binding and storage. Binding content: associate the above obtained basic information, real-time environmental data, and historical environmental data with the unique ID in the form of key-value, forming structured data. The associated data is stored in the local server using AES-256 encryption algorithm, not uploaded to the cloud, to avoid data leakage, and a paper version of the ID-meter corresponding list is generated for manual checking and tracing.

[0059] S2, collect image data, electrical performance data and communication history data of the smart meter. Calculate an aging coefficient based on historical environmental data , and dynamically adjust the detection parameters according to the aging coefficient , process the electrical performance data, and then extract features from the image data, electrical performance data and communication history data to generate image aging feature vectors , electrical performance feature vectors and communication history feature vectors . Specifically, the following includes: 1. Image data acquisition and bright spot elimination.

[0060] (1) Before collecting image data, the camera array is used to perform initial collection and identify bright spots in the image;

[0061] If the pixel ratio of the bright spot area exceeds the preset threshold, the light compensation direction is determined according to the center position of the bright spot, the power of the adjacent light compensation lamp is adjusted, and then the image is re-collected until the pixel ratio of the bright spot is below the preset threshold, and the non-interference image set is output. The hardware for image collection includes: a 12-direction industrial camera array covering the front of the meter, the side shell, the terminal, the metering chip shell and other key areas, the camera resolution is ≥2592x1944, and the frame rate is ≥15fps, ensuring that there is no dead angle imaging for each part of the meter.

[0062] Next, the image is preprocessed, which includes grayscale and Gaussian filtering:

[0063] Grayscale: convert the color image to grayscale, the formula can be:

[0064]

[0065] wherein, , , are the red, green and blue channel values of the pixel; is the gray value, which focuses the image information on the brightness feature.

[0066] Gaussian filtering: use a 5x5 filter kernel to denoise (standard deviation = 1.2) to reduce the interference of image noise on bright spot identification.

[0067] At this time, set the highlight threshold , which is determined by training 1000 groups of meter images under different lighting conditions. The gray value of the non-bright spot area is mostly below 220, and the pixel ratio of the image , is the bright spot pixel ratio.

[0068] For example, set , the proportion of bright spots will seriously block the surface features of the meter, affecting defect recognition, and being determined as effective bright spots. In the front image of a certain single-phase intelligent meter (model DDZY102) collected initially, bright spots appear in the display area due to the direct light of the top light of the detection station. After preprocessing, it is counted that Therefore, the bright spot elimination process is triggered.

[0069] (2) Next, bright spot elimination is performed, mainly including dynamic adjustment of light compensation direction and power:

[0070] First, the center of the bright spot is located: the Canny edge detection algorithm (low threshold 100, high threshold 200) is used to extract the bright spot contour, and the center coordinates of the bright spot area are calculated ;

[0071] Then, the light compensation direction is selected: the image plane is divided into four quadrants in the pipeline direction (X axis) and the vertical pipeline direction (Y axis), according to the quadrant in which it is located, the current light compensation lamp is turned off and the light compensation lamp in the adjacent quadrant is turned on, so as to avoid direct light compensation of the bright spot area.

[0072] Then adjust the light compensation power: the light compensation lamp is a 10W-50W adjustable power LED lamp, and the power adjustment formula can be:

[0073]

[0074] Among them, is the original light compensation power; when , the power is reduced by 10% for every 1% exceeded, to avoid overexposure.

[0075] Next, repeat the verification: after adjusting the light compensation, re-collect the image, repeat the bright spot identification steps, until , and output the interference-free image set.

[0076] By way of example, the bright spot center of the above-mentioned DDZY102 meter is located in the first quadrant, the original light compensation power , , then the adjusted power is:

[0077]

[0078] After re-collection, the proportion of bright spot pixels is reduced to 4.2%, meeting the requirements, and the interference-free image is output.

[0079] As Figure 2 shows a comparison diagram of intelligent meter image bright spot detection and elimination effect, including three subgraphs: (a) original meter image, (b) bright spot image, (c) image after eliminating bright spots. Figure 2The image brightness is represented by grayscale, ranging from black 0 to white 1, and the red contour line precisely marks the detected bright spot area.

[0080] As shown in Figure 2 , before image acquisition, the camera array performs initial acquisition and identifies bright spots in the image. If the pixel ratio of the bright spot area exceeds the preset threshold (5%), the light compensation direction is determined according to the center position of the bright spot, and the power of the adjacent light compensation lamp is adjusted for reacquisition. The bright spot (within the red contour) in the upper right area of the meter in subgraph (b) is clearly displayed, with a pixel ratio of 8.2%, exceeding the 5% threshold set in the patent.

[0081] Figure (c) shows the image effect after bright spot elimination processing, which effectively eliminates the bright spot (bright spot ratio reduced to 2.1%) and maintains the clarity and detail integrity of the image. Unlike simple blur processing, this method adjusts the bright spot area to a similar brightness level to the surrounding environment by analyzing the ambient brightness around the bright spot area, thereby maintaining the clarity of the meter surface texture, display screen content, and scale markings.

[0082] 2. Aging coefficient Dynamic adjustment of detection parameters is calculated;

[0083] (1) Aging coefficient is used to quantify the degree of aging of the meter due to the storage environment. The formula can be:

[0084]

[0085] wherein, : Total storage time after the meter is returned (unit: month); : Monthly average relative humidity during storage (unit: %); : Number of extreme environments with temperature > 40°C or humidity > 80% during storage; Coefficients 0.4, 0.3, 0.3: Weight of each factor on aging (this value is determined by a large number of meter aging experiments and failure data regression analysis, with storage duration having a slightly higher impact, and humidity and extreme environment frequency having similar impacts); and the denominators 60, 60, and 10 in the above formula are the standardized baseline values of each factor, making convenient for subsequent parameter interval division.

[0086] For example, a three-phase meter (model DT862) is stored for 24 months after being returned, with a monthly average humidity of 50% and an extreme environment frequency of 4 times, then:

[0087]

[0088] (2) The content of dynamic adjustment of detection parameters includes: according to The adjustment of the measurement accuracy tolerance, the current sampling frequency and the terminal contact resistance detection threshold value are adjusted according to different intervals of the value of K.

[0089] For example, the following K value division is set:

[0090] When K < 0.3, the meter is slightly aged, the measurement accuracy tolerance is ± 2% (initial value), the current sampling frequency is 1 time per second, and the terminal contact resistance detection threshold value is 0.5Ω (initial value); When 0.3 < K < 0.8, the meter is moderately aged, the measurement accuracy tolerance is ± 1.5%, the current sampling frequency is 3 times per second, and the terminal contact resistance detection threshold value is 0.4Ω;

[0091] When K > 0.8, the meter is severely aged, the measurement accuracy tolerance is ± 1%, the current sampling frequency is 5 times per second, and the terminal contact resistance detection threshold value is 0.3Ω.

[0092] The adjustment logic is as follows: the more serious the aging (the larger the K value), the higher the risk of performance degradation of the internal elements of the meter, so the measurement accuracy tolerance is tightened (false positives are reduced), the current sampling frequency is increased (fine current fluctuations are captured), and the contact resistance threshold value is reduced (terminal oxidation faults are detected earlier). For example, the above DT862 meter

[0093] Therefore, the measurement accuracy tolerance is adjusted to ± 1.5%, the current sampling frequency is increased from 1 time per second to 3 times per second, and the terminal contact resistance detection threshold value is reduced from

[0094] . . .

[0095] As Figure 3 The surface diagram shows the relationship between the aging coefficient K and the environmental factors. Figure 3 The total storage time and the average monthly humidity are taken as the horizontal and vertical axes, the aging coefficient is taken as the elevation and color display, and the number of extreme environments is fixed at 4 times.

[0096] The color of the surface changes from cold to warm, representing a continuous change from low aging to high aging: the color of the surface is gradually changed from blue to red, the blue area (K < 0.3) represents slight aging, the green to yellow area (0.3 < K < 0.8) represents moderate aging, and the red area (K > 0.8) represents severe aging. This color coding intuitively reflects the aging risk level of the meter under different environmental conditions. Figure 3 ​​Two black isograms K=0.3 and K=0.8 in the middle of the superposition provide intuitive threshold reference. When K crosses around 0.3, 0.8, the system will trigger the adjustment of fusion weight, the similarity judgment and the downstream step logic of fault level division, etc. The red dot in the figure marks the typical working condition in the patent: the total storage time is 24 months, the monthly average humidity is 50% RH, and the extreme environment times is 4 times, at this time k≈0.53, which falls in the moderate interval. In this interval, the corresponding detection parameters are dynamically adjusted: the metering accuracy allowable deviation is about , the current sampling frequency is raised to about 3 times / s, and the wiring terminal contact resistance threshold is lowered to about .

[0097] It is particularly worth noting that when the storage time exceeds 40 months and the monthly average humidity is higher than 70%, the K value quickly exceeds 0.8, corresponding to the severe aging state defined in the patent, and such a meter needs to use more stringent detection parameters.

[0098] 3, Electric performance data acquisition and standardization processing;

[0099] (1) Electric performance parameter acquisition: Through high-precision electric parameter test module, such as power analyzer, resistance tester, collect the data of voltage, current, power, metering deviation, wiring terminal contact resistance, etc. When collecting, the voltage input uses standard source, such as output AC voltage source, and the current input is set according to the rated current of the meter.

[0100] (2) Data standardization (Min-Max standardization), in order to make different dimensional electric performance data can be fused, the formula can be:

[0101]

[0102] Among them, : original electric performance data (such as voltage, current, resistance value, etc.); : the minimum design value of the parameter (such as voltage , contact resistance ; : the maximum design value of the parameter (such as voltage , contact resistance ); The standardized data is convenient for subsequent feature vector construction.

[0103] For example, the original value of the wiring terminal contact resistance of a meter is , , , then after standardization:

[0104]

[0105] 4. Communication history data collection and feature vectorization;

[0106] (1) Communication history data collection: Through the communication interface of the electric meter, such as RS485, LoRa, read the built-in history running log of the electric meter, extract key information, including: the maximum measurement deviation in the past 3 years , , (unit: %); number of communication interruptions (unit: times); last fault record, such as inaccurate measurement, communication timeout, etc. .

[0107] (2) Feature vectorization processing: convert the extracted communication history information into a numerical feature vector , including the following steps:

[0108] ① Min-Max standardization is performed on , , (benchmark value: min=0%, max=5%, usually serious fault when measurement deviation exceeds 5%);

[0109] ② Normalize (benchmark value: min=0 times, max=20 times, communication stability is poor when exceeding 20 times);

[0110] ③ Fault label encoding is performed on , for example, no fault is coded as 0, inaccurate measurement is coded as 1, communication timeout is coded as 2, etc., and finally generate with matching dimensions as other feature vectors.

[0111] For example, the maximum measurement deviation of a certain electric meter in the past 3 years is 1.2%, 1.5%, and 1.8%, respectively, the number of communication interruptions is 12 times, and the last fault is inaccurate measurement (coded as 1). Then:

[0112] , , ;

[0113] ; fault code is 1; combined into (here it is assumed that the dimension is 5).

[0114] 5. Generation process of electric performance feature vector includes: combining standardized voltage, current, power, measurement deviation, and contact resistance data in the preset order of voltage → current → power → measurement deviation → contact resistance, generating a The specific dimensions are determined by the number of electrical performance parameters and subsequent fusion requirements.

[0115] 6. Image aging feature vector The generation process includes:

[0116] (1) In the shallow network of the convolutional neural network, the overall outline deformation degree of the electric meter and the shell scratch density are extracted to generate a shallow aging feature vector In the deep network of the convolutional neural network, the display screen yellowing coefficient and the indicator light transmittance are extracted to generate a deep aging feature vector Specifically, the following includes:

[0117] ① Extraction of shallow aging feature vector Based on the CNN shallow network;

[0118] First, network and training: the first 3 layers of VGG16 convolutional neural network (including 2 convolution blocks, each block is 2 layers of 3x3 convolution + ReLU + 2x2 pooling, with a step of 1, 2) are pre-trained on a 1000+ multi-model electric meter aging image dataset, so that the network learns the visual patterns of two types of features.

[0119] Then calculate the outline deformation degree, including:

[0120] For the pre-processed interference-free image, use the edge detection convolution kernel of the shallow network to extract the outline edge of the electric meter;

[0121] Calculate the shape matching degree with the standard non-deformed electric meter outline template (the outline mask image of the new electric meter of the same model), which can be expressed as:

[0122]

[0123] Wherein, is the outline deformation degree, ∈[0, 1], 0 represents no deformation, and 1 represents complete deformation.

[0124] Then calculate the shell scratch density: use the texture extraction convolution kernel in the shallow network to output the scratch probability heat map;

[0125] Statistical heat map scratch pixel ratio (scratch pixel number / total number of electric meter shell pixels), ∈[0, 1].

[0126] Finally, generate : combine D and Z in order, such as V1=[ , ].

[0127] ② Deep aging feature vector The extraction is based on a CNN deep network;

[0128] First, network training: the 4-6 layers of VGG16 (the last 3 convolutional blocks) are used to pre-train on the above data set (the true values of the display yellowing coefficient and the indicator light transmittance), focusing on fine visual features.

[0129] Then, the display yellowing coefficient is calculated: the display screen area is located by the YOLOv5-tiny target detection sub-network;

[0130] The display screen RGB image is calculated for the yellowing coefficient, and the formula can be:

[0131]

[0132] Among them, , , is the RGB channel value of the pixel ; , is the number of high and wide pixels of the display screen; , 0 represents no yellowing, and 1 represents severe yellowing.

[0133] Then, the indicator light transmittance is calculated: the indicator light area is located by the target detection sub-network, and if the indicator light is not lit, the lighting effect image is generated by simulating the lighting algorithm;

[0134] The transmittance is calculated, and the formula can be:

[0135]

[0136] Among them, the standard new indicator light brightness is the measured value of the same type of new meter, , 1 represents the transmittance 100%).

[0137] Finally, the is generated: , are combined in order, such as .

[0138] (2) The above obtained and are weighted and summed to generate an image aging feature vector , and the calculation formula is:

[0139]

[0140] Among them, and are preset weight coefficients, and . The weight The determination was made using a sample of 1000 electricity meters with known aging levels (using actual performance degradation rate as the aging rate standard). Through linear regression optimization, it was determined that when α=0.3 and β=0.7, The Pearson correlation coefficient with the actual degree of aging is 0.91, indicating a very strong correlation.

[0141] For example, if (D=0.2, S=0.3) (F=0.6, G=0.5), then:

[0142]

[0143] like Figure 4 This diagram illustrates the shallow and deep feature extraction capabilities of a CNN network in smart meter image processing, comprising five sub-images: (c) original meter image, (d) shallow feature: contour deformation response, (e) shallow feature: scratch density response, (f) deep feature: display screen yellowing coefficient response, and (g) deep feature: indicator light transmittance response. The diagram uses a JET color mapping, with a gradient from blue (low response value) to red (high response value) visually representing the response intensity of different feature regions.

[0144] The overall contour deformation degree and shell scratch density of the meter are extracted in the shallow layer of the convolutional neural network to generate a shallow aging feature vector. The shallow feature response subplots in the figure clearly illustrate this process: the contour deformation response map (d) shows the intensity change in the edge area of ​​the meter casing, with the highest response value reaching 0.85, reflecting the technical preparation for shape matching calculation with the standard undeformed meter contour template; the scratch density response map (e) shows the surface texture features, with obvious red high response areas at the scratch location, and a response value of about 0.7, which is consistent with the description above of using texture in the shallow network to extract convolution kernels and output scratch probability heatmaps.

[0145] The deep feature response sub-maps demonstrate the deep network feature extraction capabilities described in the patent: the yellowing coefficient response map (f) of the display screen adopts a radial gradient mode, with the highest response value in the central area (up to 0.9) and gradually weakening towards the edge, simulating the mathematical calculation process of calculating the yellowing coefficient F mentioned above; the light transmittance response map (g) of the indicator light shows a uniform response in the circular area (response value of about 0.6), corresponding to the calculation process of calculating the light transmittance G mentioned above.

[0146] Figure 4The middle blue area represents weak feature response, and the red area represents strong feature response. This visualization interprets the multi-scale feature fusion concept. The shallow feature focuses on macro-structure information (outline, scratch), and the deep feature focuses on micro-material changes (yellowing, light transmittance reduction). This multi-level feature extraction capability is not available in traditional image processing methods, highlighting the technical advantages of this method in the field of intelligent meter fault diagnosis.

[0147] It should be noted that due to uneven image lighting, the shallow network has large errors in extracting contour deformation and scratch density. For example, the outline is blurred under strong light, and the scratch is hidden under weak light. At this time, a dynamic light compensation attention mechanism module can be introduced, which is specifically:

[0148] Before the input layer of the CNN, an improved CBAM attention subnetwork is inserted: first, analyze the image lighting distribution to generate a lighting weight map (low weight for bright areas, high weight for dark areas); then dynamically weight the pixels according to the weight map (decrease value for bright areas, increase value for dark areas) to simulate uniform lighting.

[0149] In addition, due to the insufficient sensitivity of the deep network to slight aging features (such as slight yellowing and light transmittance reduction), early aging may be missed. Therefore, a multi-scale feature fusion deep network can be introduced. For the feature maps of different convolution layers (such as the 4th, 5th, and 6th layers) of the deep network, a feature pyramid network (FPN) is used for multi-scale fusion. After the fusion of small-scale features (capturing minor details such as slight yellowing) and large-scale features (capturing overall trends), the recognition rate of slight aging features is improved.

[0150] S3, combine real-time environmental data with aging coefficient , , , perform dynamic weighted fusion, and resolve the multi-modal data conflicts generated during the fusion process to generate a final comprehensive feature vector When the aging coefficient is greater than the preset aging threshold, the fusion weight of the electrical performance feature vector and the image aging feature vector is increased based on the initial weight.

[0151] In the initial state, the fusion weights of each modal feature vector are set to a set of reference values, for example, the image modal is 0.3, the electrical performance modal is 0.4, and the communication modal is 0.3. When the aging coefficient exceeds the preset aging threshold, the system automatically adjusts the fusion weight, increases the weights of the image modal and the electrical performance modal on the basis of the original weight, for example, the image modal is increased from 0.35 to 0.45, the electrical performance modal is increased from 0.4 to 0.5, and the weight of the communication modal is correspondingly reduced to 0.05, so as to reflect the higher reliability of the electrical performance and the image representation in the aging state.

[0152] By way of example, the final comprehensive feature vector The calculation formula can be expressed as:

[0153]

[0154] wherein, , and each weight coefficient is dynamically adjusted according to the aging coefficient and real-time environmental data. The dynamic weighting mechanism can adapt to different aging states and environmental conditions, and ensure the effectiveness and robustness of the comprehensive feature vector in the diagnosis process.

[0155] Optionally, the conflict resolution process comprises:

[0156] calculating the image modal confidence score , the electrical performance modal confidence score and the communication modal confidence score ; when the difference between the confidence scores of any two modes is greater than a preset conflict threshold, and the fault types indicated by the corresponding features contradict each other, it is determined that there is a multi-modal data conflict; a high-priority modal is determined by calling a preset fault type-modal priority association library to resolve the conflict; if the conflict is still not resolved, additional data is collected, and the feature vector and the confidence are recalculated.

[0157] In the above fusion process, if there is a multi-modal data conflict, a conflict resolution process is further performed. Specifically, the system first calculates the confidence scores of the image modal, the electrical performance modal and the communication modal, respectively, and the score range is defined as [0, 1]. The scores are based on: image clarity and feature extraction confidence, electrical performance sampling stability and fluctuation amplitude, communication log integrity and loss rate. When the difference between the confidence scores of any two modes is greater than a preset conflict threshold, and the fault types indicated by the corresponding features contradict each other, it is determined that there is a multi-modal data conflict.

[0158] After the conflict occurs, the system calls the preset fault type-mode priority association library to determine the high priority mode and adopt its conclusion, and the fault type-mode priority association library is pre-inputted with a large number of classifications and priority information about mode selection. For example, for structural faults, the image mode is preferred; for metering deviation faults, the electrical performance mode is preferred; and for communication abnormality faults, the communication mode is preferred.

[0159] If the conflict cannot be resolved through the priority library, the system triggers a supplementary collection mechanism, including: re-collecting images to optimize lighting conditions, increasing electrical performance sampling frequency to capture subtle fluctuations, or re-reading communication logs to exclude temporary interference. After completing the supplementary collection, the system recalculates the feature vector and credibility score until the conflict is resolved.

[0160] As an example, in the detection process of an old meter, feature extraction of image, electrical performance and communication history data has been completed, and the following feature vectors and parameters are obtained:

[0161] Image aging feature vector (corresponding to contour deformation and display screen yellowing coefficient, respectively);

[0162] Electrical performance feature vector corresponding to metering deviation, current fluctuation, contact resistance, voltage stability, etc.

[0163] Communication history feature vector corresponding to maximum metering deviation in the past 3 years, number of communication interruptions, fault label code = 1, indicating inaccurate metering;

[0164] Aging coefficient greater than the set threshold of 0.8; at the same time, the real-time environmental temperature = 31°C, humidity = 68%, without extreme fluctuations, and then dynamic weighted fusion is performed:

[0165] The initial weight setting is: image mode 0.3, electrical performance mode 0.4, communication mode 0.3. Since the weight adjustment mechanism is triggered: the image mode weight is raised to ; the electrical performance mode weight is raised to ; and the communication mode weight is reduced to .

[0166] The comprehensive feature vector is calculated as:

[0167]

[0168] During the specific calculation, the system first normalizes and extends the mapping of the vectors in different dimensions, and then weights each element. After calculation, it is obtained that:

[0169] Subsequently, the system calculates the credibility of the three types of modalities respectively: image modality credibility score = 0.78, the image is clear without bright spot interference, and the model classification confidence is high; electrical performance modality credibility score = 0.91, the data sampling is stable with small fluctuations; communication modality credibility score = 0.52, the communication log is missing part of the monthly data.

[0170] It can be further concluded that the electrical performance modality judgment result is → poor terminal contact; the communication modality judgment result is → inaccurate metering; the fault types of the two contradict each other, and the credibility difference = 0.91-0.52 = 0.39 > 0.3 (preset threshold), at this time the conflict is triggered.

[0171] The system starts to call the fault type-modality priority library: for the metering deviation type of fault, the priority is electrical performance modality > communication modality. Therefore, the conflict is automatically resolved by the priority library, and the electrical performance modality conclusion is adopted.

[0172] If the resolution fails (such as the same priority), the system will trigger supplementary collection, for example, the current sampling frequency is increased from 3 times / second to 5 times / second, the features and credibility are calculated again to ensure that the conflict is resolved.

[0173] Finally, the comprehensive feature vector generated by the system is used for subsequent similarity comparison and fault level division.

[0174] Through the above dynamic weighted fusion and conflict resolution mechanism, this step can ensure the reliability and accuracy of the final comprehensive feature vector under multi-modal conditions, effectively reduce the misjudgment rate, especially in old meters and extreme environmental conditions, the diagnosis accuracy is greatly improved compared with traditional single modal method, and the precision and stability of fault diagnosis and sorting are significantly improved.

[0175] As Figure 5 shows the dynamic adjustment strategy of multi-modal feature fusion weight with aging coefficient K, which clearly presents how the weight distribution of image modality (red curve), electrical performance modality (blue curve) and communication modality (green curve) is intelligently adjusted according to the aging degree of the meter. In the figure, the X-axis represents the aging coefficient K (0-1 range), and the Y-axis represents the fusion weight of each modality (0-1 range), and the background color is divided into three areas of light aging (light green), moderate aging (light yellow) and severe aging (light red), which corresponds to the fault severity level division in this paper.

[0176] As Figure 5As shown, when the aging coefficient K is less than 0.3, each modality adopts an equal weight distribution (image 0.30, electrical performance 0.40, communication 0.30), which reflects that in the mild aging state, each data source has similar reliability. When the K value enters the moderate aging interval of 0.3-0.8, the system starts a linear transition mechanism, the image and electrical performance modality weights gradually increase, and the communication modality weight decreases accordingly, Figure 5 The figure clearly shows this trend: the image weight increases from 0.30 to 0.45, the electrical performance weight increases from 0.40 to 0.50, and the communication weight decreases from 0.30 to 0.05. This adjustment logic is based on the technical insight that the more severe the aging, the higher the reliability of electrical performance and image representation.

[0177] It is particularly important that when the K value exceeds the severe aging threshold of 0.8 (red marked point in the figure), the weight distribution changes significantly: the image modality weight stabilizes at 0.45, the electrical performance modality weight reaches 0.50, and the communication modality weight decreases to 0.05. This distribution strategy embodies the innovative method of increasing the fusion weight of the electrical performance feature vector over the image aging feature vector based on the initial weight.

[0178] Figure 5 Red represents the image modality, emphasizing the importance of appearance aging features; blue represents the electrical performance modality, reflecting the reliability of electrical performance parameters; green represents the communication modality, whose weight decreases with increasing aging degree, consistent with the actual situation that the reliability of the communication module decreases in the aging meter. Compared with the traditional fixed weight fusion method, this self-adaptive adjustment strategy based on the aging coefficient can more accurately reflect the reliability differences of each data source under different aging states, significantly improving the accuracy and robustness of fault diagnosis.

[0179] S4, step S4 is the decision-making core link after multi-modal data fusion, which receives the final comprehensive feature vector generated in step S3, realizes accurate fault type determination through standard vector similarity matching, and completes fault severity classification in combination with the aging coefficient calculated in step S2, while relying on the knowledge base self-updating mechanism to ensure long-term diagnosis reliability. The specific implementation of step S4 is described in detail as follows:

[0180] 1. Construction of pre-set knowledge base;

[0181] The knowledge base is used to store standard feature vectors and associated metadata of different fault types, and is the benchmark database for fault determination. Its construction needs to meet the requirements of comprehensive sample coverage, accurate feature labeling, and unified dimensions, and the specific process is as follows:

[0182] (1) Sample collection and screening: Collect more than 1,000 smart meter samples covering normal conditions and typical fault types. The fault types cover three major categories: communication, structure, and core components (for various faults and corresponding repair methods, and examples of standard vector identifiers, please refer to the fault type and standard vector identifier comparison table below); the samples should include different manufacturing batches (2018-2024) and different aging levels (aging coefficient). The range of values ​​is This is to avoid judgment bias caused by a single sample.

[0183]

[0184] Fault Type and Standard Vector Identifier Comparison Table

[0185] (2) Standard feature vector generation

[0186] For each sample meter, strictly execute steps S1 (multi-modal data acquisition) and S2 (aging coefficient). After calculation and step S3 (dynamic weighted fusion of feature vectors), a standard comprehensive feature vector is generated. To ensure the validity of similarity calculations, all With the output of step S3 The dimensions are uniform (e.g., 32-dimensional), and each fault type corresponds to at least 20 sample vectors. The mean vector of these vectors is taken as the benchmark vector for that fault type to reduce the random error of a single sample.

[0187] (3) Metadata annotation

[0188] For each baseline standard vector Bind metadata, including fault type name, repair method, and typical aging factor. Value range and feature matching priority. For example, measurement-related faults have a higher matching priority than appearance-related faults, providing a basis for subsequent fault type conflict resolution and severity level classification.

[0189] 2. The above knowledge base has a self-updating mechanism: after completing the sorting task of a specified batch, the feature matching accuracy of various faults is calculated. When the accuracy of any fault type is lower than the preset accuracy threshold, the multimodal feature combination corresponding to that fault type is automatically updated.

[0190] As electricity meter technology evolves, such as with new chips and new fault modes, the knowledge base needs to be dynamically updated to avoid a decline in accuracy. The triggering conditions and execution flow of the update mechanism are quantitatively designed, as follows:

[0191] (1) Update trigger conditions:

[0192] Batch cycle: 100 meters as a designated batch;

[0193] Accuracy threshold: the feature matching accuracy of a certain fault type is ≥95% (below this value indicates that the standard vector is not suitable for new samples);

[0194] Accuracy calculation method: accuracy = .

[0195] (2) Automatic update process: taking terminal oxidation fault as an example, if the accuracy of this type in a batch decreases to 92% (<95%), the system automatically executes:

[0196] Sample supplement: automatically select samples that are incorrectly judged in this batch (such as 3 meters that are misjudged as normal) + collect 5 new terminal oxidation fault meters (to ensure sample representativeness);

[0197] New standard vector generation: generate 8 new feature vectors according to S1-S3 steps for 8 supplementary samples, and calculate the mean vector ;

[0198] Vector replacement: replace the old terminal oxidation standard vector in the knowledge base with , and mark the update time (such as 2024-05-10) and the basis for updating, such as supplementing 8 samples, the accuracy is improved from 92% to 96.5%;

[0199] Verification: after updating, the next batch (>100 meters) is used for verification to ensure that the accuracy of this fault type is ≥95%, if it still does not meet the requirements, repeat the above process.

[0200] (3) Update effect verification: assuming that after 10 batches (1000 meters) of long-term testing, the average accuracy of each fault type is stable at 95.2%-98.5% after enabling the self-updating mechanism, without updating, the accuracy of the 10th batch decreases to 89.3%; The misjudgment rate of unknown faults decreases from 0.3% to 0.1%, and at this point, new fault modes are quickly included in the knowledge base.

[0201] 3. Calculate the matching degree of the final comprehensive feature vector with the standard comprehensive feature vector in the knowledge base using cosine similarity. Cosine similarity can effectively measure the direction consistency of high-dimensional vectors (i.e. the similarity of fault features), avoiding the defects of Euclidean distance affected by vector length. The specific implementation is as follows:

[0202] (1) Vector dimension consistency guarantee: is consistent with all The dimensions must be consistent (e.g., 32 dimensions). If there are differences in the dimensions of the image feature vector (2 dimensions), electrical performance feature vector (25 dimensions), and communication feature vector (5 dimensions) in step S2, they will be unified to 32 dimensions during fusion in step S3 through zero-padding and feature mapping operations, with each dimension corresponding to the same physical meaning. For example, dimensions 1-5 correspond to communication features, dimensions 6-30 correspond to electrical performance features, and dimensions 31-32 correspond to image features.

[0203] (2) Similarity calculation process: The cosine similarity is calculated strictly according to the following formula, and the calculation result is retained to 4 decimal places to avoid precision loss:

[0204]

[0205] Among them, molecules The dot product of two vectors (if , The dot product is ); denominator for L2 norm (i.e. ); for The L2 norm is calculated in the same way.

[0206] (3) Determination of the judgment threshold: 5-fold cross-validation was performed using 1000 sets of meters in known states. The diagnostic accuracy, false positive rate, and false negative rate under different similarity thresholds were statistically analyzed, and the final similarity judgment threshold was determined to be 0.85. At this point, the accuracy of fault type determination (number of correctly judged samples / total number of samples) reached 98.2%, the false positive rate (the proportion of normal states misjudged as faults or fault types misjudged) was 1.5%, and the false negative rate ( With all The proportion of samples with similarity less than the threshold is 0.3%, and the missed samples need to trigger a manual review process to avoid system misjudgment.

[0207] (4) Fault type determination rule: If there is one and only one fault type satisfy Then the meter to be diagnosed is directly determined to be this one. Corresponding fault type;

[0208] If there are multiple satisfy Then select similarity The largest Corresponding fault type; if maximum similarity If they are the same, the final fault type is determined by referring to the feature matching priority in the knowledge base (e.g., measurement-related faults take precedence over appearance-related faults);

[0209] If all similarity of the similarity score If so, it is determined as an unknown fault, and the manual review process is triggered.

[0210] As Figure 6 The intuitive display shows the distribution characteristics of the similarity score after multi-modal feature fusion and its relationship with the decision threshold. Figure 6 In the figure, four colors are used to distinguish the score distribution of different fault states: blue represents the normal state, green represents the first fault, yellow represents the second fault, and red represents the third fault. The black dashed line represents the decision threshold defined in the patent (0.85). Figure 6 The high-contrast bright green (right) and bright red (left) in the background area represent the two decision areas of (normal / repairable) and (determined as fault), respectively, making the threshold segmentation effect more obvious.

[0211] Figure 6 In the figure, the score distribution clearly shows the distinguishing characteristics of different fault states: the normal state scores are concentrated in the high-score area of 0.85-1.0 (average score 0.91), the first fault score distribution is in the range of 0.6-0.95 (average score 0.75), the second fault score distribution is in the range of 0.4-0.85 (average score 0.55), and the third fault score distribution is in the range of 0.2-0.7 (average score 0.35).

[0212] The decision threshold 0.85 (black dashed line) divides the score space into two obvious decision areas: the bright green area on the right (score ≥ 0.85) represents the decision of normal or repairable state, and the bright red area on the left (score < 0.85) represents the decision of fault state. This threshold setting is consistent with the description in the above 1000 known state ammeter samples through 5-fold cross-validation, and the final similarity decision threshold is 0.85. Figure 6 The distribution characteristics shown in the figure prove that the method can effectively distinguish different fault states, supporting the innovative proposition of improving fault diagnosis accuracy through multi-modal data fusion.

[0213] 4. Fault severity level division combined with aging coefficient K. Since fault severity needs to consider both the severity of the fault itself and the aging state of the ammeter (K), to avoid misclassification caused by a single factor (such as core component damage, which requires division into a serious fault even if K is small, or a software bug, which requires division into a minor fault even if K is large). The specific division logic and quantitative standards are as follows:

[0214] First fault: minor fault, defined as aging coefficient Less than 0.5, and the fault type is a fault that can be repaired by software or simple manual intervention. Typical examples of faults include communication timeout of the electric meter (parameter configuration error), and metering deviation <1% (software calibration repairable). The processing suggestion is automatic repair (such as remote issuance of calibration instructions) + marking for continued use.

[0215] Secondary fault: moderate fault, defined as the aging coefficient Between 0.3 and 0.8, and the fault type is a fault that requires professional calibration or replacement of secondary components. Typical examples of faults include oxidation of the terminal (contact resistance >0.4Ω), and light transmittance of the indicator lamp <60%. The processing suggestion is manual replacement of components + professional calibration + marking for use after repair.

[0216] Third-level fault: serious fault, defined as the aging coefficient Greater than 0.8 or the fault type is a fault that cannot be repaired, such as burning of the metering chip, short circuit of the mainboard circuit, and K=0.85 (severe aging). The processing suggestion is direct sorting to the scrap area + recording of the fault cause (for manufacturer improvement).

[0217] S5, according to the corresponding automatic sorting operation of the fault level, and based on the feedback of the sorting result, the key parameters in the diagnosis process are optimized in a closed loop. In step S5, on the one hand, the efficient classification and disposal of the faulty electric meter is realized through automatic sorting, and the sorting efficiency in the industrial scene is improved; on the other hand, based on the deviation feedback of the sorting result, the key parameters in the diagnosis process (such as the aging coefficient calculation in S2 and the similarity determination in S4) are dynamically optimized, the problem of poor adaptability caused by manual sorting and fixed parameters in the traditional method is solved, and the long-term diagnosis reliability of the system under the iteration of the electric meter batch and the change of the environment is guaranteed. Step S5 specifically includes two parts of automatic sorting operation and closed loop optimization, wherein:

[0218] 1. The specific implementation process of the automatic sorting operation: this process aims to convert the fault level instruction output by S4 into high-speed and accurate physical sorting actions on the production line, realizing efficient classification and disposal of the faulty electric meter.

[0219] (1) Input: the final diagnosis conclusion output by step S4, for example: secondary fault: communication module abnormality or third-level fault: metering chip damage.

[0220] (2) Execution logic and hardware cooperation: information transmission: the diagnosis system sends instructions containing a unique identifier ID and a fault level to the main control PLC (Programmable Logic Controller) of the production line.

[0221] Physical positioning: the electric meter moves on the conveyor belt, and when it passes through the code reader (such as an industrial camera or a code scanning gun), its unique identifier ID is read, and the PLC confirms its physical location.

[0222] (3) Execution sorting: When the meter reaches the designated sorting gate, the PLC drives the corresponding execution mechanism (such as a pneumatic push rod, a sorting arm, or a flipper) according to the received fault level instructions to perform the following operations:

[0223] Level 1 fault: The meter is diverted to the software calibration / easy maintenance channel;

[0224] Level 2 fault: The meter is diverted to the manual fine repair / replacement parts channel;

[0225] Level 3 fault: The meter is diverted to the scrap / recycle channel;

[0226] No fault: The meter is diverted to the qualified / reuse channel.

[0227] The entire sorting process is precisely controlled by the PLC, with instruction transmission and physical action corresponding one-to-one, completely eliminating the problems of fatigue, misjudgment, and missed judgment that may occur in manual sorting, and the sorting accuracy can approach 100% (depending on the stability of the mechanical execution mechanism). Compared with the traditional manual sorting method relying on the naked eye and simple tools, the processing speed of the automatic sorting line can be improved by tens of times, greatly improving the industrialized disposal capacity. Significantly reduces the number of sorting workers, reduces labor costs and management costs.

[0228] 2. Specific implementation process of closed-loop optimization: This is the key innovation point that distinguishes this method from traditional one-time diagnostic methods. It establishes a feedback loop from sorting results to diagnostic parameters, enabling the system to have self-learning and adaptive capabilities.

[0229] (1) Trigger mechanism: This optimization process is not performed in real time, but is triggered after completing a specified batch (e.g., 1000 meters) of sorting tasks.

[0230] (2) Feedback data collection, including:

[0231] System diagnostic data: Collect the original diagnostic results of all meters in this batch (fault type distribution, number of each level, etc.);

[0232] Manual review / maintenance feedback: Collect real fault data from downstream maintenance and quality inspection. For example, a meter that was judged by the system as a level 2 fault was confirmed by the engineer as unrepairable during maintenance, and its real fault level should be level 3. This difference is the key source of error rate.

[0233] (3) Analysis and decision-making:

[0234] The system compares the system diagnostic data with the real fault data, focusing on analyzing two indicators:

[0235] Fault type distribution change: compare the fault type composition of the current batch with historical batches (e.g., the past 10 batches). For example, historically, communication faults accounted for an average of 5%, but in the current batch, it surged to 20%.

[0236] Error rate change: whether the misjudgment rate of a specific fault type exceeds the warning threshold.

[0237] When the change of any indicator exceeds the preset trend threshold (e.g., the proportion of a certain type of fault changes by more than 15%, or the error rate exceeds 5%), the system determines that the adaptability of the diagnosis model may have decreased, and needs to be automatically adjusted.

[0238] (4) Parameter automatic adjustment includes:

[0239] The system automatically adjusts one or more key parameters in the upstream steps according to the pre-set problem-parameter association rule library. For example: if a large number of communication faults are misjudged or missed, the system may automatically increase the communication signal gain to adapt to the new batch of meters that may have generally weak signals;

[0240] If minor scratches on the appearance are frequently misjudged as faults, the system may increase the feature similarity judgment threshold, making the model more tolerant to minor flaws;

[0241] If the number of misjudgments of display screen problems caused by reflection increases, the system will lower the bright spot threshold and force the front-end image acquisition to make more stringent light adjustment;

[0242] If the overall metering performance fluctuation detection is not sensitive, the system will shorten the sampling interval and perform higher frequency data collection.

[0243] For example, Figure 7 The figure shows the significant improvement in the performance of the smart meter fault diagnosis by the closed-loop optimization mechanism. Figure 7 The upper graph (h) shows the trend of diagnosis accuracy, and the lower graph (i) shows the improvement of misjudgment rate and missed judgment rate.

[0244] Figure 7 The closed-loop optimization mechanism in the figure is triggered after completing the sorting task of a specified batch (e.g., 100 meters per batch), and automatically adjusts the system parameters by comparing the fault type distribution and misjudgment rate change of the current batch with historical batches. Figure 7 The figure clearly shows that the diagnosis accuracy before optimization fluctuates at a low level, while after closed-loop optimization, the accuracy is significantly improved and stabilized at a high level, fully meeting and exceeding the 95% accuracy threshold target set in the above (black dotted line).

[0245] The red curve in the upper graph (h) represents the performance before optimization, and the green curve represents the excellent results after optimization. The strong color contrast highlights the positive effect of closed-loop optimization. In particular, as the batch size increases, the accuracy of the optimized model stabilizes at a high level, which is consistent with the description above that as the optimization mechanism continues to work, the system performance gradually stabilizes at a high level.

[0246] The lower graph (i) further demonstrates the improvement of the misjudgment rate and the missed judgment rate: before optimization, the misjudgment rate and the missed judgment rate fluctuate at a high level, and after optimization, both error rates are significantly reduced. The significant reduction in error rate verifies the technical advantages of the key parameter closed-loop optimization in the diagnostic process through feedback.

[0247] Figure 7 The optimization effect of the initial batch gradually appears, and the subsequent batches stabilize at a high level. This performance improvement trend proves that the method can continuously improve the diagnostic performance through self-learning and parameter adjustment, which is an advantage that traditional fixed parameter diagnostic methods do not have.

[0248] In addition, the following problems need to be noted:

[0249] Due to the significant difference in communication data format between different factory batches of smart meters (such as RS485 protocol meters produced in 2018 and LoRa protocol meters produced in 2024): RS485 protocol data contains 3 types of features (dimension 3) such as voltage, current, and power, while LoRa protocol data adds 2 types of features (dimension 5) such as battery power and network signal strength. The existing fusion scheme uses zero padding to unify the dimensions (such as filling RS485 data to 5 dimensions), which will cause distortion of the feature space - zero value features cannot reflect the actual state of the meter, thereby affecting the discrimination of the fusion vector.

[0250] Specifically, (1) feature redundancy and information loss: zero padding causes the fusion vector of RS485 meters to contain meaningless dimensions, while the key features of LoRa meters (such as network signal strength) cannot effectively participate in similarity calculation due to mixing with zero value dimensions of RS485 meters, resulting in a 25% decrease in the accuracy of S4 in diagnosing communication faults;

[0251] (2) Poor adaptability of knowledge base: when the standard vector in the knowledge base based on a single protocol (such as the LoRa protocol standard vector with dimension 5) is matched with the fusion vector of RS485 meters (dimension 5, containing 2 zero values), the similarity is generally lower than the threshold, causing the proportion of unknown fault diagnosis to rise to 12%.

[0252] Therefore, an adaptive protocol analysis and transfer learning feature mapping engine is proposed here, which designs a fusion module with protocol self-recognition and feature space unification capability, and the specific implementation process is as follows:

[0253] Protocol automatic identification: deploy a protocol feature extractor at the data acquisition end, automatically identify the communication protocol type of the electric meter by analyzing the start symbol of the communication frame, such as the RS485 protocol start symbol 0x01, the LoRa protocol 0x0A, the data length, the verification method and other characteristics;

[0254] Core feature extraction: for different protocols, define a set of core features associated with faults, for example, for the RS485 protocol, extract the voltage fluctuation amplitude, current harmonic content and power factor; for the LoRa protocol, extract the voltage fluctuation amplitude, current harmonic content, power factor, network signal strength and battery power decay rate, to ensure that the features of each protocol are strongly related to fault determination, and to eliminate redundant fields, such as the device number in the LoRa protocol, which is not a fault-related feature;

[0255] Transfer learning feature mapping: a transfer learning model based on a convolutional neural network (CNN) is used to map the core features of different protocols to a unified feature space of 8 dimensions. The model uses a multi-protocol fault sample set as training data, and through a pre-trained feature extraction layer (such as the first 8 layers of ResNet18), it converts each protocol feature into a high-dimensional vector, and then through a fully connected layer, it is mapped to an 8-dimensional unified space. For example, the 3-dimensional core features of the RS485 protocol are mapped to an 8-dimensional vector, and the 5-dimensional core features of the LoRa protocol are also mapped to an 8-dimensional vector, and the features of different protocol electric meters of the same fault type (such as communication interruption) in the unified space have a distance of less than 0.1 (cosine distance);

[0256] Feature importance weighting: the feature importance of each dimension in the unified space is calculated by a random forest algorithm, for example, the importance weight of the voltage fluctuation amplitude is 0.2, and the network signal strength is 0.15, and the importance is assigned a weight during fusion to ensure that key features have a higher contribution to the fusion vector.

[0257] Through the above scheme, the feature matching accuracy of different protocol electric meters can be improved, and the proportion of unknown fault determination can be greatly reduced.

[0258] Embodiment two:

[0259] As shown in Figure 8 , corresponding to the above method, the present application also proposes an intelligent electric meter sorting and fault diagnosis system based on multi-modal data fusion, comprising:

[0260] A data acquisition module is used to acquire the basic information, real-time environmental data, historical environmental data, electrical performance data, image data and communication history data of the intelligent electric meter to be processed;

[0261] A data processing module is used to calculate the aging coefficient based on the historical environmental data, and dynamically adjust the detection parameters of the electrical performance data according to the coefficient, and then process the electrical performance data;

[0262] a feature extraction module configured to extract an image aging feature vector, an electrical performance feature vector, and a communication history feature vector from the image data, the electrical performance data, and the communication history data, respectively;

[0263] a data fusion module configured to combine real-time environmental data and an aging coefficient, dynamically weight and fuse various types of data, and eliminate multi-modal data conflicts during the fusion process to generate a final comprehensive feature vector;

[0264] a fault diagnosis module configured to perform similarity calculation between the final comprehensive feature vector and a standard feature vector preset in a knowledge base, to determine a fault type of the electric meter, and to perform fault severity level division according to the fault type and the aging coefficient;

[0265] an automatic sorting module configured to perform an automatic sorting operation according to the fault severity level and to perform key parameter closed-loop optimization in the diagnosis process through feedback.

[0266] The system successfully solves the limitations in the traditional intelligent electric meter fault diagnosis method by introducing multi-modal data fusion technology. By comprehensively utilizing image data, electrical performance data, communication history data, and environmental information, the application can comprehensively analyze the running state and potential faults of the electric meter from multiple dimensions, effectively improving the accuracy and reliability of fault diagnosis. In particular, in the process of dynamically adjusting detection parameters and fusing various data features, the application not only improves the identification accuracy of fault types, but also reasonably divides the fault severity levels, thereby realizing automatic sorting of the intelligent electric meter and closed-loop optimization of key parameters.

[0267] Embodiment Three

[0268] Corresponding to the above-mentioned embodiments, the application further provides an electronic device.

[0269] As shown in Figure 9 The structure of the electronic device 200 is not limited to the application embodiments. The electronic device 200 includes a processor 201 and a memory 203. The processor 201 and the memory 203 are connected, for example, through a bus 202. Optionally, the electronic device 200 can also include a transceiver 204. It should be noted that in actual applications, the transceiver 204 is not limited to one, and the structure of the electronic device 200 does not constitute a limitation on the application embodiments.

[0270] The processor 201 can be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in conjunction with the present disclosure. The processor 201 can also be a combination of computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0271] The bus 202 can include a path for transmitting information between the above-mentioned components. The bus 202 can be a PCI bus or an EISA bus, etc. The bus 202 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 9 In the figure, only one thick line is used, but it does not mean that there is only one bus or one type of bus.

[0272] The memory 203 is used to store a computer program corresponding to the intelligent electric meter sorting and fault diagnosis method of the multi-modal data fusion of the above-mentioned embodiments of the present application, which is controlled and executed by the processor 201. The processor 201 is used to execute the computer program stored in the memory 203 to realize the content shown in the foregoing method embodiments.

[0273] Among them, the electronic equipment 200 includes but is not limited to: notebook computers, PAD (tablet computers) and other mobile terminals, and fixed terminals such as desktop computers and the like. Figure 9 The electronic equipment 200 shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0274] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description, as represented by the above listed elements, by the steps recited in the flow charts, and by the examples that follow, without departing from the spirit of the application. Accordingly, the scope of the present application is intended to be defined only by the appended claims.

[0275] It should be understood that aspects of the application can be implemented in hardware, software, firmware or a combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, can be used: a hybrid of the above technologies, a combination of any of the above technologies, etc.

[0276] In the description of the present application, the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" are intended to mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. The illustrative appearances of the above-mentioned terms in various places in the specification are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0277] Furthermore, the terms "first", "second", etc. are used only for descriptive purposes and do not connote or imply relative importance or a quantity of the indicated technical features. Thus, a feature defined with "first", "second", etc. can include at least one of the features implicitly or explicitly. In the description of the present application, the meaning of "a plurality" is at least two, for example, two, three, etc., unless otherwise specifically defined.

[0278] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A multi-modal data fusion based smart meter sorting and fault diagnosis method, characterized in that, The method comprises the following steps: S1, obtaining the basic information of the smart meter to be processed, collecting the real-time environment data and historical environment data thereof, assigning a unique identifier to each meter, and associating and binding the information and data; S2, collecting image data, electrical performance data and communication history data of the smart meter; an aging coefficient is calculated based on the historical environmental data and the detection parameters are dynamically adjusted according to the aging coefficient an aging coefficient is calculated based on the historical environmental data and the detection parameters are dynamically adjusted according to the aging coefficient an aging coefficient is calculated based on the historical environmental data and the detection parameters are dynamically adjusted according to the aging coefficient an aging coefficient is calculated based on the historical environmental data The dynamic adjustment of the detection parameters comprises: adjusting the measurement precision allowable deviation, the current sampling frequency and the terminal contact resistance detection threshold according to different intervals of the values, and standardizing the electrical performance data after the adjustment of the detection parameters. Then features are extracted from the image data, the normalized electrical performance data, and the communication history data, respectively, to generate an image aging feature vector , an electrical performance feature vector , and a communication history feature vector ; S3, combine the real-time environment data with the aging coefficient , the real-time environment data is used to dynamically weight the fusion of the image aging feature vector and the electrical performance feature vector , , , when the aging coefficient is greater than a preset aging threshold, the fusion weight of the image aging feature vector and the electrical performance feature vector is increased on the basis of the initial weight ​​​ And the multi-modal data conflict generated in the fusion process is resolved, and a high priority mode is determined by calling a preset fault type-mode priority association library to resolve the conflict and generate a final comprehensive feature vector ; S4. The final integrated feature vector The similarity is calculated with the standard feature vectors preset in the knowledge base to determine the fault type, and the aging coefficient is also considered. Classify the severity of the faults into levels; S5, performing corresponding automatic sorting operation according to the fault level, and feeding back based on the sorting result to optimize the key parameters in the diagnosis process in a closed loop.

2. The method of claim 1, wherein, In step S2, the aging coefficient The calculation formula is: wherein the total storage duration is the total storage time after the meter is returned, in months; the average monthly humidity is the average monthly relative humidity during the storage, in percent; and the extreme environment times is the number of times when the environment temperature is higher than 40°C or the humidity is higher than 80% during the storage.

3. The method of claim 1, wherein, In step S2, before collecting image data, the camera array is used to perform initial collection and identify bright spots in the image; If the pixel proportion of the bright spot area exceeds a preset threshold, the light compensation direction is determined according to the center position of the bright spot, the power of the adjacent light compensation lamp is adjusted, and then the collection is performed again until the pixel proportion of the bright spot is lower than the preset threshold, and a set of non-interference images are output.

4. The method of claim 1, wherein, In step S2, the image aging feature vector The generation process includes: extracting the overall contour deformation degree of the electric meter and the shell scratch density in the shallow network of the convolutional neural network to generate a shallow aging feature vector ; In the deep network of the convolutional neural network, the display yellowing coefficient and the indicator light transmittance are extracted to generate a deep aging feature vector ; performing weighted summation on the image aging feature vectors generated by the image aging feature vector generation unit and performing weighted summation on the image aging feature vectors generated by the image aging feature vector generation unit , and the calculation formula is wherein, and are preset weight coefficients, and .

5. The method of claim 1, wherein, In step S3, the process of resolving the multi-modal data conflict generated in the fusion process comprises: Respectively calculating image modality credibility scores , electrical performance modality credibility scores , and communication modality credibility scores ; When the difference between the credibility scores of any two modalities is greater than a preset conflict threshold, and the fault types indicated by the corresponding features are contradictory, it is determined that there is a multi-modal data conflict; If the conflict is still not resolved, additional data is collected, and the feature vector and credibility are recalculated.

6. The method of claim 1, wherein, In step S4, the knowledge base has a self-updating mechanism: After completing the sorting task of a specified batch, the feature matching accuracy of each type of fault is counted, and when the accuracy of any fault type is lower than a preset accuracy threshold, the multi-modal feature combination corresponding to the fault type is automatically updated.

7. The method according to claim 1 or 6, characterized in that, In step S4, the similarity calculation employs a preset similarity metric function The calculation result is defined as a similarity score, and the similarity score The calculation formula is: wherein, is a standard integrated feature vector stored in the knowledge base; When the calculated similarity score is greater than or equal to a preset decision threshold, the power meter is preliminarily determined as the corresponding state.

8. The method of claim 1, wherein, In step S4, the fault severity is divided into three levels, including: First failure: minor failure, defined as the aging factor Less than 0.5 and the failure type is a failure that can be repaired by software or simple manual intervention; Secondary failure: moderate failure, defined as an aging coefficient between 0.3 and 0.8 and the failure type is a failure requiring professional calibration or replacement of a secondary component; Level 3 failure: Critical failure, defined as an aging coefficient Greater than 0.8 or a failure type of core component damage, unrepairable failure.

9. The method of claim 1, wherein, In step S5, the closed loop optimization includes: Comparing the fault type distribution and error rate of the current batch with those of the historical batches, if the change exceeds a preset trend threshold, at least one system parameter is automatically adjusted, and the system parameters include communication signal gain, sampling interval, bright spot threshold or feature similarity determination threshold.

Images