An electric meter image key information recognition and abnormal state diagnosis system and method

Abstract

The application belongs to the field of electric power information systems, and provides an electric meter image key information recognition and abnormal state diagnosis system and method, visible light images, infrared thermal imaging data and sound signals of an electric meter are synchronously acquired; the visible light images are subjected to graying, denoising and distortion correction; the contrast and edges of the visible light images are subjected to weighted correction based on infrared thermal distribution, and an enhanced electric meter image is obtained; a corresponding recognition model is selected according to a phenotype, and a preliminary recognition result of an electric meter reading is extracted; fluctuation characteristics are compared with spectral fluctuation characteristics of the sound signals in the same period; after comparison is completed by an infrared fluctuation verification unit, a time sequence spectral fluctuation mode of the sound signals is compared with reading fluctuation corresponding to the preliminary recognition result; the comparison results of the preliminary recognition result infrared fluctuation verification unit and the sound fluctuation verification unit are combined, and key information and an abnormal state diagnosis conclusion of the electric meter are output.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of electric power information system, and particularly relates to an electric meter image key information recognition and abnormal state diagnosis system and method. BACKGROUND

[0002] With the popularization of smart grid and smart energy management, the accuracy and real-time demand of electric energy measurement are continuously improved. The traditional manual meter reading method is gradually replaced by automatic meter reading technology based on image recognition due to its low efficiency and error-prone. By installing an industrial camera to capture electric meter images and using optical character recognition or deep learning models to analyze the images, remote automated reading acquisition can be achieved. This method can achieve good recognition results in an ideal environment, and is widely used in daily measurement scenarios of residential, commercial and industrial electric meters.

[0003] The commonly used means in the prior art mainly includes two categories. One is the OCR recognition for electronic digital meters, which obtains the reading by positioning the digital region and recognizing the characters one by one. The other is the image analysis method for mechanical pointer meters and roller type meters, which usually relies on edge detection, geometric correction and angle measurement to realize reading calculation. Some researches also introduce deep learning models such as convolutional neural networks to improve the recognition rate under uneven lighting, stain shielding or character blur.

[0004] However, the prior art relies on a single image signal, which is easily disturbed by external environment and electric meter aging factors in actual application. When the electric meter glass has scratches or yellowing, or the surface is covered with dust and oil stains, the image quality is significantly reduced, resulting in reduced recognition accuracy. At the same time, the image method cannot directly reflect the physical running state of the electric meter in the working process. Once there are abnormalities such as reading stagnation, roller jamming or being tampered by human, it is difficult to make a reliable judgment only by image recognition, thereby limiting its application value in complex scenarios. SUMMARY

[0005] In order to solve the problems in the prior art, the present application provides an electric meter image key information recognition and abnormal state diagnosis system, comprising:

[0006] A multi-modal data acquisition module for synchronously acquiring visible light images, infrared thermal imaging data and sound signals of the electric meter;

[0007] An image preprocessing unit for performing grayscale, denoising and distortion correction on the visible light images;

[0008] An infrared guided enhancement unit for pixel-level alignment of the infrared thermal imaging data and the preprocessed visible light images, and weighted correction of the contrast and edges of the visible light images based on the infrared thermal distribution to obtain an enhanced electric meter image.

[0009] a key information recognition module configured to perform phenotype identification on the enhanced meter image and select a corresponding recognition model according to the phenotype to extract a preliminary recognition result of the meter reading;

[0010] an infrared fluctuation verification unit configured to perform time series analysis on the collected infrared thermal imaging data in a current meter reading cycle, extract an infrared fluctuation feature sequence including a temperature mean value change curve, a temperature gradient change curve, and a spectrum fluctuation feature formed therefrom, and perform corresponding comparison between the infrared fluctuation feature sequence and a spectrum fluctuation feature extracted based on a short-time frequency spectrum energy distribution of the sound signal in the same meter reading cycle to verify whether the sound signal is consistent with the actual running state of the meter;

[0011] a sound fluctuation verification unit configured to compare a time series spectrum fluctuation pattern of the sound signal with a reading fluctuation corresponding to the preliminary recognition result after the comparison by the infrared fluctuation verification unit is completed;

[0012] an abnormal state diagnosis module configured to output key information and an abnormal state diagnosis conclusion of the meter in combination with the preliminary recognition result and comparison results of the infrared fluctuation verification unit and the sound fluctuation verification unit.

[0013] Further, the image preprocessing unit includes three steps of grayscale processing, noise suppression, and distortion correction, the noise suppression adopts a Gaussian filtering or median filtering mode, and the distortion correction remaps the image by using camera calibration parameters to eliminate barrel or pillow distortion.

[0014] Further, the infrared guided enhancement unit performs pixel-level alignment in a feature point registration mode, selects four corners of a meter outer frame, a digital window edge, or a scale area edge as registration feature points, and realizes mapping of infrared images and visible light images through affine or perspective transformation.

[0015] Further, after the pixel-level alignment is completed, a weight map is generated based on an infrared temperature gradient, a higher edge enhancement weight is given to a region with a large temperature gradient in the visible light image, and the contrast of a pointer, a number, or a scale area is improved.

[0016] Further, the key information recognition module includes a phenotype identification subunit and a multi-type recognition subunit, the phenotype identification subunit is configured to determine whether the meter is a mechanical pointer meter, an electronic digital meter, or a roller type meter, and the multi-type recognition subunit outputs a reading according to different recognition strategies.

[0017] Further, the infrared fluctuation verification unit extracts the regional average temperature, the regional high-quantile temperature and the regional internal temperature difference when analyzing the infrared time sequence, and calculates the lag relationship between the infrared curve and the sound energy envelope in the trend comparison to confirm the credibility of the sound signal.

[0018] Further, the sound fluctuation verification unit extracts the sound spectrum energy distribution by short-time Fourier transform, and calculates the energy envelope on the preset target frequency band, and further matches the sound fluctuation features with the reading change trend corresponding to the image recognition result.

[0019] Further, the sound fluctuation verification unit allows small-range time elasticity in the comparison process, performs coarse alignment and refined alignment on the sound event and the reading event, and generates a consistency score according to the trend isotropy, key event correspondence and duration matching degree.

[0020] Further, the abnormal state diagnosis module performs step-by-step analysis on data integrity, consistency and conflict attribution when fusing the judgment, and outputs a normal, abnormal or undeterminable state conclusion according to the judgment result, while attaching explanatory metadata.

[0021] The present application also provides an electric meter image key information recognition and abnormal state diagnosis method, which uses the electric meter image key information recognition and abnormal state diagnosis system to recognize the key information of the electric meter image and diagnose the abnormal state.

[0022] The present application introduces multi-modal verification of infrared and sound based on image recognition, and constructs a closed-loop process of step-by-step checking. The connection between the recognition result and the physical running state is established, which not only avoids the limitations caused by relying solely on images, but also enables the system to maintain stable recognition accuracy in the case of dust coverage, glass aging or complex light, thereby improving the reliability of electric meter automatic recognition.

[0023] Through the fluctuation comparison of infrared thermal imaging and sound signal, the system can capture the temperature and acoustic characteristics of the electric meter under load change, and then judge whether the image recognition result is consistent with the actual running state. This chain verification mechanism not only effectively discovers reading abnormalities, equipment failures or human interference, but also provides reliable evidence information to ensure that the output diagnosis conclusion has traceability and engineering interpretability.

[0024] The overall architecture of the present application closely integrates acquisition, enhancement, recognition and diagnosis to form a complete processing chain from data acquisition to result output. Compared with single modal method, the system can work stably in various complex scenes, reduce the occurrence of misjudgment and omission, and provide safer, more reliable and efficient technical support for smart energy management and intelligent meter reading. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described below only show some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0026] Figure 1 is the overall schematic diagram of the present application. DETAILED DESCRIPTION

[0027] The ammeter image key information recognition and abnormal state diagnosis system of the present application will be described in detail below in combination with the drawings and specific embodiments, but the embodiments should not be understood as limiting the scope of protection of the present application.

[0028] In the ammeter image recognition process, the visible light image is the most intuitive, but it is extremely susceptible to external environmental interference, for example, the reflection of the surface glass, the long-term accumulated dust stains and the change of the light condition, which will all lead to the decrease of the image contrast or the blurring of the key information, thereby affecting the recognition accuracy. The infrared thermal imaging can compensate for this deficiency to a certain extent, by reflecting the heat distribution characteristics generated by the through-flow inside the ammeter, to assist in highlighting the profile of the pointer, the number or the roller region. However, the infrared signal itself has thermal inertia and response lag, and is not sensitive enough to the small changes of the short-time reading, and cannot support the confirmation of the instantaneous metering reading alone. The sound signal is derived from the electromagnetic hum, the gear meshing sound and the roller carry sound in the ammeter running process, which can quickly respond to the reading changes and is more sensitive to the slight fluctuations in a short time, but at the same time, it is extremely susceptible to external noise interference. Based on this principle, the present application proposes a fusion processing of the visible light, infrared and sound signals, uses the infrared to enhance the visible light image to improve the visual quality, then verifies the credibility of the sound through the infrared fluctuation, and finally uses the verified sound signal to check the image recognition reading, forming a step-by-step verification chain from robust to sensitive, so as to ensure the reliability of the ammeter reading and the accuracy of the diagnosis in complex environment.

[0029] To realize the above functions, the overall architecture of the system of the present application is as follows:

[0030] The ammeter image key information recognition and abnormal state diagnosis system described in the embodiments is designed in a hierarchical and modular manner as a whole, to ensure that the synchronous acquisition, step-by-step enhancement, type identification and chain verification of the multi-modal data can be continuously executed, forming a complete processing closed loop from bottom to top, and the overall principle is as shown in Figure 1 .

[0031] The system comprises the following functional modules: a multi-modal data acquisition module, an image preprocessing unit, an infrared guidance enhancement unit, a key information identification module, an infrared fluctuation verification unit, a sound fluctuation verification unit, and an abnormal state diagnosis module.

[0032] The multi-modal data acquisition module is used for multi-dimensional data sensing of the electric meter. The types of signals collected include visible light images, infrared thermal imaging data, and sound signals. The various sensing units are coordinated through a unified time synchronization mechanism to ensure that the different modal data collected have consistent time references within the same meter reading period. After data acquisition, the raw signals are transmitted to the subsequent processing unit.

[0033] The image preprocessing unit receives visible light images and performs grayscale, denoising, and distortion correction on them. The output images are used for subsequent infrared guidance enhancement operations and serve as the visible light base for enhancement input.

[0034] The infrared guidance enhancement unit receives the preprocessed visible light images and synchronously acquired infrared thermal imaging data. It fuses the infrared thermal distribution with the visible light images through a pixel-level alignment method. The contrast and edges are weighted and corrected, and the enhanced electric meter image is output.

[0035] The key information identification module performs phenotype discrimination on the enhanced electric meter image, identifying it as a mechanical pointer meter, an electronic digital meter, or a roller meter. According to the phenotype, it calls the corresponding recognition sub-model to analyze the reading, and outputs the preliminary recognition result of the electric meter reading.

[0036] The infrared fluctuation verification unit analyzes the time series fluctuation characteristics of the infrared thermal imaging data within the current meter reading period. It compares the infrared fluctuation characteristics with the spectral fluctuation characteristics of the sound signal to verify the effectiveness of the sound signal.

[0037] After the infrared fluctuation verification is completed, the sound fluctuation verification unit performs time series division and spectral feature extraction on the sound signal. It compares the fluctuation pattern of the sound signal with the reading fluctuation trend corresponding to the preliminary recognition result to verify the image recognition result.

[0038] The abnormal state diagnosis module integrates the preliminary recognition result, the infrared fluctuation verification result, and the sound fluctuation verification result to perform abnormal state diagnosis. It outputs the final electric meter key information and abnormal state diagnosis conclusion, forming a closed-loop processing flow from data acquisition, information extraction to result output.

[0039] The system described in this embodiment collects and processes visible light images, infrared thermal images and sound signals under a unified time reference through a multi-modal collaborative architecture, uses infrared guidance enhancement to improve image quality, combines phenotype discrimination to achieve adaptive recognition of different types of electric meters, and performs step-by-step verification of reading results through a chain verification mechanism of infrared fluctuation and sound fluctuation, and finally outputs the results in the abnormal state diagnosis module. The overall architecture ensures that the system still has strong recognition stability and diagnostic reliability in the case of meter surface aging, environmental interference or poor collection conditions.

[0040] In the specific implementation, the system is composed of multiple functional modules, each module is independent and works collaboratively in hardware deployment and algorithm implementation, forming a complete solution from collection to diagnosis. The structure and implementation of the multi-modal data collection module, image preprocessing unit, infrared guidance enhancement unit, key information recognition module, infrared fluctuation verification unit, sound fluctuation verification unit and abnormal state diagnosis module are described below to better understand the technical solution of the present application.

[0041] The multi-modal data collection module is used to obtain multi-source information of the meter state during the operation of the electric meter. The collection content includes visible light images, infrared thermal imaging data and sound signals. The module is composed of an industrial camera, an infrared detector and a high-sensitivity microphone, and is synchronously controlled through a unified time reference to ensure the time consistency of different modal data within the same meter reading period.

[0042] In actual application, the collection device is usually installed in a centralized meter box, a building meter room, a public meter wall of a residential area or an industrial power distribution control cabinet. Such locations facilitate unified monitoring of multiple electric meters and have fixed installation conditions, ensuring the stability and reliability of the collection device for long-term operation.

[0043] Among them:

[0044] The visible light image collection industrial camera is fixed in front of or slightly above the electric meter, and is fixed with the meter box frame through a bracket to avoid angle deviation.

[0045] The camera resolution is preferably 1920x1080 pixels, which can clearly capture the meter dial, digital area and pointer area under common ambient light conditions.

[0046] In the case of insufficient light in the meter box, a low-power LED light supplementing device can be provided to ensure the imaging quality.

[0047] The infrared thermal imaging data collection infrared detector can be installed side by side with the camera. The detector wavelength range is preferably 8~14μm, and the frame rate is preferably 30Hz.

[0048] The detector can cover the main thermal radiation area of the meter shell and internal components, thereby forming a thermal distribution sequence during the operation of the meter.

[0049] When installed in a centralized meter box, the infrared detector can be installed through a transparent window or an infrared-transparent material panel to achieve unobstructed collection.

[0050] The sound signal collection uses a high-sensitivity microphone, and a set of sound collection devices is configured for each meter. The microphone can be directly installed on the surface of the meter shell, close to the metering core or the roller mechanism, to ensure that the electromagnetic hum, gear meshing sound, or roller position changing sound generated during the flow-through and metering process of the meter can be fully captured.

[0051] In another preferred mode, the microphone can be pre-installed inside the meter during the production and assembly stage of the meter, and is fixedly integrated with the metering assembly, thereby achieving sound collection capability upon leaving the factory. The sampling rate of sound collection is preferably 44.1 kHz, and the number of sound channels is single.

[0052] The microphone can adopt an electret capacitive structure, and a noise shield can be externally added to reduce environmental noise interference, thereby better capturing mechanical and electromagnetic acoustic signals generated during the operation of the meter.

[0053] The data of the camera, infrared detector, and microphone are synchronized through a unified timestamp mechanism or a bus trigger signal, ensuring one-to-one correspondence of the image frame, infrared frame, and sound segment in the time dimension.

[0054] This mechanism is particularly suitable for multi-meter parallel scenarios such as centralized meter boxes, and ensures the rationality of comparing the multi-modal data of each meter within the same period.

[0055] Through the above design, the multi-modal data collection module in the embodiment can not only adapt to independent monitoring of a single meter, but also can be deployed on a large scale in centralized meter box or power distribution cabinet scenarios, and has good practicality and engineering feasibility.

[0056] The image preprocessing module is used for basic processing of the collected visible light images, so that the subsequent enhancement and recognition steps can be performed under clear and stable image conditions. This unit mainly includes three steps of grayscale processing, noise suppression, and distortion correction.

[0057] First, in the grayscale processing link, the system converts the collected color image into a single-channel grayscale image. Since meter recognition mainly relies on the geometric morphology of scale lines, pointer edges, and digital characters, color information is often affected by environmental light and introduces interference, so grayscale processing can effectively remove irrelevant color factors and highlight structural information and brightness differences.

[0058] Secondly, in the noise suppression link, for the random noise and small particle interference commonly seen in the meter image, the embodiment preferably adopts Gaussian filtering or median filtering method for processing. Gaussian filtering reduces high-frequency noise through weighted neighborhood smoothing, which is suitable for removing sensor noise points in the imaging process; median filtering replaces the current pixel value with the median of the neighborhood pixels, which can effectively suppress salt and pepper noise and has good retention effect on the meter digital edge and pointer contour. In specific application, appropriate filtering method can be selected according to the image noise characteristics, or two filtering methods can be used in stages.

[0059] Thirdly, in the distortion correction link, considering that there may be camera lens distortion or installation angle deviation in the meter acquisition process, the system uses the pre-obtained camera calibration parameters to correct the image. Specifically, the distortion coefficient and intrinsic matrix are obtained through the camera calibration process, the correspondence between the image coordinate system and the real meter plane coordinate system is established, and the image is remapped, so as to eliminate the problems of barrel distortion or pillow distortion. The corrected image can ensure that the geometric proportion of the meter scale and the digital window area conforms to the standard form, avoiding reading errors caused by deformation.

[0060] In summary, the image preprocessing unit provides clear, stable and standardized image input for the subsequent infrared guided enhancement and key information recognition of the meter image through the combined processing method of grayscale, filtering and distortion correction.

[0061] The infrared guided enhancement unit is used to align and enhance the infrared thermal imaging data and the preprocessed visible light image at pixel level, to solve the problem of image quality degradation caused by factors such as dust covering on the meter surface, glass aging yellowing or complex lighting.

[0062] In the image alignment link, first, the edges of the visible light image and the infrared image are extracted respectively. The edge features of the visible light image can be extracted by Canny operator or Sobel operator, and the edge features of the infrared image can be formed by temperature gradient calculation. Then, stable feature points are selected in the two types of edge features, such as the four corners of the meter frame, the reading window boundary or the scale area edge. The system matches these feature points, and maps the infrared image to the visible light image coordinate system through affine transformation, perspective transformation or geometric correction based on least squares method, to realize pixel-level registration of the two images.

[0063] After alignment, the system enters the image enhancement stage. The system first constructs an infrared heat distribution map, taking the temperature gradient of each pixel point as a weight reference, and combining it with the pixel gray value of the visible light image. Specifically, in the area with a large temperature gradient (for example, the vicinity of the pointer, the digital display area, or the current path corresponding area), the system gives a higher edge enhancement weight to improve the profile definition and contrast of these areas; while in the area with a gentle temperature change (for example, the background, the frame or the fixed structure), the system reduces the weight to avoid unnecessary noise enhancement.

[0064] In actual implementation, the weighting correction can be completed by linear fusion or nonlinear mapping. For example, the following method can be used:

[0065] Contrast enhancement: On the basis of the gray value of the visible light image, a gain factor proportional to the infrared temperature gradient is superimposed to enhance the brightness difference of the local area;

[0066] Edge enhancement: A mask is generated using the high gradient area in the infrared image, and a sharpening convolution is performed in the corresponding area of the visible light image to strengthen the visibility of the pointer and digital edge.

[0067] After the above processing, the final meter image not only retains the detailed structure of the visible light image, but also introduces the compensation information of the infrared data, and can still maintain high recognizability in complex environments. The enhanced meter image is transmitted to the key information recognition module for subsequent phenotype discrimination and reading extraction.

[0068] The key information recognition module is used to discriminate the phenotype of the meter image processed by the infrared guided enhancement unit, and according to different phenotypes, the corresponding recognition method is called to output the preliminary recognition result of the meter reading.

[0069] In specific implementation, the key information recognition module can include a phenotype discrimination subunit and a multi-type recognition subunit. The phenotype discrimination subunit is used to extract features and classify the meter image to determine whether the meter belongs to a mechanical pointer meter, an electronic digital meter or a roller type meter. The multi-type recognition subunit uses the corresponding recognition strategy to analyze the reading according to the result of phenotype discrimination.

[0070] In the specific implementation process, the system first analyzes the structure of the entire meter image, extracts feature information such as dial outline, scale line, character line, and roller window. If a ring-shaped scale structure and a pointer-like feature are detected, it is determined to be a mechanical pointer meter; if a rectangular digital display area is detected and there are continuous digital characters, it is determined to be an electronic digital meter; if multiple equal-width rectangular windows are detected and there are scrolling digital bands in the windows, it is determined to be a roller type meter.

[0071] In the process of data reading, any technology in the prior art can be adopted, such as image recognition, OCR or neural network model, etc.

[0072] In the preferred embodiment, the phenotype discrimination can be completed by a lightweight convolutional neural network to improve the accuracy of discrimination and avoid the failure of rule discrimination in complex lighting or contamination scenarios.

[0073] When the phenotype is a mechanical pointer watch, the system performs edge detection and arc positioning in the watch face area to determine the position and scale density of the scale ring. Then, the system detects the pointer direction, preferably uses the Hough line transformation method to extract the elongated straight line feature, locates the pointer main axis, and intercepts the intersection of the pointer end and the scale ring.

[0074] In the reading mapping, the system obtains the corresponding value according to the included angle between the pointer direction and the reference scale, combined with the magnification parameter recorded in the nameplate information. For the case where the pointer is close to the scale boundary, the system can assist in judgment combined with the reading trend at the adjacent time to avoid errors caused by improper up and down.

[0075] In the preferred embodiment, the system can further adopt a multi-frame fusion strategy to vote or average multiple frames of images within the same period to reduce single-frame errors caused by reflection and jitter.

[0076] When the phenotype is an electronic digital watch, the system performs character line positioning in the digital display area to separate the main display line from other information lines. In the recognition process, the system can adopt two ways:

[0077] Splitting recognition mode: each digit is cut by vertical projection or connected domain segmentation method, and then the character recognition sub-model is called to recognize each digit.

[0078] End-to-end recognition mode: the complete digital line is taken as input, and the convolutional neural network and sequence decoding method (such as CTC decoding) are used to directly output the digital sequence.

[0079] In the preferred embodiment, for seven-segment tube display, the system checks the segment on-off combination of each character to determine whether there is a missing segment or artifact. When the recognition result is illegal, the system can correct it combined with the context digit rule and historical data. For example, when 8 is misrecognized as 0 due to missing segment, the system can correct it by detecting the reasonableness of the value jump between the upper and lower time.

[0080] In addition, the system can also recognize the decimal point, unit symbol and magnification mark, and bind them with the digital reading to ensure the integrity of the output result.

[0081] When the phenotype is a rolling wheel type meter, the system performs geometric correction on the multiple rolling wheel windows, ensuring that the digital band maintains a perpendicular relationship with the window boundary. Subsequently, the system extracts the digital region in the window and identifies the numbers within the window through template matching or a lightweight OCR model.

[0082] For the case where the rolling wheel is in the exchange position, i.e., the digital display region contains both upper and lower numbers, the system determines whether the carry is completed by detecting the half-occlusion state. If there is only a single frame of image, the system marks the bit as an uncertain state; if there are consecutive frames, the final value is determined in combination with the change trend of the adjacent frame numbers.

[0083] In a preferred embodiment, the system can perform logical verification on each digit to determine whether the high bit rolling wheel changes synchronously when the low bit rolling wheel jumps, thereby discovering carry anomalies or rolling wheel jamming problems.

[0084] After each phenotype is identified, the system formats the identified reading results uniformly and outputs them as structured numerical values, with the confidence and abnormality marker information of each reading. If the system detects reflection interference, incomplete displacement, character line conflict, or rolling wheel carry anomaly, an abnormality prompt is added to the output results.

[0085] In a preferred embodiment, the phenotype discrimination subunit uses the MobileNetV3-Small network as the classifier to perform three-class classification on the detected reading window, enabling fast phenotype identification on low-power devices.

[0086] In a preferred embodiment, the pointer detection of a mechanical pointer meter uses the Hough line transform combined with the local thinning method to improve the positioning accuracy in the presence of reflection or partial pointer bending; the reading mapping is completed through table lookup to avoid complex calculations.

[0087] In a preferred embodiment, the identification of an electronic digital meter uses an end-to-end OCR model, with a convolution + bidirectional recurrent network structure as the backbone network and CTC as the decoding method, enabling high recognition accuracy in the presence of uneven illumination and character sticking; when the OCR model confidence is lower than the set threshold, the system automatically switches to a segmentation-based identification strategy as a backup.

[0088] In a preferred embodiment, the identification of a rolling wheel type meter uses a perspective correction algorithm based on the window edge, combined with digital template matching for identification; for numbers in the exchange position, the final reading is determined through the change trend of consecutive frames, and an exchange-in-progress identifier is added to the output.

[0089] Infrared thermal imaging can reflect the temperature change process of the electric meter and its terminal, shunt, coil and other parts; sound signals can reflect the acoustic changes caused by electromagnetic hum, gear meshing sound, relay attraction sound and other acoustic changes caused by power-on operation. Both are driven by the same evolution of power consumption, but there is an explainable sequence in time: sound usually responds immediately, and temperature is affected by thermal inertia and shows slow fluctuations. The infrared fluctuation verification unit is based on this physical correlation to match the trend of the infrared time series and the sound spectrum time series in the current meter reading period, to confirm whether the sound signal is from the real operation of the meter, thereby providing reliable sound input for subsequent sound fluctuation verification readings.

[0090] Specifically, the infrared fluctuation verification unit realizes verification through the following steps:

[0091] Time synchronization and data window setting:

[0092] On the basis of the unified time stamp labeling completed by the multi-modal data acquisition module, a time window covering the current meter reading period is selected. The infrared frame and the sound data are sliced according to the unified time axis; when the sampling rates are different, the low sampling rate sequence is time interpolated or the high sampling rate sequence is summarized in whole, to ensure that the infrared and sound characteristic values are available at the same time point.

[0093] Infrared region of interest determination:

[0094] According to the previous meter window / component detection result, one or more regions are defined in the thermal image, preferably including: meter core region, outlet terminal region, shunt region and conductor region near the load side. To suppress environmental impact, a background reference region (located in the inert position of the meter shell which does not generate heat) can be set at the same time, which is used to make a relative temperature baseline or drift offset.

[0095] Infrared preprocessing:

[0096] Bad point replacement and non-uniformity correction are performed on the thermal image; in the scene where there is direct sunlight or strong reflection, the abnormal frame is avoided by cooperating with the light shielding structure or time period screening. The basic trajectory of each infrared region is extracted, including the regional average temperature, regional high quantile temperature and regional internal temperature difference, to reduce the influence of local noise on the curve shape.

[0097] Infrared fluctuation feature extraction:

[0098] The smoothed temperature change curve and the change rate curve are calculated on the basis of the trajectory; key event points such as rapid temperature rise starting point, plateau, slow cooling section and sudden jump point are identified, and an event sequence is formed. The results of multiple regions are normalized and aligned, and the comprehensive infrared fluctuation feature sequence is output, including the trend curve and the event timestamp list.

[0099] Sound sequence preparation:

[0100] The sound data is segmented by short-time window and processed by window function. The spectral energy of each segment is calculated. The frequency bands near the power frequency and its harmonics, the mechanical inherent frequency band of the equipment, and the typical frequency band of the environmental noise are considered simultaneously. The energy envelope of each target frequency band is taken as a candidate acoustic trajectory, and the rising section, peak section, decay section, and steady section are extracted.

[0101] Lag estimation and trend matching:

[0102] The lag estimation is performed between the infrared and sound sequences to find the best alignment of the event sequences. After alignment, the common trend and the same direction or sequence of key events of the infrared curve and the acoustic envelope in the same time sequence are compared. For example: the rising of sound energy should be accompanied by the subsequent rising of infrared temperature events; after the sound enters the steady section, the infrared should enter the temperature platform; after the sound energy decreases significantly, the infrared should appear delayed cooling. According to the consistency of the trend, the matching ratio of events, and the allowed time tolerance, a consistency score is generated, and the reliability of the sound signal is determined based on it.

[0103] Result output and interface:

[0104] The output sound reliability conclusion, estimated time lag, number of matched event pairs, unmatched event list, and regional and frequency band metadata used by this unit. All outputs are used as input conditions for sound fluctuation verification readings, and are used by the abnormal state diagnosis module for comprehensive judgment and tracing.

[0105] Abnormalities and fallbacks:

[0106] When the infrared sequence is approximately stationary throughout the cycle, there is a large area of obstruction or the sensor is saturated, mark the infrared as unavailable, and switch to the weak verification process based on sound; when the sound sequence is covered by a large amount of non-working condition noise, mark the sound as contaminated, and keep the infrared trend for subsequent fusion reference. All abnormalities are recorded for easy maintenance troubleshooting.

[0107] For easy understanding, some typical working condition examples are given:

[0108] Example one: resident user load surge

[0109] At the peak of night electricity, the household turns on multiple appliances. The sound energy rises rapidly near the power frequency and enters the steady state; the infrared temperature rises continuously after a short delay and enters the platform. The sequence of the two in the rising and platform stages is consistent, and the lag is stable within a reasonable range. It is determined that the sound is reliable and can be used as an effective input for subsequent reading fluctuation verification.

[0110] Example two: outdoor environment bird call interference

[0111] The sound sequence appears frequently with high-frequency short pulses, and the energy near the power frequency does not show a stable rise; the overall infrared temperature is stable without significant fluctuations. The event matching ratio is extremely low, and the trend is inconsistent. It is determined that the sound is not reliable, and the subsequent process does not use the sound sequence for reading verification.

[0112] Example Three: Periodic Load in Industrial Field

[0113] A certain production line starts the motor every fixed time. The sound appears with a rise in broadband energy at the corresponding time, and then falls back. After each start, the infrared appears with a delayed temperature rise and a slow fall. Multiple periodic events correspond one by one with stable lag. It is determined that the sound is reliable, and the lag parameter is recorded for subsequent use.

[0114] Example Four: Playback of Fake Sound

[0115] A malicious playback of recorded electromagnetic hum is played to interfere with identification. The sound track shows obvious fluctuations, but the infrared remains approximately constant throughout the cycle without corresponding temperature rise or fall. The two do not match, and it is determined that the sound is not reliable, and a suspicious event record is triggered.

[0116] Example Five: Sunlight Directly Radiates to Cause Infrared Abnormality

[0117] During the day, the sunlight directly radiates to the meter box shell, and the infrared curve rises but the sound does not appear synchronous change. The system detects that the infrared region and the reference region are simultaneously heated and the acceleration is abnormal, identifies it as an environmental thermal disturbance, enters the rollback strategy, and waits for the non-sunlight period to verify again.

[0118] The infrared thermal imaging directly reflects the temperature fluctuations of the conductors and measurement elements inside the electric meter, which are derived from the physical phenomenon of current flow heating and are not disturbed by external human voice, mechanical knocking or environmental noise, so they can be used as reliable reference signals. However, the infrared signal has obvious thermal inertia and response lag, and is not sensitive to short-term and small jumps in meter readings, so if the infrared is directly used to verify the readings, it is often not accurate to correspond to the immediate changes in the readings. Secondly, the sound signal can quickly respond to electromagnetic hum, gear meshing sound or roller carry sound when current passes through, and is more sensitive to subtle reading fluctuations, but sound is easily disturbed by environmental noise, fake sound sources or structure resonance, so if it is not screened and directly used for reading verification, it may produce false judgments. Based on this, the present application selects to verify the sound fluctuation with the infrared fluctuation first to ensure that the collected sound indeed corresponds to the load evolution of the electric meter, and then uses the verified sound signal to check the trend of the reading jump, which avoids the inertia of single infrared verification and the vulnerability of direct sound verification, and ensures the robustness and engineering rationality of the final reading verification.

[0119] The working principle of the sound fluctuation verification unit is to use infrared fluctuation verification to confirm the reliable sound signal, to describe the acoustic energy fluctuation caused by the current flow, transmission and switching action of the electric meter in the meter reading period, and to correspond and check the time and trend of the reading change trend obtained by image recognition. The increase of the electric meter reading is often accompanied by the enhancement of electromagnetic hum near the power frequency and its harmonics, the slight mechanical friction sound of gear or roller, and the transient pulse of relay and other components in state switching; these acoustic phenomena appear before or almost simultaneously with the reading change, and are stably related to the load strength. By comparing whether the sound fluctuation and the reading trend are in the same direction, whether they appear in a reasonable sequence, and whether the key events are one-to-one corresponding in the same meter reading period, the preliminary reading of image recognition can be independently checked, and the misjudgment caused by single image path in the scenes such as reflection, stain or half-step transposition can be avoided.

[0120] In the implementation level, the sound fluctuation verification unit first establishes a stable data buffer in the acquisition side or edge processing node, takes the time window passed by the infrared verification as the boundary, slices the sound sampling stream with fixed length and sets the overlap step. The analysis window length and step are given in the configuration file according to the deployment environment and device type, for example, a twenty to fifty millisecond analysis window is selected and matched with a ten to twenty-five millisecond step to balance the time resolution and frequency resolution. In order to reduce the spectral leakage and window edge effect, the system applies a preset window function to each slice, and the window function type can be switched between Hamming, Blackman or Flat Top window. Slicing, windowing and windowing are performed in a pipeline manner in the ring buffer to ensure real-time performance under the condition of low computing power of edge device.

[0121] After slicing, the system calculates the short-time Fourier transform for each time slice to obtain the discrete frequency distribution of power spectral density, and estimates the noise floor of the spectral line and energy normalization to suppress the influence of device background noise and acquisition link gain fluctuation on energy trajectory. Then, the frequency band energy is extracted according to the basic frequency band and the device characteristic frequency band, wherein the basic frequency band covers the power frequency and the low harmonic neighborhood, and the device characteristic frequency band is set in the medium frequency region according to the phenotype and transmission structure. In addition, a narrow alarm band is set in the higher frequency band to capture the transient state that may be related to electric arc and loose contact. The frequency band division adopts the strategy of combining start-up configuration and online calibration, and the system automatically samples the background spectrum in a small number of calibration periods at the beginning of deployment to generate the initial threshold and weight of each frequency band, and then makes slow drift correction according to the environmental changes during running.

[0122] To enhance robustness, the system extracts several robust descriptors in addition to the band energy, including spectral centroid, spectral bandwidth, spectral peak density, and periodicity indicators from short-time autocorrelation, and performs detrending, outlier rejection, and low-pass smoothing on all feature sequences. Detrending is used to eliminate false trends caused by long-term slow drift, outlier rejection is achieved through median filtering and context consistency check, and smoothing uses a lightweight first or second order recursive scheme to preserve event edges. After this processing, the system obtains multi-channel sound fluctuation feature sequences, with timestamps, window coverage, and quality labels attached to each channel, ensuring sufficient meta-information for the subsequent comparison stage.

[0123] The image recognition path outputs preliminary readings and corresponding change events, including the time positions of increasing segments, plateau segments, and small jumps, within the same time window. The sound fluctuation verification unit generates an event trajectory of reading trends accordingly, and inputs key nodes of reading occurrences in the form of time slice index and duration to the comparison engine. The comparison engine first performs coarse alignment by sliding the sound feature time axis within a reasonable range to find the best relative position of sound events and reading events. After coarse alignment, the system performs fine alignment within a local window, focusing on whether there is sound energy rise and steady-state maintenance before and after the reading increment, and whether the sound stabilizes or falls after the reading enters the plateau. This alignment process allows small time flexibility to cover response differences caused by different meter thermal inertia, transmission delay, and installation structure.

[0124] The consistency score is composed of three parts: trend consistency, key event one-to-one correspondence, and duration matching. Trend consistency measures the directional consistency between the rise, maintenance, and fall of the sound energy trajectory and the increment, plateau, and slow change of the reading; key event correspondence measures whether a sound event matching the reading event can be found within the window and their relative order; and duration matching measures the closeness of the two types of events in time span. The time lag prior obtained in the infrared verification stage is introduced in the scoring process to constrain the candidate set of sound events, avoiding false consistency caused by meaningless cross-window pairing. After the final score is compared with the threshold, the pass, boundary, or fail three types of conclusions are obtained, and the time alignment parameters, the number of matched event pairs, the unmatched event list, and the frequency band and window length relied on for this judgment are output as explanatory metadata.

[0125] To cope with complex noise and working condition changes, the system sets up multiple layers of fallback strategies. When the sound features are dominated by human voice or environmental pulses in most time slices, first try to switch to a narrower device feature frequency band and a stronger spectral subtraction threshold; if it still cannot restore the stable trajectory, the conclusion is indeterminate, and the weight of the sound in the subsequent fusion is reduced. When the readings change more subtly and the duration is very short, the system will shorten the analysis window and increase the overlap ratio to improve the time resolution; if shortening the window causes insufficient frequency domain resolution, a high-frequency warning band is introduced as a compensation channel to capture fast pulse events. When the image path is labeled as half-step position change uncertainty, the comparison engine searches for device transient adjacent to low-bit changes during the detailed alignment stage to improve the sensitivity of the decision.

[0126] This implementation has many advantages. First, the prior infrared credibility screening eliminates extraneous noise unrelated to the load, so that sound only enters the comparison process when it is physically related to the meter working condition, significantly reducing the probability of false positives. Second, the dual representation of band energy and events covers both continuous load evolution and transient switching, allowing tracking of long-term usage trends and capturing short-term position changes and attraction. Third, the time alignment uses coarse and fine two-level search and small range flexibility, preferably within ±5%, so that the system does not require millisecond-level hard synchronization to obtain stable decisions under diverse installation and environmental conditions. Fourth, all intermediate results are retained as traceable metadata, facilitating manual review and problem positioning during the operation and maintenance phase.

[0127] In the preferred implementation, the analysis window can use a parameter combination of thirty-two milliseconds and a sixteen-millisecond step to cover most residential and commercial scenarios; the window function is preferably a Hamming window to balance sidelobe suppression and main lobe width, and industrial strong noise scenarios can switch to a Blackman window to further suppress spectral leakage. The frequency band configuration preferably uses a three-layer structure, with the bottom layer being a narrow band for power frequency and low-order harmonics, the middle layer setting two to three medium frequency bands according to the transmission mechanism of the device type, and the high layer setting one to two narrow warning bands to capture arcs and poor contact. The noise suppression preferably uses a combination of spectral subtraction and adaptive threshold, first using a slowly updated background spectrum as a baseline, and then dynamically adjusting the threshold based on short-term energy quantiles to ensure that usable features are maintained in the presence of sudden environmental noise. Time alignment preferably first estimates coarsely using event timestamps, and then refines within a local range of no more than a few analysis window lengths to prevent global mismatch.

[0128] In a specific example, when residents turn on multiple electrical appliances in the evening, the sound appears to continuously rise and stabilize in the narrowband of power frequency and the intermediate frequency band of the device. The readings continue to increase and enter a plateau in the subsequent period of time. After coarse alignment, the comparison engine gives a small positive alignment amount. Fine alignment confirms that the rising and plateau events are aligned completely. The final determination is that it passes. When the outdoor meter box is disturbed by human voices and bird calls, the high-frequency components of the sound fluctuate significantly, but the power frequency and device band lack sustained enhancement. Infrared does not see corresponding temperature rise in the verification window. There is no reasonable alignment amount in the coarse alignment stage. The system determines that it does not pass and produces a pollution mark. In the half-step transposition scene of the roller meter, after a short mechanical pulse appears in the device band, the readings in the low position show a one-step jump, while the high position remains unchanged. Fine alignment can locate the pulse in the adjacent window and complete event pairing. The system gives a pass conclusion and provides evidence that the transposition has been completed for subsequent abnormal diagnosis. In the industrial cycle load scenario, the periodic peak value of sound energy appears due to the start of the motor according to the beat. Infrared shows delayed temperature rise after each peak. The readings increase slowly according to the beat. The coarse and fine alignment remains stable and consistent in multiple cycles. The system outputs a high-confidence matching result and stable alignment parameters.

[0129] The abnormal state diagnosis module is based on the chain logic of the preliminary identification result as the candidate reading, the infrared fluctuation verification as the physical reference, and the sound fluctuation verification as the rapid response evidence. It establishes a comprehensive judgment mechanism for the same meter reading cycle. Its working principle is as follows: the image path gives the reading and bit-level confidence information, but it may be affected by factors such as reflection, stains, and half-step transposition. The infrared path directly reflects the temperature rise evolution after conduction, has high credibility but has thermal inertia. The sound path responds quickly to electromagnetic and mechanical behavior, can capture short-term changes, but is easily disturbed by environmental noise. The diagnosis module generates the final conclusion on the operation state of the meter through joint analysis of the consistency, sequence, and event correspondence of the three. It also forms a traceable reason and basis.

[0130] In the process of implementation, the module first receives the candidate reading and its bit-level confidence, the half-step displacement flag, the decimal point and the multiplier identification and other structured information output by the key information identification module, receives the conclusion and the estimated time lag parameter of the infrared fluctuation verification unit about the sound credibility, and receives the conclusion and the matching details of the sound fluctuation verification unit about the sound-reading trend consistency. The module establishes a judgment session with the meter reading period as the time boundary, and works in the order of data integrity check-consistency check-conflict attribution-conclusion generation-recording and output. The data integrity check stage confirms whether the time coverage of the three paths meets the minimum requirement, if a path is missing in the key period, the path is marked as unusable and its influence weight in the subsequent fusion is reduced. The consistency check stage aligns the sound event and the reading event based on the time lag of the infrared output, and focuses on checking whether there is energy climbing and steady state maintenance of sound near the reading increment segment, and whether there is corresponding stability or decay of sound in the reading platform or slow decline stage. The conflict attribution stage, when there is inconsistency, the diagnosis module does not directly give an exception, but locates the cause combined with the quality mark of each path and the scene metadata, such as the overall lifting of infrared in the suspicious sunlight period, the pollution of high-frequency sound by human voice or bird calls, the existence of strong reflection in the image path or the incomplete half-step displacement of the scroll wheel, etc. The conclusion generation stage adopts a grading strategy: when the image candidate reading is consistent with the sound and the sound has been verified as credible by infrared, output normal and confirm the candidate reading as the final reading; when any path is judged as inconsistent and the reason can be attributed to the device or reading behavior (such as scroll wheel jam, pseudo reading caused by seven segment segment, pointer stagnation), output exception and mark the exception type and severity level according to the attribution; when the conflict mainly comes from the collection conditions or environmental interference and lacks sufficient evidence to support the exception, output undetermined / need to review, and give re-collection suggestions and the need for improvement of the sensing configuration. In the recording and output stage, in addition to the final reading and the state, the key event pairing, time alignment parameter, unusable path reason, exception type code and recommended treatment action supporting the conclusion are also output synchronously, and all information enters the log for traceability.

[0131] The advantage of this comprehensive judgment path is to constrain fast evidence with physical credible chain, and then to check reading candidates susceptible to appearance with fast evidence, thereby avoiding both the lag error brought by only looking at infrared and the environmental misjudgment brought by only listening to sound; at the same time, the module sets weight reduction and fallback for collection missing and environmental abnormalities, ensuring that executable conclusions can still be given stably in edge computing and complex scenes; in addition, the conclusion is accompanied by explanatory metadata output, which facilitates the value guard personnel to quickly locate the problem source and avoid the situation that black box judgment is difficult to review.

[0132] In a preferred implementation, the conclusion fusion can employ a hierarchical decision tree: the first layer judges the credibility of infrared to sound, if not passed, directly shield the sound path and only weak consistency check for image and infrared; the second layer under the premise of sound credibility, with sound-reading consistency as the main criterion, image bit-level confidence and half-step displacement marker as plus-minus score factors, form three grades of conclusions: pass, boundary and fail, the boundary grade enters the artificial review or secondary collection process. Another preferred way is to set a configurable weight table, dynamically adjust the influence of the three paths according to the scene (residential, commercial, industrial), installation form (single meter, centralized box) and time period (day / night, suspicious sunlight period), for example, reduce the dominant proportion of infrared in outdoor environment during the day, and increase the threshold of infrared and the prior requirement for sound in strong noise industrial site. It can also configure an abnormal type dictionary and a trigger condition set, when it is detected that the sound is reliable and the reading is long-term static while the infrared is slowly rising, it is attributed to pointer / roller jamming suspicion, when the reading mutates and both sound and infrared have no corresponding changes, it is attributed to reading recognition suspicion / may be affected by reflection or film, when high-frequency alarm band pulses are dense and infrared temperature rises with abnormal reading fluctuations, it is attributed to contact failure / arc suspicion, and different disposal suggestions and alarm levels are generated accordingly.

[0133] In a specific example, when the resident's evening electricity consumption rises, the sound is stably enhanced at the power frequency and equipment band and verified as reliable by infrared, the image reading then smoothly increases and enters the platform, the diagnostic module confirms that the three are in reasonable order and the event pairing is sufficient in consistency check, outputs the final reading and marks the state as normal. Another example is that an outdoor meter box is affected by sunlight at noon, the infrared curve rises as a whole and the reference region rises in the same direction, the sound path fails to pass the infrared credibility screening, the image reading is relatively stable; the diagnostic module marks the infrared as environmental thermal disturbance, reduces the weight of sound, outputs undetermined / needs review, and suggests secondary collection during non-sunlight period. For example, in the half-step displacement scene of the roller-type meter, the image path has insufficient confidence in the lowest bit recognition and gives a half-step marker, the sound path detects a short pulse at the equipment band and matches the time of the reading low bit jump, and the infrared slightly rises subsequently, the diagnostic module confirms the candidate reading as valid and outputs normal, while recording the side evidence that the low bit displacement has been completed in the explanation information. For example, a seven-segment digital meter has a missing segment that causes 8 to be mistaken for 0, the sound path and the infrared both show that the load is in the high position and maintains a steady state, the reading drops sharply and conflicts with the two physical evidences, the diagnostic module determines that the reading recognition is suspicious, outputs an exception and suggests local re-sampling and manual review for the display bit. The above implementation ensures that in various environments and various phenotypes, the system can give a final conclusion with physical basis, traceability and executability.

[0134] In another embodiment, the application also provides an electric meter image key information recognition and abnormal state diagnosis method, which uses any of the foregoing systems or combinations to recognize electric meter image key information and diagnose abnormal states.

[0135] Those skilled in the art can appreciate that the units and algorithm steps described in the embodiments disclosed herein can be realized by electronic hardware, computer software and a combination of electronic hardware and computer software. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0136] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0137] In several embodiments provided in the present application, any function realized in the form of a software function unit and sold or used as an independent product can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0138] The above is merely specific embodiments of the present application, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. The parts of the present application not specifically mentioned should be subject to the contents recorded in the prior art. The prior art mentioned in the foregoing background section and the specific embodiments section of the present application can be used as a part of the present application to understand the meanings of some technical features or parameters.

Claims

1. An electric meter image key information recognition and abnormal state diagnosis system, characterized by, The utility model relates to a kind of multi-modal meter reading method and system, including: Multi-modal data acquisition module, for synchronously obtaining visible light image, infrared thermal imaging data and sound signal of electric meter; Image preprocessing unit, for carrying out gray scale, denoising and distortion correction to the visible light image; Infrared guided enhancement unit, for carrying out pixel-level alignment of the infrared thermal imaging data and preprocessed visible light image, and based on infrared thermal distribution, the contrast and edge of the visible light image are weightedly corrected, and enhanced electric meter image is obtained; Key information identification module, for carrying out phenotype discrimination to the enhanced electric meter image, and according to phenotype, corresponding recognition model is selected, and preliminary identification result of electric meter reading is extracted; Infrared fluctuation verification unit, for carrying out time series analysis on the infrared thermal imaging data collected in current meter reading period, extracting temperature mean value change curve, temperature gradient change curve and infrared fluctuation characteristic sequence formed therefrom, and corresponding comparison of the infrared fluctuation characteristic sequence and frequency spectrum fluctuation feature extracted based on sound signal short-time frequency spectrum energy distribution in the same meter reading period is carried out, to verify whether sound signal is consistent with actual running state of electric meter; Sound fluctuation verification unit, for comparing time series frequency spectrum fluctuation mode of the sound signal with reading fluctuation corresponding to preliminary identification result after comparison of the infrared fluctuation verification unit is completed; Abnormal state diagnosis module, for combining preliminary identification result, comparison result of the infrared fluctuation verification unit and the sound fluctuation verification unit, and outputting key information and abnormal state diagnosis conclusion of electric meter.

2. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The image preprocessing unit includes three steps of gray scale processing, noise suppression and distortion correction, the noise suppression adopts Gaussian filtering or median filtering mode, and the distortion correction remaps the image using camera calibration parameters to eliminate barrel or pillow distortion.

3. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The infrared guided enhancement unit adopts feature point registration mode for pixel-level alignment, selects electric meter outer frame four corners, digital window edge or scale area edge as registration feature points, and realizes mapping of infrared image and visible light image through affine or perspective transformation.

4. The electric meter image key information recognition and abnormal state diagnosis system according to claim 3, characterized by: After completing pixel-level alignment, weight map is generated based on infrared temperature gradient, higher edge enhancement weight is given to the area with high temperature gradient in visible light image, and the contrast of pointer, digital or scale area is improved.

5. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The key information identification module includes phenotype discrimination subunit and multi-type identification subunit, the phenotype discrimination subunit is used to judge that electric meter is mechanical pointer meter, electronic digital meter or roller type meter, and the multi-type identification subunit calls different identification strategies according to phenotype to output reading.

6. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The infrared fluctuation verification unit extracts regional average temperature, regional high-partition temperature and regional internal temperature difference when analyzing infrared time series, and calculates lag relationship of infrared curve and sound energy envelope in trend comparison, to confirm the credibility of sound signal.

7. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The sound fluctuation verification unit extracts sound frequency spectrum energy distribution through short-time Fourier transform, and calculates energy envelope on preset target frequency band, further matches sound fluctuation feature with reading change trend corresponding to image recognition result.

8. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The sound fluctuation verification unit allows small range time flexibility in comparison process, performs coarse alignment and refinement alignment on sound events and reading events, and generates consistency score according to trend homogeneity, key event correspondence and duration matching degree.

9. The electric meter image key information recognition and abnormal state diagnosis system according to claim 1, characterized by: The abnormal state diagnosis module performs step-by-step analysis on data integrity, consistency and conflict attribution in fusion judgment, and outputs normal, abnormal or undeterminable state conclusion according to the judgment result, while accompanying with explanatory metadata.

10. An electric meter image key information recognition and abnormal state diagnosis method, characterized by, The electric meter image key information recognition and abnormal state diagnosis system of any one of claims 1-9 is used for electric meter image key information recognition and abnormal state diagnosis.

Images