Electric energy meter screen damage detection method and device

Abstract

The invention is suitable for the technical field of screen detection, and provides an electric energy meter screen damage detection method and device, and the method comprises the steps: obtaining the appearance images of an electric energy meter screen in a power-on state and a power-off state, extracting the features of the appearance images, calculating the correlation between the features of the appearance images and a protection layer, a display layer and a backlight layer, and determining the damage of the display layer. Marking the damage condition of the electric energy meter screen on the appearance image to form a marked image set; the annotation image set comprises the correlation between the damage condition and the protection layer, the display layer and the backlight layer; training a deep learning model based on the annotated image set; and inputting the real-time appearance image of the electric energy meter screen into the trained deep learning model, and analyzing whether the electric energy meter screen is damaged and the position of a damaged layer. The method can be integrated in the existing equipment and software, solves the problems of low compatibility of appearance characteristics and screen changes of a new generation of electric energy meters to the existing equipment and software and low manual detection efficiency, reduces manual detection errors, and improves the efficiency.

Description

Technical Field

[0001] The present application belongs to the field of screen detection technology, and in particular, relates to a method and device for detecting damage to an electric energy meter screen. Background Art

[0002] With the popularity of smart IoT meters, the appearance and display technology of the new generation of electricity meters have changed. The most obvious change is the change from the traditional seven-segment code character display to the dot matrix LCD screen. Therefore, in order to adapt to this change, it is necessary to upgrade the appearance inspection recognition software and add automatic recognition templates to meet the needs of appearance inspection of 13th, 20th and IR46 standard IoT meters. In addition, it is also necessary to replace the appearance inspection industrial computer and reconfigure the software to optimize and upgrade the appearance inspection software to adapt to the appearance characteristics of the new generation of electricity meters.

[0003] The screen of the new generation of energy meters is different from that of the existing energy meters. The LCD screen has the characteristics of larger size, higher resolution and lower power consumption, while the screen of the new generation of energy meters is a dot matrix LCD screen with reduced size and the characteristics of high brightness, high contrast and bright colors, such as Figure 1 Therefore, in the process of building the verification capability of the new generation of smart energy meter automated production lines, it is necessary to carry out technical optimization and upgrade services for the assembly line system, while reducing the difficulty of manual inspection to improve the reliability and safety of the entire assembly line inspection system.

[0004] In this context, it is necessary to solve the problems of compatibility of the appearance characteristics and screen changes of the new generation of electricity meters with existing equipment and software, and low efficiency and accuracy of manual detection, so as to meet the needs of production and calibration of the new generation of electricity meters. Summary of the invention

[0005] The embodiments of the present application provide a method and device for detecting damage to an electricity meter screen, which can be integrated into existing equipment and software, solve the problems of compatibility of the appearance characteristics and screen changes of a new generation of electricity meters with existing equipment and software and low efficiency of manual detection, reduce errors in manual detection and improve efficiency.

[0006] This application is implemented through the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for detecting damage to an electric energy meter screen, comprising:

[0008] Obtain appearance images of the electric energy meter screen in powered and unpowered states;

[0009] The features of the appearance image are extracted by an image processing algorithm, the correlations between the features of the appearance image and the protective layer, the display layer and the backlight layer are calculated, and the damage conditions of the electric energy meter screen are marked on the appearance image to form a marked image set; the marked image set includes the correlations between the damage conditions and the protective layer, the display layer and the backlight layer;

[0010] Based on the annotated image set, the pre-built deep learning model is trained to obtain a trained deep learning model;

[0011] The real-time appearance image of the electricity meter screen is input into the trained deep learning model to analyze whether the electricity meter screen is damaged and the location of the damaged layer.

[0012] In a possible implementation of the first aspect, extracting features of the appearance image by using an image processing algorithm includes:

[0013] Using edge detection to locate the screen position in the appearance image of the electric energy meter screen;

[0014] Based on the screen position, determining the screen image;

[0015] Extract the brightness features, color features, image content and texture features of the screen image.

[0016] In a possible implementation manner of the first aspect, the annotated image set includes a first annotated image set and a second annotated image set;

[0017] The damage condition of the electric energy meter screen is marked on the appearance image to form a marked image set, including:

[0018] The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-on state to form a first marked image set; the damage conditions of the electric energy meter screen in the first marked image set include dark screen, white screen, flowery screen and black screen;

[0019] The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-off state to form a second marked image set; the damage conditions of the electric energy meter screen in the second marked image set include yellowing, white spots, scratches and cracks.

[0020] In a possible implementation of the first aspect, the pre-built deep learning model includes an SDD model.

[0021] In a possible implementation of the first aspect, a real-time appearance image of the electric energy meter screen is input into a trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer, including:

[0022] The real-time appearance image of the electric energy meter screen is input into the trained deep learning model to obtain the damage condition of the real-time appearance image of the electric energy meter screen;

[0023] Based on the damage situation, the damage results of the electric energy meter screen are output; the damage results include damage, non-damage, damage type and the location of the damage layer.

[0024] In a possible implementation of the first aspect, locating a screen position in an appearance image of an electric energy meter screen by using edge detection includes:

[0025] Preprocessing the appearance image of the electric energy meter to obtain a preprocessed appearance image of the electric energy meter;

[0026] Based on the preprocessed appearance image of the electric energy meter, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen.

[0027] In a possible implementation of the first aspect, based on the preprocessed appearance image of the electric energy meter, locating the screen position in the appearance image of the electric energy meter screen by using edge detection includes:

[0028] A two-dimensional Gaussian template is used to perform smoothing and filtering on the pre-processed appearance image data of the electric energy meter;

[0029] Performing edge detection processing on the appearance image of the electric energy meter after smoothing and filtering processing to obtain an edge response image;

[0030] Distinguish the contour points and non-contour points of the edge response image to obtain contour lines;

[0031] Based on the contour line, the screen position in the appearance image of the electric energy meter screen is determined.

[0032] In a possible implementation of the first aspect, determining a screen position in an appearance image of an electric energy meter screen based on a contour line includes:

[0033] Based on the contour line, extract the edge line segment of the screen position;

[0034] Calculate the area of ​​the screen based on the edge segments;

[0035] Based on the area of ​​the screen and an area threshold of a preset screen specification, a screen position in the appearance image of the electric energy meter screen is determined.

[0036] In a possible implementation manner of the first aspect, acquiring appearance images of a screen of an electric energy meter in a powered-on state and a powered-off state includes:

[0037] Acquire appearance images of the electric energy meter in powered and unpowered states;

[0038] The appearance image of the electric energy meter is processed in blocks to obtain the serial number of the electric energy meter and the unrelated appearance images of the screen of the electric energy meter in the power-on and power-off states;

[0039] The serial number of the electric energy meter and the unassociated appearance images of the electric energy meter screen in the powered and unpowered states are associated to obtain the appearance images of the electric energy meter screen in the powered and unpowered states.

[0040] In a second aspect, an embodiment of the present application provides an electric energy meter screen damage detection device, which executes the electric energy meter screen damage detection method as in the first aspect, including:

[0041] An image acquisition module is used to acquire the appearance images of the electric energy meter screen in the powered and unpowered states;

[0042] A feature extraction module is used to extract features of the appearance image through an image processing algorithm, calculate the correlation between the features of the appearance image and the protective layer, the display layer and the backlight layer, and mark the damage of the electric energy meter screen on the appearance image to form a marked image set; the marked image set includes the correlation between the damage and the protective layer, the display layer and the backlight layer;

[0043] A model training module is used to train a pre-built deep learning model based on a labeled image set to obtain a trained deep learning model;

[0044] The result output module is used to input the real-time appearance image of the electric energy meter screen into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer.

[0045] Compared with the related art, the embodiments of the present application have the following beneficial effects:

[0046] The method and device for detecting damage to the electric energy meter screen in the embodiment of the present application, by acquiring the appearance images of the electric energy meter screen in the power-on and power-off states, helps to collect data in different states, and provides rich samples for subsequent model training. Extracting the features of the appearance image and marking the damage of the electric energy meter screen through the image processing algorithm helps to establish a detailed data set, which also includes the correlation between the damage and the protective layer, the display layer and the backlight layer, so that the location of the damage can be trained and identified, and the damage of the electric energy meter screen can be identified more accurately and efficiently, thereby improving the accuracy and efficiency of identification. The real-time appearance image is input into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer, thereby realizing the automatic monitoring and analysis of the state of the electric energy meter screen, which is conducive to the fixed-point detection of the subsequent repair staff, enhancing the automation and intelligence level of the system, solving the compatibility of the appearance characteristics and screen changes of the new generation of electric energy meters with existing equipment and software and the low efficiency of manual detection, reducing the error of manual detection and improving efficiency.

[0047] The beneficial effects of the above-mentioned second aspect embodiment refer to the beneficial effects of the first aspect embodiment, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 It is an advanced schematic diagram of an electric energy meter provided by an embodiment of the present application;

[0050] Figure 2 It is a flowchart of a method for detecting damage to an electric energy meter screen provided in one embodiment of the present application;

[0051] Figure 3 is a schematic diagram of obtaining an appearance image of an electric energy meter provided by an embodiment of the present application;

[0052] Figure 4 This is an example of a situation where the screen of an electric energy meter is partially damaged according to an embodiment of the present application;

[0053] Figure 5 It is a structural schematic diagram of an electric energy meter screen damage detection device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0056] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0057] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0058] References to "an embodiment", "one embodiment" or "some embodiments" described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0059] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] Figure 2 is a flow chart of a method for detecting damage to an electric energy meter screen provided in an embodiment of the present application, with reference to Figure 2 The electric energy meter screen damage detection method comprises:

[0061] Step 101, obtaining appearance images of the electric energy meter screen in the powered-on and powered-off states.

[0062] A high-definition camera and a set of auto-zoom lenses can be used. The high-definition camera is used to capture the appearance of the electric energy meter, and the auto-zoom lens automatically adjusts the focal length according to the size and distance of the subject to ensure that the captured image is clear and distortion-free. In addition, an adjustable LED background light source can be set to provide a uniform, non-reflective background light when shooting to improve image contrast and clarity.

[0063] For example, since the screen of the electric energy meter is on when powered on and off when powered off, the screen brightness is different when the screen is on and when it is off. When the screen is off, it is easy to detect damage such as scratches, but it is not easy to detect damage such as dark screen, white screen, flowery screen and black screen caused by partial damage to the liquid crystal pixel matrix. Therefore, when obtaining the appearance image of the electric energy meter screen, at least two images including the powered on state and the unpowered state must be obtained. Step 101 includes:

[0064] Get images of the appearance of the energy meter in powered and unpowered states.

[0065] The appearance image of the electric energy meter is processed in blocks to obtain the serial number of the electric energy meter and the unrelated appearance images of the electric energy meter screen in the power-on and power-off states. Figure 3 In the dashed box, the appearance image of the electric energy meter can be the entire image directly including the electric energy meter screen and the serial number. After the entire image is divided into regions, it is processed into blocks, and the regular image of the electric energy meter screen is retained to obtain the appearance image of the serial number and the electric energy meter screen that are not associated. The appearance image of the electric energy meter can be the image of the electric energy meter screen and the image of the serial number that are separated. After the block processing, the appearance image of the serial number and the electric energy meter screen that are not associated is obtained. Among them, the serial number includes a digital sequence, a barcode or a QR code, etc., which indicates the unique identity of the electric energy meter.

[0066] The serial number of the electric energy meter and the unassociated appearance images of the electric energy meter screen in the powered and unpowered states are associated to obtain the appearance images of the electric energy meter screen in the powered and unpowered states. After association, various data of the electric energy meter, such as the subsequent identified damage to the screen of the electric energy meter, can be uploaded to the cloud platform or other data terminals in real time for remote monitoring and management by staff.

[0067] Step 102, extracting the features of the appearance image through an image processing algorithm, calculating the correlation between the features of the appearance image and the protective layer, the display layer and the backlight layer, and marking the damage of the electric energy meter screen on the appearance image to form a marked image set.

[0068] Exemplarily, in step 102, extracting features of the appearance image by an image processing algorithm includes:

[0069] Step 1021: Use edge detection to locate the screen position in the appearance image of the electric energy meter screen.

[0070] Exemplarily, step 1021 includes:

[0071] The appearance image of the electric energy meter is preprocessed to obtain a preprocessed appearance image of the electric energy meter. The captured image is preprocessed, including grayscale, denoising, binarization and other steps to improve the contrast and clarity of the image.

[0072] Based on the preprocessed appearance image of the electric energy meter, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen.

[0073] Exemplarily, based on the preprocessed appearance image of the electric energy meter, using edge detection to locate the screen position in the appearance image of the electric energy meter screen includes:

[0074] A two-dimensional Gaussian template is used to perform smoothing filtering on the data of the preprocessed appearance image of the electric energy meter; edge detection is performed on the appearance image of the electric energy meter after smoothing filtering to obtain an edge response image; contour points and non-contour points of the edge response image are distinguished to obtain contour lines; based on the contour lines, the screen position in the appearance image of the electric energy meter screen is determined.

[0075] Exemplarily, if the outline of the electric energy meter housing and the outline of the electric energy meter screen are identified, the outline of the electric energy meter screen can be screened out according to the position relationship or area size.

[0076] Based on the contour line, the screen position in the appearance image of the electric energy meter screen is determined, including:

[0077] Based on the contour line, the edge line segments of the screen position are extracted; based on the edge line segments, the area of ​​the screen is calculated; based on the area of ​​the screen and the area threshold of the preset screen specification, the screen position in the appearance image of the electric energy meter screen is determined.

[0078] The area threshold of the preset screen specification can determine an area range through the specifications of the electric energy meter corresponding to the serial number. Areas exceeding or less than the preset screen specification threshold are not the screen positions of the electric energy meter. If the specifications of the calibrated electric energy meter are changed on the production line, the area threshold of the preset screen specification can also be updated in time through serial number recognition, which is compatible with the screen detection process for recycled old electric energy meters.

[0079] If the area of ​​the screen does not meet the area threshold, the repositioning can be assisted by the shape feature or external structure of the electric energy meter screen. For example, the shape feature of the electric energy meter can be a chamfer feature, and the external structure of the electric energy meter can be an icon or a marker near the screen.

[0080] Step 1022, determine the screen image based on the screen position.

[0081] Exemplarily, the interior of the circled screen position can be identified as the screen image.

[0082] Step 1023, extracting brightness features, color features, image content and texture features of the screen image.

[0083] Exemplarily, since the normal characteristics of the screen image in the powered and unpowered states are different, the brightness characteristics, color characteristics, image content and texture characteristics of the screen image in the powered and unpowered states are respectively extracted to obtain a feature matrix for each characteristic arranged according to the liquid crystal pixel matrix of the screen.

[0084] Among them, the OpenCV library in Python can be used to calculate the average brightness value of the image or the histogram equalization method to extract brightness features. The color information in the image can be extracted using color space conversion technology (such as RGB, HSV, Lab, etc.), and the distribution ratio and average hue of each color in the image can be calculated, or a clustering algorithm can be used to extract the main color features. Image recognition and target detection technology, such as convolutional neural network (CNN), can be used to analyze and extract image content. Texture analysis methods, such as gray-level co-occurrence matrix, can be used to extract texture features in the image to represent the details and texture information of the image.

[0085] Exemplarily, the electric energy meter screen includes a protective layer, a display layer and a backlight layer. The protective layer is usually located at the outermost layer of the screen, and is used to protect the internal display layer and backlight layer from physical damage. Its damage is mainly manifested as scratches, cracks or peeling; the display layer is the core part of the screen, responsible for displaying images and text, and its damage may be manifested as pixel necrosis, display abnormality (such as color distortion, uneven brightness), etc.; the backlight layer provides light source for the display layer, enabling it to display images and text, and its damage may be manifested as uneven brightness, backlight leakage or no backlight at all.

[0086] In step 102, the correlation between the features of the appearance image and the protection layer, the display layer and the backlight layer is calculated, including:

[0087] The characteristic matrix of each feature arranged according to the liquid crystal pixel matrix of the screen in the powered and unpowered states is correlated with the standard characteristic matrices of various types of damage to the protective layer, display layer and backlight layer. The lower the correlation, the smaller the possibility of damage to the layer at this position, and the higher the correlation, the greater the possibility of damage to the layer at this position.

[0088] In step 102, the annotated image set includes the correlation between the damage condition and the protective layer, the display layer and the backlight layer. The annotated image set includes a first annotated image set and a second annotated image set. The damage condition of the electric energy meter screen is annotated on the appearance image to form an annotated image set, including:

[0089] The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-on state to form a first marked image set; the damage conditions of the electric energy meter screen in the first marked image set include dark screen, white screen, flowery screen and black screen. The damage conditions such as dark screen, white screen, flowery screen and black screen have a certain correlation with the protection layer, display layer and backlight layer.

[0090] The damage of the electric energy meter screen is marked on the appearance image of the electric energy meter screen in the power-off state to form a second marked image set; the damage of the electric energy meter screen in the second marked image set includes yellowing, white spots, scratches and cracks. The yellowing, white spots, scratches and cracks have a certain correlation with the protective layer, the display layer and the backlight layer.

[0091] The first annotated image set includes the damage conditions of different electric energy meter screens in the powered state and the corresponding brightness features, color features, image content and texture features. The second annotated image set includes the damage conditions of different electric energy meter screens in the unpowered state and the corresponding brightness features, color features, image content and texture features.

[0092] Step 103: Based on the annotated image set, a pre-built deep learning model is trained to obtain a trained deep learning model.

[0093] Exemplarily, the pre-built deep learning model includes an SDD model. The SDD model uses multi-scale features for detection, which can divide the screen into more units and distinguish small damaged targets of the liquid crystal pixel matrix.

[0094] The labeled image set is set to a training set, a validation set, and a test set with set proportions, such as Figure 2 As shown in the figure, according to the set sample ratio, the SSD model will automatically generate text files in formats corresponding to the training set, validation set, and test set according to the set ratio. The content recorded in the text may include: the name of the picture, the location of the marked real box, and the damage condition.

[0095] Exemplarily, the training set is input into the SSD model for detection training, and a training target feature map is output; the training target feature map is marked with a priori box. During the model training process, the training loss is calculated based on the difference between the priori box and the true box, and the SSD model is updated based on the training loss.

[0096] The number of prior frames for each feature point in each feature layer is pre-set, and the size of the prior frames is different in each feature layer. After the feature layer is processed, the prior frames of each feature layer are compared with the real frames, and the one with the largest IOU value is selected. Finally, the model convergence is determined based on the training loss and the set number of iterations to obtain the trained SSD model.

[0097] Step 104: input the real-time appearance image of the electric energy meter screen into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer.

[0098] The real-time appearance image of the electric energy meter screen must include the real-time appearance image of the electric energy meter screen when powered on and the real-time appearance image of the electric energy meter screen when not powered on.

[0099] Exemplarily, step 104 includes:

[0100] The real-time appearance image of the electric energy meter screen is input into the trained deep learning model to obtain the damage of the real-time appearance image of the electric energy meter screen. Based on the damage, the damage result of the electric energy meter screen is output; the damage result includes damage, non-damage, damage type and location of the damage layer.

[0101] For example, Figure 4 An example of partial damage to the screen of an electric energy meter is given. At least one of the above damage conditions can be detected simultaneously on the screen of the same electric energy meter.

[0102] Among them, when outputting the damage result of the electric energy meter screen, the confidence level of the damage result recognition will also be attached to prevent recognition errors, such as Figure 4 As shown, the confidence levels of the current recognition of white spots 01, cracks 02, scratches 03, screen distortion 04, black screen 05 and white screen 06 are 0.64, 0.70, 0.79, 0.83, 0.90 and 0.93 respectively.

[0103] For example, while outputting the damage results of the electricity meter screen, the location of the damaged layer, as well as the location and correlation of the damaged layer with the damage results and the position layer can also be output. This is not marked in the figure, which is beneficial for subsequent repair staff to conduct layer-based inspection and repair.

[0104] The method for detecting damage to the electric energy meter screen of the present invention further comprises: using common evaluation indicators for target detection to analyze the damage in step 104 and test the performance of the network model; the evaluation indicators are AP value, mAP value, Params value and FPS, wherein the AP value is the average accuracy of a single target, the mAP value is the average value of the average accuracy AP of all categories, the Params value is the size of the model parameters, and the FPS is the size of the detection speed of the model. The calculation formulas of these evaluation indicators are:

[0105]

[0106] Where N is the number of all target types, P is the accuracy of the target, R is the recall rate, TP is the number of correct target detections, and M is the number of total targets.

[0107] This embodiment uses an appearance image data set of an electric energy meter screen for testing, and outputs damage results of the electric energy meter screen based on the damage condition. The average accuracy of the damage results reaches 90.16%, and the detection accuracy is relatively high.

[0108] Exemplarily, since in step 101, the serial number of the electricity meter has been associated with the unassociated appearance images of the electricity meter screen in the powered and unpowered states, the damage results of the electricity meter screen have been associated with the serial number of the electricity meter. The staff can easily summarize the data of the damage results of the electricity meter screen, grasp the inspection conditions on the assembly line, and subsequently perform corresponding maintenance processing on the damaged electricity meter.

[0109] In summary, the method for detecting damage to the electric energy meter screen provided in the embodiment of the present application helps to collect data in different states by acquiring the appearance images of the electric energy meter screen in the powered and unpowered states, and provides rich samples for subsequent model training. Extracting the features of the appearance image and marking the damage of the electric energy meter screen through the image processing algorithm helps to establish a detailed data set and provide suitable training samples for the deep learning model. Model training using a deep learning model enables the model to more accurately and efficiently identify the damage of the electric energy meter screen, thereby improving the accuracy and efficiency of recognition. The real-time appearance image is input into the trained deep learning model to analyze whether the electric energy meter screen is damaged, thereby realizing the automatic monitoring and analysis of the electric energy meter screen status, enhancing the automation and intelligence level of the system, solving the compatibility of the appearance characteristics and screen changes of the new generation of electric energy meters with existing equipment and software and the low efficiency of manual detection, reducing the error of manual detection and improving efficiency.

[0110] See also Figure 5 An embodiment of the present application provides an electric energy meter screen damage detection device, which executes the electric energy meter screen damage detection method as described above. The device includes an image acquisition module 201, a feature extraction module 202, a model training module 203 and a result output module 204.

[0111] An image acquisition module 201 is used to acquire appearance images of the electric energy meter screen in the powered and unpowered states;

[0112] The feature extraction module 202 is used to extract the features of the appearance image through an image processing algorithm, calculate the correlation between the features of the appearance image and the protective layer, the display layer and the backlight layer, and mark the damage of the electric energy meter screen on the appearance image to form a marked image set; the marked image set includes the correlation between the damage and the protective layer, the display layer and the backlight layer;

[0113] A model training module 203 is used to perform model training on a pre-built deep learning model based on a set of annotated images to obtain a trained deep learning model;

[0114] The result output module 204 is used to input the real-time appearance image of the electric energy meter screen into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer.

[0115] Exemplarily, the feature extraction module 202 includes an extraction module and a labeling module; the extraction module is used to extract the features of the appearance image through an image processing algorithm; the extraction module is specifically used to:

[0116] Using edge detection to locate the screen position in the appearance image of the electric energy meter screen;

[0117] Based on the screen position, determining the screen image;

[0118] Extract the brightness features, color features, image content and texture features of the screen image.

[0119] Exemplarily, the annotated image set includes a first annotated image set and a second annotated image set;

[0120] The annotation module is used to annotate the damage of the electric energy meter screen on the appearance image to form an annotation image set; the annotation module is specifically used to:

[0121] The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-on state to form a first marked image set; the damage conditions of the electric energy meter screen in the first marked image set include dark screen, white screen, flowery screen and black screen;

[0122] The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-off state to form a second marked image set; the damage conditions of the electric energy meter screen in the second marked image set include yellowing, white spots, scratches and cracks.

[0123] Exemplary, pre-built deep learning models include SDD models.

[0124] Exemplarily, the result output module 204 is specifically used for:

[0125] The real-time appearance image of the electric energy meter screen is input into the trained deep learning model to obtain the damage condition of the real-time appearance image of the electric energy meter screen;

[0126] Based on the damage situation, the damage results of the electric energy meter screen are output; the damage results include damage, non-damage, damage type and the location of the damage layer.

[0127] Exemplarily, in the extraction module, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen, including:

[0128] Preprocessing the appearance image of the electric energy meter to obtain a preprocessed appearance image of the electric energy meter;

[0129] Based on the preprocessed appearance image of the electric energy meter, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen.

[0130] Exemplarily, in the extraction module, based on the preprocessed appearance image of the electric energy meter, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen, including:

[0131] A two-dimensional Gaussian template is used to perform smoothing and filtering on the pre-processed appearance image data of the electric energy meter;

[0132] Performing edge detection processing on the appearance image of the electric energy meter after smoothing and filtering processing to obtain an edge response image;

[0133] Distinguish the contour points and non-contour points of the edge response image to obtain contour lines;

[0134] Based on the contour line, the screen position in the appearance image of the electric energy meter screen is determined.

[0135] Exemplarily, in the extraction module, determining the screen position in the appearance image of the electric energy meter screen based on the contour line includes:

[0136] Based on the contour line, extract the edge line segment of the screen position;

[0137] Calculate the area of ​​the screen based on the edge segments;

[0138] Based on the area of ​​the screen and an area threshold of a preset screen specification, a screen position in the appearance image of the electric energy meter screen is determined.

[0139] Exemplarily, the image acquisition module 201 is specifically used for:

[0140] Acquire appearance images of the electric energy meter in powered and unpowered states;

[0141] The appearance image of the electric energy meter is processed in blocks to obtain the serial number of the electric energy meter and the unrelated appearance images of the screen of the electric energy meter in the power-on and power-off states;

[0142] The serial number of the electric energy meter and the unassociated appearance images of the electric energy meter screen in the powered and unpowered states are associated to obtain the appearance images of the electric energy meter screen in the powered and unpowered states.

[0143] The beneficial effects of the embodiments of the above-mentioned device for detecting damage to the electric energy meter screen can be found in the beneficial effects of the embodiments of the method for detecting damage to the electric energy meter screen, which will not be described in detail here.

[0144] It should be noted that although several units / modules or sub-units / modules of the electric energy meter screen damage detection device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiment of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules for embodiment.

[0145] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0146] The present application also provides a computer-readable storage medium, in which computer-readable storage medium is stored computer execution instructions. When a processor executes the computer execution instructions, the method for detecting damage to the electric energy meter screen provided in the above embodiment of the present application is implemented.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0148] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting damage to an electric energy meter screen, characterized in that: include: Obtain appearance images of the electric energy meter screen in powered and unpowered states; Extracting features of the appearance image by an image processing algorithm, calculating the correlations between the features of the appearance image and the protective layer, the display layer and the backlight layer respectively, and marking the damage of the electric energy meter screen on the appearance image to form a marked image set; the marked image set includes the correlations between the damage and the protective layer, the display layer and the backlight layer; Based on the annotated image set, a pre-built deep learning model is trained to obtain a trained deep learning model; The real-time appearance image of the electric energy meter screen is input into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer.

2. The method for detecting damage to an electric energy meter screen according to claim 1, characterized in that: The extracting the features of the appearance image by using an image processing algorithm comprises: Using edge detection to locate the screen position in the appearance image of the electric energy meter screen; Based on the screen position, determining a screen image; The brightness features, color features, image content and texture features of the screen image are extracted.

3. The method for detecting damage to an electric energy meter screen according to claim 1, characterized in that: The annotated image set includes a first annotated image set and a second annotated image set; The step of marking the damage condition of the electric energy meter screen on the appearance image to form a marked image set includes: The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-on state to form the first marked image set; the damage conditions of the electric energy meter screen in the first marked image set include dark screen, white screen, flowery screen and black screen; The damage conditions of the electric energy meter screen are marked on the appearance image of the electric energy meter screen in the power-off state to form the second marked image set; the damage conditions of the electric energy meter screen in the second marked image set include yellowing, white spots, scratches and cracks.

4. The method for detecting damage to an electric energy meter screen according to claim 1, characterized in that: The pre-built deep learning model includes an SDD model.

5. The method for detecting damage to an electric energy meter screen according to claim 1, characterized in that: The step of inputting the real-time appearance image of the electric energy meter screen into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer includes: Inputting the real-time appearance image of the electric energy meter screen into the trained deep learning model to obtain the damage condition of the real-time appearance image of the electric energy meter screen; Based on the damage situation, the damage result of the electric energy meter screen is output; the damage result includes damage, non-damage, damage type and location of the damage layer.

6. The method for detecting damage to an electric energy meter screen according to claim 2, characterized in that: The method of locating the screen position in the appearance image of the electric energy meter screen by using edge detection includes: Preprocessing the appearance image of the electric energy meter to obtain a preprocessed appearance image of the electric energy meter; Based on the preprocessed appearance image of the electric energy meter, edge detection is used to locate the screen position in the appearance image of the electric energy meter screen.

7. The method for detecting damage to an electric energy meter screen according to claim 6, characterized in that: The method of locating the screen position in the appearance image of the electric energy meter screen by using edge detection based on the preprocessed appearance image of the electric energy meter includes: A two-dimensional Gaussian template is used to perform smoothing and filtering on the pre-processed appearance image data of the electric energy meter; Performing edge detection processing on the appearance image of the electric energy meter after smoothing and filtering processing to obtain an edge response image; Distinguishing contour points and non-contour points of the edge response image to obtain contour lines; Based on the contour line, a screen position in the appearance image of the electric energy meter screen is determined.

8. The method for detecting damage to an electric energy meter screen according to claim 7, characterized in that: The step of determining the screen position in the appearance image of the electric energy meter screen based on the contour line includes: Based on the contour line, extracting edge line segments of the screen position; Calculating the area of ​​the screen based on the edge line segments; Based on the area of ​​the screen and an area threshold of a preset screen specification, a screen position in the appearance image of the electric energy meter screen is determined.

9. The method for detecting damage to an electric energy meter screen according to any one of claims 1 to 8, characterized in that: The obtaining of the appearance image of the electric energy meter screen in the power-on and power-off states includes: Acquire appearance images of the electric energy meter in powered and unpowered states; The appearance image of the electric energy meter is processed in blocks to obtain a serial number of the electric energy meter and unassociated appearance images of the screen of the electric energy meter in a powered-on state and a powered-off state; The serial number of the electric energy meter and the unassociated appearance images of the electric energy meter screen in the powered and unpowered states are associated to obtain the appearance images of the electric energy meter screen in the powered and unpowered states.

10. An electric energy meter screen damage detection device, executing the electric energy meter screen damage detection method according to any one of claims 1 to 9, characterized in that: include: An image acquisition module is used to acquire the appearance images of the electric energy meter screen in the powered and unpowered states; a feature extraction module, configured to extract features of the appearance image by using an image processing algorithm, calculate the correlations between the features of the appearance image and the protective layer, the display layer and the backlight layer, and mark the damage conditions of the electric energy meter screen on the appearance image to form a marked image set; the marked image set includes the correlations between the damage conditions and the protective layer, the display layer and the backlight layer; A model training module, used to perform model training on a pre-built deep learning model based on the annotated image set to obtain a trained deep learning model; The result output module is used to input the real-time appearance image of the electric energy meter screen into the trained deep learning model to analyze whether the electric energy meter screen is damaged and the location of the damaged layer.

Images