Real-time collecting and recording method, device and equipment for electric power test detection instruments and meters

Abstract

The invention relates to the technical field of power equipment monitoring, and discloses a power test detection instrument real-time collecting and recording method, device and equipment, and the method comprises the steps: obtaining a background image containing a non-communication type power test detection instrument from background images collected in real time, and taking the background image as a target image; identifying the target image by using the trained target detection identification model, and obtaining the type of the non-communication type electric power test detection instrument in the target image; if the type is a pointer type instrument, extracting a dual-channel image based on a pointer and scales, and acquiring data of the pointer type instrument by using a reading calculation method to complete real-time acquisition and recording; and if the type is a digital display instrument, extracting a digital text, and obtaining data of the digital display instrument by using a character positioning identification method to complete real-time acquisition and recording.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment monitoring, and in particular to a real-time data acquisition method, device and equipment for power test and detection instruments. Background Art

[0002] With the continuous development and expansion of the power system, the number and types of power equipment are increasing; the status monitoring and index measurement of power equipment is one of the important means to ensure safe production and intelligent operation and maintenance of power, and is a key component of the new power system. Through real-time monitoring and index measurement of power equipment, potential failures of equipment can be discovered in time, major accidents can be prevented, and the safe and stable operation of the power system can be ensured.

[0003] However, for power test and detection instruments that do not have network communication capabilities, there is still a part of the equipment that lacks the necessary data interface due to technical limitations or early design, and cannot accurately read the instrument data. Therefore, it is difficult to directly include it in the scope of digital management, and it is impossible to achieve flexible real-time access and data interconnection of the detection equipment terminals; this hinders the instant sharing and integrated analysis of data, and also brings substantial difficulties to the coordination and optimization of the test process. Summary of the invention

[0004] To this end, the technical problem to be solved by the present invention is to overcome the problem in the prior art that the instrument data of the power test and detection instrument that does not have network communication capability cannot be read in time, resulting in untimely data reading and reduced working efficiency of the instrument.

[0005] In order to solve the above technical problems, the present invention provides a real-time recording method for electric power test and detection instruments, comprising:

[0006] From the background images collected in real time, a background image including non-communication type electric power test and detection instruments is obtained as a target image;

[0007] The trained target detection and recognition model is used to identify the target image and obtain the type of non-communication type power test and detection instruments in the target image;

[0008] If the type is a pointer instrument, extract the dual-channel image based on the pointer and scale, use the reading calculation method to obtain the data of the pointer instrument, and complete real-time recording;

[0009] If the type is a digital instrument, extract the digital text, use the text positioning and recognition method to obtain the data of the digital instrument and complete real-time recording.

[0010] Preferably, extracting a dual-channel image of a pointer instrument includes:

[0011] Input the target image of the pointer instrument into the U-Net network model to obtain the segmentation mask map;

[0012] Traverse each pixel in the segmentation mask image, divide the pixel with a pixel value of 1 into the pointer channel, and divide the pixel with a pixel value of 2 into the scale channel to obtain a dual-channel image.

[0013] Preferably, the method of using the reading calculation method to obtain the data of the pointer instrument includes:

[0014] Based on the dual-channel image of the pointer instrument, the central axis of the pointer of the pointer instrument is made parallel to its adjacent scale lines, and the center of the pointer instrument is taken as the origin of the polar coordinate system;

[0015] Convert the electricity in the polar coordinate system to a rectangular coordinate system, wherein the vertical downward direction of the origin is the negative direction of the y-axis;

[0016] Take the negative direction of the y-axis as the starting line and rotate clockwise to scan the dual-channel image of the pointer instrument;

[0017] According to the difference in pixel values ​​representing scale, pointer and background, the total number of scales corresponding to the pointer instrument is accumulated;

[0018] Based on the pointer deflection angle, the ratio of the number of scale lines on the left side of the pointer to the total number of scale lines is calculated to obtain the pointer reading as the data of the pointer instrument.

[0019] Preferably, extracting digital text from a digital display instrument comprises: extracting digital text from a target image of the digital display instrument using image segmentation technology.

[0020] Preferably, the method of using the text positioning and recognition method to obtain the data of the digital instrument includes: using ORC technology to recognize the digital text extracted from the digital instrument to obtain the data of the digital instrument.

[0021] Preferably, the trained single-stage target detection YOLOv5 model is used to identify the target image, and the type of non-communication type power test and detection instrument in the target image is obtained, including:

[0022] Input the target image to the input end for Mosaic data enhancement to obtain an enhanced image;

[0023] The enhanced image is input into the backbone network, and passes through the CSPDarknet53 module and the Mish activation function in sequence to obtain the activation feature map;

[0024] Input the activated feature map into the Neck layer, use FPN to fuse high-level semantic features into low-level features layer by layer in a top-down manner to obtain down-sampled features; use PAN to obtain up-sampled features in a bottom-up manner; fuse the up-sampled features with the down-sampled features to obtain a fused feature map;

[0025] The fused feature map is input into the output layer, and CIoU_Loss is used as the loss function to perform weighted non-maximum suppression to obtain the type of non-communication type power test and detection instruments in the target image.

[0026] Preferably, the trained two-stage target detection model is used to identify the target image, and the type of non-communication type power test and detection instrument in the target image is obtained, including:

[0027] Obtain multiple candidate box regions from the target image;

[0028] Use non-maximum suppression to eliminate redundant candidate box areas with high overlap and retain multiple possible candidate box areas;

[0029] Use convolutional neural network to extract regional features of each possible candidate box area;

[0030] Based on the regional features of each possible candidate frame area, a classifier is used to perform classification to obtain the type of non-communication type power test and detection instruments in the target image.

[0031] Preferably, after acquiring the target image, the method further includes: preprocessing the target image, including removing noise and correcting the tilt of the target image; the tilt correction includes template matching and affine transformation of the target image.

[0032] This embodiment provides a real-time data acquisition and recording device for electric power test and detection instruments, including:

[0033] A target image acquisition module is used to acquire a background image containing non-communication type electric power test and detection instruments and meters from the background image collected in real time as a target image;

[0034] A type recognition module is used to recognize the target image using the trained target detection and recognition model, and obtain the type of non-communication type power test and detection instruments in the target image;

[0035] The recording module, if the type is a pointer instrument, extracts the dual-channel image based on the pointer and scale, uses the reading calculation method to obtain the data of the pointer instrument, and completes real-time recording; if the type is a digital instrument, extracts the digital text, uses the text positioning and recognition method to obtain the data of the digital instrument, and completes real-time recording.

[0036] This embodiment provides a real-time recording device for electric power test and detection instruments, including:

[0037] Processor; and

[0038] a memory for storing computer program instructions;

[0039] When the computer program instructions are loaded and run by the processor, the processor executes the real-time data collection and recording method of the electric power test and detection instrument as described above.

[0040] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0041] The real-time data collection and recording method for electric power test and detection instruments and meters described in the present invention uses a target detection and recognition model to identify different types of non-communication electric power test and detection instruments and meters in a target image; for pointer-type instruments, a dual-channel image is extracted based on the pointer and the scale, and the data of the pointer-type instrument is obtained using a reading calculation method; for digital-display instruments, digital text is extracted, and the data of the digital-display instrument is obtained using a text positioning and recognition method. The present invention constructs corresponding data reading methods for different types of electric power test and detection instruments and meters that do not have network communication capabilities, realizes flexible real-time access and data interconnection of electric power test and detection instruments and meters, optimizes the test and detection process, improves the test and detection efficiency, and enhances the intelligent level of test and detection of electric power equipment, while reducing manpower and time costs and improving the safety and reliability of test data collection and recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

[0043] Figure 1 It is a flow chart of the steps of the real-time data acquisition method of the electric power test and detection instrument provided by the present invention;

[0044] Figure 2 It is a principle flow chart of the real-time data acquisition method of the electric power test and detection instrument provided by the present invention;

[0045] Figure 3 It is a schematic diagram of another device structure provided by the present invention;

[0046] Figure 4 It is a schematic diagram of the hardware structure of the computer device provided by the present invention;

[0047] Figure 5 It is a structural block diagram of the mobile terminal provided by the present invention. DETAILED DESCRIPTION

[0048] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0049] Reference Figure 1 As shown, the flowchart of the real-time recording method of the electric power test detection instrument of the present invention includes the following specific steps:

[0050] S101: acquiring, from the background images collected in real time, a background image including non-communication type electric power test and detection instruments and meters as a target image;

[0051] S102: using the trained target detection and recognition model to recognize the target image, and obtaining the type of non-communication type power test and detection instruments in the target image;

[0052] S103: If the type is a pointer instrument, extract a dual-channel image based on the pointer and the scale, use a reading calculation method to obtain data of the pointer instrument, and complete real-time recording;

[0053] S104: If the type is a digital instrument, extract the digital text, use the text positioning and recognition method to obtain the data of the digital instrument, and complete the real-time recording.

[0054] Specifically, in step S103, the real-time recording of the pointer instrument includes:

[0055] S103-1: Input the target image of the pointer instrument into the U-Net network model to obtain a segmentation mask image;

[0056] S103-2: traverse each pixel point in the segmentation mask image, divide the pixel points with a pixel value of 1 into the pointer channel, and divide the pixel points with a pixel value of 2 into the scale channel, to obtain a dual-channel image;

[0057] S103-3: Based on the dual-channel image of the pointer instrument, the central axis of the pointer of the pointer instrument is made parallel to its adjacent scale lines, and the center of the pointer instrument is taken as the origin of the polar coordinate system;

[0058] S103-4: Convert the electricity in the polar coordinate system to a rectangular coordinate system, wherein the rectangular coordinate system takes the vertical downward direction of the origin as the negative direction of the y-axis;

[0059] S103-5: Taking the negative direction of the y-axis as the starting line, rotate clockwise to scan the dual-channel image of the pointer-type instrument;

[0060] S103-6: accumulating the total number of scales corresponding to the pointer instrument according to the difference in pixel values ​​representing the scales, the pointer and the background;

[0061] S103-7: Based on the pointer deflection angle, calculate the ratio of the number of scale lines on the left side of the pointer to the total number of scale lines, and obtain the pointer reading as the data of the pointer instrument.

[0062] In this embodiment, after the target image of the pointer instrument is processed by the U-Net model, the output segmentation mask is a picture of the same size as the original image, and the value of each pixel indicates which category the pixel belongs to (for example, 1 represents the pointer and 2 represents the scale). Create two blank channels of the same size as the input image, channel 1 is used to store the pixels of the pointer, and channel 2 is used to store the pixels of the scale. Traverse each pixel of the mask image and check the category of the pixel: if the value of a certain pixel is 1, then its value is set to 1 in channel 1, indicating that this pixel belongs to the pointer. If the value of a certain pixel is 2, then its value is set to 1 in channel 2, indicating that this pixel belongs to the scale. After this processing step, the pixels of the pointer will be drawn into channel 1, and the pixels of the scale will be drawn into channel 2, thereby obtaining a dual-channel image. The segmentation process of this embodiment can realize the rapid and accurate identification of key information such as the pointer and scale of the instrument equipment through image pixel comparison, and improve the accuracy of data reading.

[0063] Specifically, in step S104, real-time recording of the digital display instrument includes:

[0064] S104-1: extracting digital text from a digital instrument, including: extracting digital text from a target image of the digital instrument using image segmentation technology;

[0065] S104-2: Use ORC technology to recognize the digital text extracted from the digital instrument to obtain the data of the digital instrument.

[0066] This embodiment includes a text positioning recognition and reading calculation process, which can realize the calculation of the readings in the instrument equipment through OCR technology. Compared with the traditional method of manual data entry, the instrument equipment readings are recognized through computer recognition technology, which can achieve the purpose of improving the authenticity and reliability of the data collected and recorded by the power test and detection instruments.

[0067] Specifically, in step S102 of the embodiment of the present invention, the trained target detection and recognition model includes a single-stage target detection YOLOv5 model and a two-stage target detection model; the recognition process specifically includes:

[0068] ① Use the trained single-stage target detection YOLOv5 model to identify the target image and obtain the types of non-communication power test and detection instruments in the target image, including:

[0069] Input the target image to the input end for Mosaic data enhancement to obtain an enhanced image;

[0070] The enhanced image is input into the backbone network, and passes through the CSPDarknet53 module and the Mish activation function in sequence to obtain the activation feature map;

[0071] Input the activated feature map into the Neck layer, use FPN to fuse high-level semantic features into low-level features layer by layer in a top-down manner to obtain down-sampled features; use PAN to obtain up-sampled features in a bottom-up manner; fuse the up-sampled features with the down-sampled features to obtain a fused feature map;

[0072] The fused feature map is input into the output layer, and CIoU_Loss is used as the loss function to perform weighted non-maximum suppression to obtain the type of non-communication type power test and detection instruments in the target image.

[0073] ② Use the trained two-stage target detection model to identify the target image and obtain the types of non-communication power test and detection instruments in the target image, including:

[0074] Obtain multiple candidate box regions from the target image;

[0075] Use non-maximum suppression to eliminate redundant candidate box areas with high overlap and retain multiple possible candidate box areas;

[0076] Use convolutional neural network to extract regional features of each possible candidate box area;

[0077] Based on the regional features of each possible candidate frame area, a classifier is used to perform classification to obtain the type of non-communication type power test and detection instruments in the target image.

[0078] Specifically, in an embodiment of the present invention, after acquiring the target image, it also includes: preprocessing the target image, including removing noise and correcting the tilt of the target image; the tilt correction includes template matching and affine transformation of the target image. This embodiment uses image processing technology to remove noise interference and improve the image quality and prominence of the main part of the instrument in view of the image shooting environment and light conditions. The tilt correction is to perform template matching and affine transformation on the instrument in the original image. First, the feature contours of the template image and the image to be corrected are extracted, and the contour similar to the template contour is searched by sliding the template image on the image to be corrected. After the matching result is obtained, the rotation angle is calculated, and then the transformation matrix is ​​calculated according to the angle. Finally, the affine transformation technology is used to rotate the instrument image to be corrected according to the transformation matrix to achieve the purpose of correction. By preprocessing the image, the image quality and prominence of the instrument equipment in the target image are improved, which is conducive to improving the recognition accuracy of the instrument reading.

[0079] The real-time data collection and recording method for electric power test and detection instruments and meters described in the present invention uses a target detection and recognition model to identify different types of non-communication electric power test and detection instruments and meters in a target image; for pointer-type instruments, a dual-channel image is extracted based on the pointer and the scale, and the data of the pointer-type instrument is obtained using a reading calculation method; for digital-display instruments, digital text is extracted, and the data of the digital-display instrument is obtained using a text positioning and recognition method. The present invention constructs corresponding data reading methods for different types of electric power test and detection instruments and meters that do not have network communication capabilities, realizes flexible real-time access and data interconnection of electric power test and detection instruments and meters, optimizes the test and detection process, improves the test and detection efficiency, and enhances the intelligent level of test and detection of electric power equipment, while reducing manpower and time costs and improving the safety and reliability of test data collection and recording.

[0080] Specifically, the embodiment of the present invention divides the instruments and meters in the real power test scenario into four main categories, including:

[0081] 1) Equipped with backend software equipment: Design a collection and docking tool that can shield the differences between different data, and obtain instrument data through database docking or service docking (such as webservice or API provided by the product);

[0082] 2) Communication equipment: Design intelligent access adaptation middleware system, use virtual entity technology to intensively process the communication protocols involved in communication-type power instruments and meters, and realize the conversion and interconnection of heterogeneous protocols;

[0083] 3) Non-communication equipment: Research on the enhancement scheme of computer vision technology for the detection, segmentation and recognition of instruments and meters in power test scenarios, and design customized schemes for instrument and meter recording in power test scenarios;

[0084] 4) Questionable equipment: There are some devices in the ledger data that cannot be accurately classified due to missing characteristic information. Further field research is needed to improve the record information and achieve classification adaptation of the questionable equipment.

[0085] The embodiments of this application are aimed at instruments and meters that do not have data communication capabilities, and study the automated recording technology of power equipment test and detection instruments and meters. The intelligent image detection and recognition technology for data recording scenarios is studied, and the target detection and recognition model and image segmentation model based on deep learning networks are studied for digital display instruments and pointer instruments respectively, and the mobile AI meter recognition function is developed to realize the real-time recording and image storage of detection data.

[0086] The embodiment of the present invention is aimed at real-time recording of non-communication equipment, referring to Figure 2 As shown, it is a principle flow chart of the real-time recording method of the electric power test detection instrument according to the embodiment of the present application, and the specific steps include:

[0087] S201 Instrument detection process: when it is detected that the type of the electric power test detection instrument is a non-communication type instrument device, a target image including the instrument device is acquired;

[0088] Extracting image features from the target image, and using a trained target detection and recognition model to identify the type of instrument equipment corresponding to the image features; or

[0089] A candidate frame area is extracted from the target image, image features of the candidate frame area are extracted, and the type of instrument equipment corresponding to the image features is identified using a trained target detection and recognition model.

[0090] S202: Preprocessing the target image containing the instrument device, including:

[0091] Perform denoising on the target image;

[0092] Performing template matching and radiation transformation on the instrument equipment in the target image;

[0093] Among them, the template matching and radial transformation include: extracting feature contours from the template image and the target image; searching for contours whose similarity with the template contour is higher than the target threshold by sliding the template image on the target image; calculating the rotation angle; calculating the transformation matrix based on the angle; and rotating the target image according to the transformation matrix using affine transformation technology.

[0094] S203: using the trained target detection and recognition model to recognize the target image, and outputting the type of instrument equipment in the target image; wherein the types include digital display instruments and pointer instruments;

[0095] S204: extracting key information of the instrument device from the target image;

[0096] For pointer instruments: divide the target image containing the pointer instrument into two channels and process them separately; traverse each pixel of the mask image and compare it with the label pixels of the scale and pointer respectively. The part with pixel equal to 1 is determined as the pointer and merged into the pointer channel; the part with pixel equal to 2 is determined as the scale and merged into the scale channel;

[0097] For digital instruments: Use image segmentation technology to extract digital text from the target image of the digital instrument;

[0098] S205: performing text location recognition and reading calculation on the key information;

[0099] For pointer instruments: scan the target image, and calculate the total number of scales of the instrument device based on the different pixel values ​​of the scale, pointer and background image; the proportion of the index scale to the range is expressed as the number of scale lines on the left side of the pointer divided by the total number of scale lines;

[0100] For digital instruments: Use ORC technology to recognize the digital text extracted from the digital instrument and obtain the data of the digital instrument;

[0101] Specifically, in step S203, there are two methods for identifying the type of instrument equipment in the target image, namely:

[0102] ① Extract image features from the target image, and use the trained target detection and recognition model to identify the type of instrument equipment corresponding to the image features.

[0103] ② Extract a candidate frame area from the target image, extract the image features of the candidate frame area, and use the trained target detection and recognition model to identify the type of instrument equipment corresponding to the image.

[0104] The meter recognition process based on computer vision mainly includes meter detection, image preprocessing, meter segmentation, text positioning and recognition, etc. The recognition of pointer-type meters also involves the reading calculation method; specifically:

[0105] 1) Instrument detection: It uses target detection technology to detect and accurately locate instruments in complex background images, which is the basis for a series of subsequent work such as instrument segmentation and reading recognition. The target detection algorithm based on deep learning can be divided into two-stage target detection algorithm and single-stage target detection algorithm. The single-stage target detection directly obtains the classification and probability of the object to be detected, while the two-stage target detection needs to generate the candidate box area first and then obtain the detection result. Taking the single-stage target detection YOLOv5 algorithm as an example, the input end uses Mosaic data enhancement to enhance the training speed and detection accuracy of the model; the backbone network is used to extract image features, using CSPDarknet53 and Mish activation function; the Neck layer uses the FPN+PAN combined structure to complete downsampling and upsampling, which can improve the network's ability to integrate feature information; the output layer uses CIoU_Loss (Complete-IoU) as the loss function, uses weighted non-maximum suppression, and outputs position coordinates and classification results.

[0106] 2) Image preprocessing: Its purpose is to remove noise and correct the tilt of the original image. According to the image shooting environment and lighting conditions, image processing technology is used to remove noise interference and improve the image quality and prominence of the main part of the instrument. Tilt correction is to perform template matching and affine transformation on the instrument in the original image. First, the feature contours of the template image and the image to be corrected are extracted. The template image is slid on the image to be corrected to search for contours similar to the template contour. After the matching result is obtained, the rotation angle is calculated. Secondly, the transformation matrix is ​​calculated based on the angle. Finally, the affine transformation technology is used to rotate the instrument image to be corrected according to the transformation matrix to achieve the purpose of correction.

[0107] 3) Instrument segmentation: It is to separate the key parts of the instrument image, such as dial scale, pointer, digital text, etc., through image segmentation technology. Image segmentation has always been an important task in the field of computer vision. The network is proposed for the task of salient target detection, which can extract the most attractive target in the image. At present, most segmentation tasks rely on semantic segmentation models, such as U-Net, U2-Net, etc. The salient detection tasks are all binary classification tasks, which are committed to segmenting the most attractive targets and backgrounds in the image. For example, when annotating the pointer instrument, the scale and pointer are annotated. The pointer and scale need to be input into the network separately, so the image needs to be divided into two channels for separate processing. Each pixel of the mask image is traversed and compared with the label pixels of the scale and pointer respectively. The part with a pixel equal to 1 is determined as the pointer and merged into the pointer channel; the part with a pixel equal to 2 is determined as the scale and merged into the scale channel; this method is used to draw the pointer and scale on different channels respectively.

[0108] 4) Text positioning recognition and reading calculation: At present, the technology of identifying the extracted text with the help of OCR technology is relatively mature. For digital display instruments, the required data can be accurately extracted by combining the corresponding preset templates. The data recognition algorithm of pointer-type instruments is relatively complex. Taking the distance method to recognize a circular dial as an example, its principle is based on the assumption that the central axis of the pointer is parallel to the adjacent scale lines. The center point of the instrument can be regarded as the origin of the polar coordinate system. Each point (ρ, θ) in the polar coordinates is converted into a rectangular coordinate system (x, y). The vertical downward direction of the center of the instrument is taken as the negative direction of the y-axis. The -y axis is used as the starting line to rotate and scan clockwise. The angle between the current pixel point and the -y axis is calculated by the initial set hyperparameters. After the instrument image enters the network, it will be scanned clockwise. According to the different pixel values ​​of the scale, pointer and background, they are accumulated into the corresponding categories respectively. The total number of scales of the instrument can be calculated by accumulation. The proportion of the pointer reading to the range is expressed as the number of scale lines on the left side of the pointer divided by the total number of scale lines, and the scale ratio of the instrument can be obtained.

[0109] The present application adopts the above technical scheme. Compared with the prior art, the real-time data recording method of electric power test and detection instruments and meters provided in the embodiment of the present application obtains a target image containing the instrument and meter when it is detected that the type of the electric power test and detection instrument and meter is a non-communication type instrument and meter; uses a trained target detection and recognition model to recognize the target image, and outputs the type of the instrument and meter in the target image, wherein the type includes a digital display instrument and a pointer instrument; extracts key information of the instrument and meter from the target image; performs text positioning recognition and reading calculation on the key information, thereby solving the problem that the electric power test and detection instruments and meters in the related technology lack a unified data interface, and manual recording cannot provide authenticity, real-time and reliability guarantees, and achieves the effect of providing authenticity, real-time and reliability of the data recorded by the electric power test and detection instruments and meters.

[0110] Based on the above embodiments, the present invention also provides a real-time recording device for electric power test and detection instruments, including:

[0111] The target image acquisition module 100 is used to acquire a background image including non-communication type electric power test and detection instruments and meters from the background image collected in real time as a target image;

[0112] The type recognition module 200 is used to recognize the target image using the trained target detection and recognition model, and obtain the type of non-communication type power test and detection instruments in the target image;

[0113] The recording module 300, if the type is a pointer instrument, extracts the dual-channel image based on the pointer and the scale, uses the reading calculation method to obtain the data of the pointer instrument, and completes the real-time recording; if the type is a digital instrument, extracts the digital text, uses the text positioning and recognition method to obtain the data of the digital instrument, and completes the real-time recording.

[0114] The real-time recording device for electric power test and detection instruments of the present embodiment is used to implement the aforementioned real-time recording method for electric power test and detection instruments. Therefore, the specific implementation method of the real-time recording device for electric power test and detection instruments can be seen in the embodiment part of the real-time recording method for electric power test and detection instruments in the previous text. For example, the target image acquisition module 100 and the type recognition module 200 are respectively used to implement steps S101 and S102 in the above-mentioned real-time recording method for electric power test and detection instruments; the recording module 300 is used to implement steps S103 and S104 in the above-mentioned real-time recording method for electric power test and detection instruments. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0115] Specifically, refer to Figure 3As shown, another schematic diagram of the structure of the device according to the embodiment of the present invention specifically includes:

[0116] The target image acquisition module 100 comprises:

[0117] An acquisition unit 31 is used to acquire a target image including the instrument device when it is detected that the type of the power test detection instrument device is a non-communication type instrument device;

[0118] A denoising unit 32, configured to perform denoising processing on the target image after acquiring the target image containing the instrument device;

[0119] A transformation unit is used to perform template matching and radial transformation on the instrument equipment in the target image, wherein the template matching and radial transformation include: extracting feature contours from the template image and the target image; searching for contours whose similarity to the template contour is higher than a target threshold by sliding the template image on the target image; calculating a rotation angle; calculating a transformation matrix according to the angle; and rotating the target image according to the transformation matrix using an affine transformation technique;

[0120] The type identification module 200 includes:

[0121] The recognition unit is used to recognize the target image using the trained target detection and recognition model, and output the type of the instrument device in the target image, which includes:

[0122] A first extraction subunit is used to extract image features from the target image and identify the type of instrument equipment corresponding to the image features using a trained target detection and recognition model; or

[0123] A second extraction subunit is used to extract a candidate frame area from the target image, extract image features of the candidate frame area, and use a trained target detection and recognition model to identify the type of instrument equipment corresponding to the image features;

[0124] The recording module 300 includes:

[0125] An extraction unit 33, used to extract key information of the instrument device from the target image;

[0126] A division module, used for dividing the target image including the pointer instrument into two channels for separate processing;

[0127] The comparison module is used to traverse each pixel of the mask image and compare it with the label pixels of the scale and pointer respectively. The part with a pixel equal to 1 is determined as the pointer and incorporated into the pointer channel; the part with a pixel equal to 2 is determined as the scale and incorporated into the scale channel;

[0128] The reading unit 34 is used to perform text location recognition and reading calculation on the key information.

[0129] Among them, the types include digital instruments and pointer instruments; when the type of the instrument equipment is a pointer instrument, the reading unit 34 includes: a scanning module, used to scan the target image, and according to the different pixel values ​​of the scale, pointer and background image, the total number of scales of the instrument equipment is calculated by accumulation, and the proportion of the pointer reading to the range is expressed as the number of scale lines to the left of the pointer divided by the total number of scale lines.

[0130] Based on the above embodiments, the embodiments of the present invention further provide a real-time recording device for electric power test and detection instruments, including:

[0131] Processor; and

[0132] a memory for storing computer program instructions;

[0133] When the computer program instructions are loaded and run by the processor, the processor executes the real-time data collection and recording method of the electric power test and detection instrument as described above.

[0134] In combination with the embodiment of the present application, the real-time data collection and recording method of the electric power test and detection instrument can be implemented by a computer device. Figure 4 4 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present application, including a processor 41 and a memory 42 storing computer program instructions.

[0135] The processor 41 may include a central processing unit (CPU), or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0136] The memory 42 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 42 may include a hard disk drive HDD (Hard Disk Drive), a floppy disk drive, a solid state drive SSD (Solid State Drive), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus USB (Universal Serial Bus) drive or a combination of two or more of these. In appropriate cases, the memory 42 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 42 may be inside or outside the data processing device. In a specific embodiment, the memory 42 is a non-volatile memory. In a specific embodiment, the memory 42 includes a read-only memory ROM (Read-Only Memory) and a random access memory RAM (Random Access Memory). In appropriate cases, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory), an erasable PROM (Erasable Programmable Read-Only Memory), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory), an electrically alterable ROM (Electrically Alterable Read-Only Memory) or a flash memory (FLASH) or a combination of two or more of these. In appropriate cases, the RAM may be a static random access memory SRAM (Static Random-Access Memory) or a dynamic random access memory DRAM (Dynamic Random Access Memory), wherein the DRAM may be a fast page mode dynamic random access memory FPMDRAM (Fast Page Mode Dynamic Random Access Memory), an extended data output dynamic random access memory EDODRAM (Extended Date Out Dynamic Random Access Memory), a synchronous dynamic random access memory SDRAM (Synchronous Dynamic Random-Access Memory), etc.

[0137] The memory 42 can be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 41. The processor 41 reads and executes the computer program instructions stored in the memory 42 to implement any one of the real-time recording methods for electric power test detection instruments in the above embodiments.

[0138] In some embodiments, the real-time recording device for electric power test and detection instruments may also include a communication interface 43 and a bus 40. Figure 4 As shown, the processor 41, the memory 42, and the communication interface 43 are connected via a bus 40 and communicate with each other.

[0139] The communication interface 43 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present application. The communication interface 43 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0140] The bus 40 includes hardware, software or both, and couples the components of the computer device to each other. The bus 40 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 40 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB) bus, a Hyper Transport (HT) interconnection, an Industry Standard Architecture (ISA) bus, an InfiniBand (InfiniBand) interconnection, a Low Pin Count (LPC) bus, a Memory Bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association (VLB) bus, or other suitable buses or a combination of two or more of these. Where appropriate, bus 40 may include one or more buses. Although the present application describes and illustrates a specific bus, the present application contemplates any suitable bus or interconnection.

[0141] Specifically, the processor of the embodiment of the present invention is configured to implement the real-time data collection and recording method of the electric power test and detection instrument provided by the present invention.

[0142] The real-time recording device for electric power test and detection instruments provided in the above embodiment is applied to a mobile terminal in the field of electric power test and detection. Figure 5, is a block diagram of a mobile terminal according to an embodiment of the present application, wherein the mobile terminal specifically includes: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a wireless fidelity (WiFi) module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that Figure 5 The structure of the mobile terminal shown in the figure does not constitute a limitation on the mobile terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0143] ① The RF circuit 110 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 180 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier LNA (Low Noise Amplifier), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication GSM (Global System of Mobile communication), General Packet Radio Service GPRS (General Packet Radio Service), Code Division Multiple Access CDMA (Code Division Multiple Access), Wideband Code Division Multiple Access WCDMA (Wideband Code Division Multiple Access), Long Term Evolution LTE (Long Term Evolution), email, short message service SMS (Short Messaging Service), etc.

[0144] ② The memory 120 can be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile terminal by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile terminal (such as audio data, a phone book, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0145] ③The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile terminal. Specifically, the input unit 130 may include a touch panel 131 and other input devices 132. The touch panel 131, also known as a touch screen, can collect the user's touch operations on or near it (such as the user's operation on the touch panel 131 or near the touch panel 131 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 180, and can receive and execute commands sent by the processor 180. In addition, the touch panel 131 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0146] ④ The display unit 140 can be used to display information input by the user or information provided to the user and various menus of the mobile terminal. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Furthermore, the touch panel 131 may cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 5 In the embodiment, the touch panel 131 and the display panel 141 are used as two independent components to implement the input and output functions of the mobile terminal, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated to implement the input and output functions of the mobile terminal.

[0147] ⑤ Sensor 150, such as light sensor, motion sensor and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 141 and / or the backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile terminal, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0148] ⑥ The speaker 161 and microphone 162 in the audio circuit 160 can provide an audio interface between the user and the mobile terminal. The audio circuit 160 can transmit the received audio data to the speaker 161 after converting the electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the processor 180 for processing, and then sent to another mobile terminal through the RF circuit 110, or the audio data is output to the memory 120 for further processing.

[0149] ⑦WiFi is a short-range wireless transmission technology. The mobile terminal can help users send and receive emails, browse web pages and access streaming media through the WiFi module 170. It provides users with wireless broadband Internet access. Figure 5 A WiFi module 170 is shown, but it is understandable that it is not an essential component of the mobile terminal and can be omitted as needed without changing the essence of the invention, or replaced with other short-range wireless transmission modules, such as a Zigbee module or a WAPI module.

[0150] ⑧The processor 180 is the control center of the mobile terminal. It uses various interfaces and lines to connect various parts of the entire mobile terminal. It executes various functions of the mobile terminal and processes data by running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, so as to monitor the mobile terminal as a whole. Optionally, the processor 180 may include one or more processing units; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 180.

[0151] ⑨ The mobile terminal also includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption management.

[0152] ⑩The mobile terminal may also include a camera, a Bluetooth module, etc.

[0153] Based on the above embodiments, this embodiment further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the real-time data acquisition and recording method of the electric power test and detection instrument as described above is implemented.

[0154] In addition, in combination with the real-time recording method of the electric power test detection instrument in the above embodiment, the embodiment of the present application can provide a computer-readable storage medium for implementation. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the real-time recording methods of the electric power test detection instrument in the above embodiment is implemented.

[0155] The real-time data collection and recording method for electric power test and detection instruments and meters described in the present invention uses a target detection and recognition model to identify different types of non-communication electric power test and detection instruments and meters in a target image; for pointer-type instruments, a dual-channel image is extracted based on the pointer and the scale, and the data of the pointer-type instrument is obtained using a reading calculation method; for digital-display instruments, digital text is extracted, and the data of the digital-display instrument is obtained using a text positioning and recognition method. The present invention constructs corresponding data reading methods for different types of electric power test and detection instruments and meters that do not have network communication capabilities, realizes flexible real-time access and data interconnection of electric power test and detection instruments and meters, optimizes the test and detection process, improves the test and detection efficiency, and enhances the intelligent level of test and detection of electric power equipment, while reducing manpower and time costs and improving the safety and reliability of test data collection and recording.

[0156] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0157] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0158] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0159] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0160] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A real-time recording method for electric power test and detection instruments, characterized in that: include: From the background images collected in real time, a background image including non-communication type electric power test and detection instruments is obtained as a target image; The trained target detection and recognition model is used to identify the target image and obtain the type of non-communication type power test and detection instruments in the target image; If the type is a pointer instrument, extract the dual-channel image based on the pointer and scale, use the reading calculation method to obtain the data of the pointer instrument, and complete real-time recording; If the type is a digital instrument, extract the digital text, use the text positioning and recognition method to obtain the data of the digital instrument and complete real-time recording.

2. The real-time recording method of electric power test and detection instruments according to claim 1 is characterized in that: Extract dual-channel images of pointer instruments, including: Input the target image of the pointer instrument into the U-Net network model to obtain the segmentation mask map; Traverse each pixel in the segmentation mask image, divide the pixel with a pixel value of 1 into the pointer channel, and divide the pixel with a pixel value of 2 into the scale channel to obtain a dual-channel image.

3. The real-time recording method of electric power test and detection instruments according to claim 2 is characterized in that: The method of using the reading calculation to obtain the data of the pointer instrument includes: Based on the dual-channel image of the pointer instrument, the central axis of the pointer of the pointer instrument is made parallel to its adjacent scale lines, and the center of the pointer instrument is taken as the origin of the polar coordinate system; Convert the electricity in the polar coordinate system to a rectangular coordinate system, wherein the vertical downward direction of the origin is the negative direction of the y-axis; Take the negative direction of the y-axis as the starting line and rotate clockwise to scan the dual-channel image of the pointer instrument; According to the difference in pixel values ​​representing scale, pointer and background, the total number of scales corresponding to the pointer instrument is accumulated; Based on the pointer deflection angle, the ratio of the number of scale lines on the left side of the pointer to the total number of scale lines is calculated to obtain the pointer reading as the data of the pointer instrument.

4. The real-time recording method of electric power test and detection instruments according to claim 1 is characterized in that: Extracting digital text from a digital instrument includes: extracting digital text from a target image of the digital instrument using image segmentation technology.

5. The real-time recording method of electric power test and detection instruments according to claim 4 is characterized in that: The method of using the text positioning and recognition method to obtain the data of the digital display instrument includes: using the ORC technology to recognize the digital text extracted from the digital display instrument to obtain the data of the digital display instrument.

6. The real-time recording method of electric power test and detection instruments according to claim 1 is characterized in that: The trained single-stage target detection YOLOv5 model is used to identify the target image and obtain the types of non-communication power test and detection instruments in the target image, including: Input the target image to the input end for Mosaic data enhancement to obtain an enhanced image; The enhanced image is input into the backbone network, and passes through the CSPDarknet53 module and the Mish activation function in sequence to obtain the activation feature map; Input the activated feature map into the Neck layer, use FPN to fuse high-level semantic features into low-level features layer by layer in a top-down manner to obtain down-sampled features; use PAN to obtain up-sampled features in a bottom-up manner; fuse the up-sampled features with the down-sampled features to obtain a fused feature map; The fused feature map is input into the output layer, and CIoU_Loss is used as the loss function to perform weighted non-maximum suppression to obtain the type of non-communication type power test and detection instruments in the target image.

7. The real-time recording method of electric power test and detection instruments according to claim 1 is characterized in that: The trained two-stage target detection model is used to identify the target image and obtain the types of non-communication power test and detection instruments in the target image, including: Obtain multiple candidate box regions from the target image; Use non-maximum suppression to eliminate redundant candidate box areas with high overlap and retain multiple possible candidate box areas; Use convolutional neural network to extract regional features of each possible candidate box area; Based on the regional features of each possible candidate frame area, a classifier is used to perform classification to obtain the type of non-communication type power test and detection instruments in the target image.

8. The real-time recording method of electric power test and detection instruments according to claim 1 is characterized in that: After acquiring the target image, the method further includes: preprocessing the target image, including removing noise and correcting the tilt of the target image; the tilt correction includes performing template matching and affine transformation on the target image.

9. A real-time recording device for electric power test and detection instruments, characterized in that: include: A target image acquisition module is used to acquire a background image containing non-communication type electric power test and detection instruments and meters from the background image collected in real time as a target image; A type recognition module is used to recognize the target image using the trained target detection and recognition model, and obtain the type of non-communication type power test and detection instruments in the target image; The recording module, if the type is a pointer instrument, extracts a dual-channel image based on the pointer and scale, uses the reading calculation method to obtain the data of the pointer instrument, and completes real-time recording; If the type is a digital instrument, extract the digital text, use the text positioning and recognition method to obtain the data of the digital instrument and complete real-time recording.

10. A real-time recording device for electric power test and detection instruments, characterized in that: include: processor; as well as a memory for storing computer program instructions; Wherein, when the computer program instructions are loaded and executed by the processor, the processor executes the real-time data acquisition and recording method of the electric power test and detection instrument as described in any one of claims 1 to 8.

Images