Multi-model-based equipment information identification method and device, equipment, medium and product
By using a multi-model-based approach, the equipment information of secondary power equipment is automatically identified, solving the problem of low identification efficiency in existing technologies and achieving efficient and automated equipment information extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUIZHOU POWER SUPPLY BUREAU OF GUANGDONG POWER GRID CO LTD
- Filing Date
- 2026-01-12
- Publication Date
- 2026-04-24
AI Technical Summary
In existing technologies, the process of identifying equipment information for secondary power equipment is time-consuming and inefficient, requiring manual intervention and resulting in high costs.
A multi-model-based approach is adopted. The first model is used to determine the screen area in the device image and perform preprocessing. The second model is used to identify the device information in the screen image and remove invalid information to achieve automated recognition.
It has enabled automated identification of information from secondary power equipment, shortening identification time, improving efficiency, and reducing labor costs.
Smart Images
Figure CN121921561A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the fields of image processing technology and text recognition technology, and in particular to a method, apparatus, device, medium and product for recognizing device information based on multiple models. Background Technology
[0002] In the power systems of power plants and substations, there are various types of electrical equipment, including primary power equipment and secondary power equipment. Secondary power equipment is used to protect, measure, and monitor the operation of primary power equipment; and it also possesses corresponding equipment information.
[0003] Before secondary power equipment is connected to the power system, its equipment information needs to be identified and recorded. Some technologies use manual methods such as taking photos or writing on paper to identify and record this information. However, these methods are time-consuming and inefficient.
[0004] Therefore, there is an urgent need for a solution that can automatically and efficiently identify equipment information of secondary power equipment. Summary of the Invention
[0005] The methods, apparatus, devices, media, and products for identifying equipment information based on multiple models provided in this application are intended to achieve automated and efficient identification of equipment information for secondary power equipment.
[0006] In a first aspect, embodiments of this application provide a method for identifying device information based on multiple models, including:
[0007] The equipment image of the secondary power equipment is acquired, and the equipment image is input into the first model for processing to determine the screen area in the equipment image that includes the equipment screen; wherein, the screen area includes equipment information;
[0008] The device image is preprocessed based on the screen area to obtain the screen image of the secondary power equipment;
[0009] The screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment.
[0010] In one possible implementation, the second model is a pre-trained text recognition model; the screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment, including:
[0011] The screen image is input into the second model for recognition to obtain the text information in the screen image; the text information includes device information and invalid information.
[0012] The text information is matched with a preset list of invalid information to obtain invalid information in the text information; and the invalid information in the text information is removed to obtain the equipment information of the secondary power equipment.
[0013] In one possible implementation, the screen image is input into a second model for recognition to obtain text information in the screen image, including:
[0014] From the screen image, at least one text region is identified; wherein the text region includes text information;
[0015] Character recognition is performed on each text region to obtain the text information in the screen image.
[0016] In one possible implementation, determining at least one text region from the screen image includes:
[0017] Determine the probability that each pixel in the screen image belongs to the text region to obtain a probability map;
[0018] The positional information of the text region is determined from the probability map, and at least one text region is determined based on the positional information; wherein, the positional information includes at least one of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
[0019] In one possible implementation, preprocessing the device image based on the screen area to obtain the screen image of the secondary power equipment includes:
[0020] The device image is resized according to the screen area to obtain a first image; wherein the size of the first image is the same as the size of the screen area.
[0021] The first image is converted to grayscale to obtain the second image; wherein the second image is a grayscale image.
[0022] The second image is then subjected to median filtering to obtain the third image;
[0023] The third image is binarized to obtain the screen image of the power secondary equipment.
[0024] In one possible implementation, binarizing the third image to obtain a screen image of the secondary power equipment includes:
[0025] The third image is divided into multiple computation regions, and the gray value threshold of the computation region is determined based on the gray value of the pixels in the computation region.
[0026] Within each calculation area, if the gray value of a pixel is determined to be greater than or equal to the gray value threshold, the gray value of the pixel is set to 255; otherwise, the gray value of the pixel is set to 0 to obtain the screen image of the secondary power equipment.
[0027] In one possible implementation, the method further includes:
[0028] Construct a first training set and a second training set; wherein the first training set includes multiple first training images and position information of screen regions in the first training images, and the second training set includes multiple second training images and text information in the second training images;
[0029] The first training set is input into the first initial model for training to obtain the trained first model; and the second training set is input into the second initial model for training to obtain the trained second model.
[0030] Secondly, embodiments of this application provide a device for identifying device information based on multiple models, comprising:
[0031] The first processing module is used to acquire equipment images of secondary power equipment, input the equipment images into the first model for processing, and determine the screen area in the equipment image that includes the equipment screen; wherein, the screen area includes equipment information;
[0032] The second processing module is used to preprocess the device image according to the screen area to obtain the screen image of the power secondary equipment.
[0033] The third processing module is used to input the screen image into the second model for recognition, and obtain the equipment information of the power secondary equipment.
[0034] In one possible implementation, the second model is a pre-trained text recognition model; the screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment; the third processing module is used for:
[0035] The screen image is input into the second model for recognition to obtain the text information in the screen image; the text information includes device information and invalid information.
[0036] The text information is matched with a preset list of invalid information to obtain invalid information in the text information; and the invalid information in the text information is removed to obtain the equipment information of the secondary power equipment.
[0037] In one possible implementation, the screen image is input to the second model for recognition to obtain the text information in the screen image, and the third processing module is used for:
[0038] From the screen image, at least one text region is identified; wherein the text region includes text information;
[0039] Character recognition is performed on each text region to obtain the text information in the screen image.
[0040] In one possible implementation, at least one text region is determined from the screen image, and the third processing module is used to:
[0041] Determine the probability that each pixel in the screen image belongs to the text region to obtain a probability map;
[0042] The positional information of the text region is determined from the probability map, and at least one text region is determined based on the positional information; wherein, the positional information includes at least one of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
[0043] In one possible implementation, the device image is preprocessed based on the screen area to obtain the screen image of the secondary power equipment, and the second processing module is used for:
[0044] The device image is resized according to the screen area to obtain a first image; wherein the size of the first image is the same as the size of the screen area.
[0045] The first image is converted to grayscale to obtain the second image; wherein the second image is a grayscale image.
[0046] The second image is then subjected to median filtering to obtain the third image;
[0047] The third image is binarized to obtain the screen image of the power secondary equipment.
[0048] In one possible implementation, the third image is binarized to obtain the screen image of the secondary power equipment, and the second processing module is used for:
[0049] The third image is divided into multiple computation regions, and the gray value threshold of the computation region is determined based on the gray value of the pixels in the computation region.
[0050] Within each calculation area, if the gray value of a pixel is determined to be greater than or equal to the gray value threshold, the gray value of the pixel is set to 255; otherwise, the gray value of the pixel is set to 0 to obtain the screen image of the secondary power equipment.
[0051] In one possible implementation, the first processing module is further configured to:
[0052] Construct a first training set and a second training set; wherein the first training set includes multiple first training images and position information of screen regions in the first training images, and the second training set includes multiple second training images and text information in the second training images;
[0053] The first training set is input into the first initial model for training to obtain the trained first model; and the second training set is input into the second initial model for training to obtain the trained second model.
[0054] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;
[0055] The memory stores computer-executed instructions;
[0056] The processor executes computer execution instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.
[0057] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.
[0058] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.
[0059] The embodiments of this application provide a method, apparatus, device, medium, and product for identifying device information based on multiple models. This method acquires device images of secondary power equipment, determines the area containing a screen within the device image using a first model, preprocesses the device image based on the screen area to obtain a screen image of the secondary power equipment, and determines the device information contained in the screen image using a second model. This achieves automated identification of device information of secondary power equipment and improves the efficiency of identifying device information of secondary power equipment. Attached Figure Description
[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0061] Figure 1 A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 1 ;
[0062] Figure 2A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 2 ;
[0063] Figure 3 This is a schematic diagram of a text area in an exemplary screen image;
[0064] Figure 4 A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 3 ;
[0065] Figure 5 This is a schematic diagram illustrating the effect of preprocessing an example device image.
[0066] Figure 6 This is a schematic diagram of a text area in an exemplary screen image;
[0067] Figure 7 A schematic diagram of the structure of the multi-model-based device information recognition device provided in this application;
[0068] Figure 8 A schematic diagram of the structure of the electronic device provided in this application.
[0069] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation
[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.
[0071] First, let me explain the terms used in this application:
[0072] Secondary power equipment refers to auxiliary equipment used for monitoring, measuring, controlling, protecting, and regulating primary power equipment. Primary power equipment directly participates in the production, transmission, distribution, and conversion of electrical energy within the power system. Secondary power equipment, on the other hand, ensures the safe and stable operation of primary power equipment. It may not be directly connected to the high-voltage main circuit of the power system, but rather connected to the primary power equipment through a low-voltage circuit.
[0073] Equipment information refers to relevant equipment information on secondary power equipment, which is usually displayed on the screen of the secondary power equipment.
[0074] In the power systems of power plants and substations, there are various types of electrical equipment. These devices work together to maintain the normal operation of the power system. Primary electrical equipment directly participates in the production, transmission, distribution, and conversion of electrical energy within the power system; secondary electrical equipment monitors, measures, controls, protects, and regulates the primary equipment. Each piece of secondary electrical equipment has its own equipment information, indicating its model, relevant software version, and configuration files.
[0075] Before secondary power equipment can be connected to the power system and begin operation, its equipment information needs to be identified and recorded. There are numerous secondary power devices in a power system. In some embodiments, the identification and recording of equipment information is done manually by taking photographs or recording data with pen and paper.
[0076] In the above embodiments, identifying equipment information of secondary power equipment by manually recording data is time-consuming, inefficient, and requires high labor costs.
[0077] The device information recognition method based on multiple models provided in this application acquires device images of secondary power equipment, determines the area containing the screen in the device image through a first model, preprocesses the device image based on the screen area to obtain the screen image of the secondary power equipment, and determines the device information contained in the screen image through a second model. This technical means realizes automated recognition of device information of secondary power equipment and improves the efficiency of recognizing device information of secondary power equipment.
[0078] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.
[0079] Figure 1 A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 1 ,like Figure 1 As shown, the method includes:
[0080] Step 101. Obtain the equipment image of the secondary power equipment, input the equipment image into the first model for processing, and determine the screen area in the equipment image that includes the equipment screen.
[0081] The screen area includes device information.
[0082] For example, by taking a picture of the secondary power equipment, an image of the secondary power equipment is obtained; the image includes the main structure of the secondary power equipment, and a screen is provided on the main structure of the secondary power equipment, on which the equipment information of the secondary power equipment is displayed.
[0083] The obtained device image is input into the first model. After processing by the first model, the screen area including the device screen is determined in the device image. For example, after processing by the first model, the vertex coordinates of the device screen in the device image are output. Based on the vertex coordinates of the device screen, a polygonal bounding box is drawn in the device image. This polygonal bounding box is used to indicate the screen area in the device image. The screen area includes the device screen, and the device information of the power secondary equipment is displayed on the screen.
[0084] Step 102. Preprocess the device image according to the screen area to obtain the screen image of the power secondary equipment.
[0085] For example, based on the determined screen area, the device image is preprocessed to obtain the screen image of the power secondary equipment.
[0086] Specifically, the preprocessing process may include, but is not limited to: resizing the equipment image, adjusting its color, and adjusting its tilt angle. After this preprocessing, a screen image of the secondary power equipment can be obtained, laying the foundation for subsequent identification of the equipment information on the screen.
[0087] Step 103. Input the screen image into the second model for recognition to obtain the equipment information of the secondary power equipment.
[0088] For example, the screen image obtained in step 102 is input into the second model for text recognition. After the text recognition is processed by the second model, the equipment information of the secondary power equipment displayed in the screen image is obtained.
[0089] Specifically, the device information in the screen image is identified character by character to obtain all characters in the screen image, thereby obtaining the device information of the secondary power equipment. The device information of the secondary power equipment is in text format.
[0090] The device information recognition method based on a multi-model provided in this application acquires a device image of a secondary power equipment, including the screen of the secondary power equipment; inputs the device image into a first model to determine the screen area in the device image; processes the device image based on the screen area to obtain the screen image of the secondary power equipment; and then inputs the screen image into a second model to determine the device information contained in the screen image. This achieves automated recognition and extraction of device information from secondary power equipment. On the one hand, when there are many secondary power equipment, it shortens the time spent extracting and recognizing the device information of each secondary power equipment, thereby improving the recognition efficiency of device information; on the other hand, it eliminates the need for manual observation and recording, reducing labor costs.
[0091] In practical applications, the screens of secondary power equipment may display other information besides the equipment information itself. This other information is invalid for the purpose of extracting and identifying the equipment information. Therefore, based on the aforementioned embodiments, this embodiment explains how to perform text recognition on screen images to obtain equipment information.
[0092] Figure 2 A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 2 ,like Figure 2 As shown in one example, the second model is a pre-trained text recognition model; in the process of the second model recognizing the screen image to obtain the equipment information of the secondary power equipment, the method includes:
[0093] Step 201. Input the screen image into the second model for recognition to obtain the text information in the screen image.
[0094] The text information includes device information and invalid information.
[0095] For example, a screen image is input into a pre-trained second model. After the second model performs text recognition processing, the text information in the screen image is obtained. This text information may include valid device information and invalid information.
[0096] Specifically, the second model can achieve text recognition of screen images through the following steps:
[0097] Step 2011. Identify at least one text region from the screen image; wherein the text region includes text information.
[0098] For example, the screen image includes multiple text areas, each containing text information. For instance, the screen of a secondary power equipment may include valid equipment information and invalid information. The equipment information may include, but is not limited to, equipment model, software name, device version number, etc.
[0099] For example, Figure 3 This is a schematic diagram of a text area in an exemplary screen image, such as... Figure 3 As shown, the device screen of an exemplary secondary power device includes multiple text areas, such as... Figure 3 The area enclosed by the dashed rectangle is shown. Each text area contains text information.
[0100] Furthermore, the text region can be determined from the screen image through the following steps:
[0101] The probability of each pixel in the screen image belonging to the text region is determined to obtain a probability map.
[0102] For example, a screen image includes multiple pixels. Based on a pre-trained text recognition model, the probability that each pixel in the screen image belongs to a text region can be determined. It can be understood that if a pixel belongs to a text region, the probability should be close to 1.
[0103] After determining the probability that each pixel in the screen image belongs to a text region, the screen image can be converted into a probability map. The probability represented by each point in this probability map is the probability that the corresponding pixel in the screen image belongs to a text region.
[0104] The location information of the text region is determined from the probability map, and at least one text region is identified based on the location information. The location information includes at least one of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
[0105] For example, a contour detection algorithm can be used to identify multiple points in the probability map whose probability values are greater than or equal to a threshold, and contour detection can be performed on these multiple points to obtain the location information of the text region.
[0106] Specifically, the positional information of the text region may include one or more of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
[0107] The text region's vertex coordinates indicate the positional coordinates of each vertex within the text region on the screen image. The text region's center coordinates indicate the positional coordinates of the text region's center point on the screen image. The text region's width indicates its distance along the width direction. The text region's height indicates its distance along the height direction. The text region's rotation angle indicates the angle of rotation between the text region's width-direction edge and the screen image's width-direction edge.
[0108] Based on the location information mentioned above, multiple corresponding text regions can be identified.
[0109] In the example above, the probability of each pixel in the screen image belonging to a text region is determined, and based on this probability, the location information of the text region in the screen image is determined. Based on this location information, one or more text regions in the screen image are identified. This allows us to determine which regions in the screen image contain text information, laying the foundation for subsequent text recognition based on these regions.
[0110] Step 2012. Perform character recognition on each text region to obtain the text information in the screen image.
[0111] For example, character recognition processing is performed on each text region in the screen image in sequence to obtain all the text information in the screen image.
[0112] Combination Figure 3 Explanation, for Figure 3 Character recognition is performed on each text area in the screen image shown to obtain text information of the text type in the exemplary screen image, namely: "View", "Set", "Preset", "Print", "Device Information", "PRS-753 Microcomputer Circuit Protection Device", "Software Name", "PRS-753AM-NW-200516", "Device Version Number", "V3.50-2M", "Protection CRC Code", "BB7C", "Management Reset", "267", "Fault Number", "00293".
[0113] In the example above, a pre-trained second model is used to determine the text region from the screen image, and the text information contained in the text region is obtained through character recognition. This enables automated recognition and extraction of text information from screen images, eliminating the need for manual recognition and recording, and improving the efficiency of recognizing text information on the screen of secondary power equipment.
[0114] Step 202. Match the text information with a preset list of invalid information to obtain invalid information in the text information; and remove invalid information in the text information to obtain the equipment information of the secondary power equipment.
[0115] As shown in the example above, the text information on the screen of secondary power equipment includes both valid and invalid information. Only the equipment information needs to be recorded; therefore, after obtaining the text information from the screen image, a filtering process is required.
[0116] For example, the text information is sequentially matched against a pre-defined list of invalid information to determine which text information is invalid. This invalid information is then removed to obtain the device information. Invalid information refers to information on the screen of the secondary power equipment that does not belong to the device information. For instance, the screen of the secondary power equipment may include some indicator information in addition to device information. For example, indicator information may include "Confirm," "Operating Status," "Next Page," etc., which are also displayed on the screen of the secondary power equipment, but these information are not part of the device information. This information on the screen is pre-selected to form an invalid information list.
[0117] The preset invalid information list includes multiple invalid information. In each matching process, the matching degree between the text information and the multiple invalid information in the preset invalid information list is calculated. If the matching degree is greater than or equal to the preset matching degree threshold, the text information is determined to be invalid information and is removed.
[0118] Optionally, the identified device information can be stored in text format.
[0119] In the above embodiments, a pre-trained text recognition model determines the probability that each pixel in the screen image belongs to a text region, obtaining a probability map. Based on the probability map, text regions are then identified, and character recognition is performed on each text region to obtain text information. After the text recognition model obtains the text information, it is matched against a pre-defined list of invalid information to remove invalid information, thus ultimately obtaining the device information from the screen image. This enables automated identification of the required device information from screen images, improving the efficiency of device information recognition and eliminating the need for manual identification, thereby reducing labor costs.
[0120] As described in the foregoing embodiments, after determining the screen area from the device image, the device image can be preprocessed based on the screen area to obtain the screen image of the secondary power equipment. Based on the foregoing embodiments, this embodiment explains how to perform image preprocessing on the device image to obtain the screen image.
[0121] Figure 4 A flowchart illustrating the multi-model-based device information identification method provided in this application. Figure 3 ,like Figure 4 As shown, in one example, the method includes the following steps during the preprocessing of a device image based on a screen area:
[0122] Step 401. Adjust the size of the device image according to the screen area to obtain the first image.
[0123] The size of the first image is the same as the size of the screen area.
[0124] For example, the device image is cropped according to the size of the screen area to obtain a first image. It can be understood that the size of the first image is the same as the size of the screen area.
[0125] For example, in a device image, an image within a screen area is cropped to obtain the first image.
[0126] Optionally, to speed up processing, the size of the first image should not be too large. If the size of the first image is still larger than 2000×2000, the size of the first image can be further adjusted. Here, 2000×2000 means that the width and height of the image are both 2000 pixels.
[0127] In one alternative implementation, the size of the first image can be further adjusted using the resize function in the OpenCV library.
[0128] Step 402. Perform grayscale processing on the first image to obtain the second image.
[0129] The second image is a grayscale image.
[0130] For example, in general, the captured device images are color images. In order to facilitate the identification of device information, it is necessary to convert the color images into grayscale images.
[0131] In one alternative implementation, the first image can be converted to grayscale using the cvtColor function in the OpenCV library, thus obtaining a second image in grayscale space.
[0132] Specifically, when performing grayscale conversion, the grayscale value of each pixel can be calculated using the following formula:
[0133] Gray value = 0.299 × R + 0.587 × G + 0.114 × B;
[0134] Where R represents the intensity value of a pixel in the red channel; G represents the intensity value of a pixel in the green channel; and B represents the intensity value of a pixel in the blue channel.
[0135] Step 403. Perform median filtering on the second image to obtain the third image.
[0136] For example, the captured device image may contain a lot of noise. An appropriate filtering method can be selected to denoise the second image.
[0137] In one alternative implementation, the medianBlur function in the OpenCV library can be used to perform median filtering on the second image to obtain the third image.
[0138] Specifically, during the median filtering process, the gray value of each pixel is replaced by the median of the gray values of all pixels in its neighborhood, thereby effectively removing noise from the image.
[0139] In one example, the original noise intensity of the second image is 68.03. After denoising by median filtering, the filtered noise intensity of the third image is 67.43, thus reducing the noise intensity.
[0140] Step 404. Binarize the third image to obtain the screen image of the power secondary equipment.
[0141] For example, to better identify device information, the third image needs to be binarized. Specifically, the binarization process can be achieved through the following steps:
[0142] Step 4041. Divide the third image into multiple calculation regions, and determine the gray value threshold of the calculation region based on the gray value of the pixels in the calculation region.
[0143] For example, the third image is divided into multiple computational regions, each including multiple pixels, and each pixel having a corresponding grayscale value. Taking the first computational region as an example, a grayscale threshold for the first computational region is determined based on the grayscale value of each pixel within the first computational region.
[0144] Specifically, the median grayscale value of all pixels within the first calculation area can be used as the grayscale threshold for that first calculation area.
[0145] Repeat the above steps until the grayscale threshold for each calculation region is determined.
[0146] Step 4042. Within each calculation area, if the gray value of a pixel is determined to be greater than or equal to the gray value threshold, then the gray value of the pixel is set to 255; otherwise, the gray value of the pixel is set to 0, so as to obtain the screen image of the secondary power equipment.
[0147] For example, continuing with the first calculation region, the gray value of each pixel is compared with the gray value threshold of the first calculation region.
[0148] If the gray value of a pixel is determined to be greater than or equal to the gray value threshold, then the gray value of that pixel is set to 255; if the gray value of a pixel is determined to be less than the gray value threshold, then the gray value of that pixel is set to 0.
[0149] Repeat the above operation until the binarization of each pixel in the first calculation area is completed; then repeat the above operation until the binarization of pixels in all calculation areas is completed. This yields the screen image of the secondary power equipment.
[0150] In one alternative implementation, the adaptive threshold binarization of the third image can be performed using the adaptiveThreshold function in the OpenCV library to obtain the screen image of the power secondary equipment.
[0151] In the example above, binarization converts the screen image from grayscale to black and white, thereby improving the accuracy of subsequent text recognition. Furthermore, by dividing the computational region and implementing adaptive threshold binarization, it is possible to ensure good binarization results for the screen image under different lighting conditions.
[0152] In the above embodiments, by performing size adjustment, grayscale processing, median filtering, and binarization on the device image, a screen image can be obtained, laying the foundation for the subsequent process of recognizing device information based on the screen image. Through the above image preprocessing, the recognition accuracy of text information by the second model can be improved, thereby indirectly improving the recognition accuracy of device information for secondary power equipment.
[0153] In one example, Figure 5 This is a schematic diagram illustrating the effect of a preprocessing procedure for an example device image. Figure 5 As shown, the device image is resized according to the screen area to obtain the first image; the first image is converted to grayscale to obtain the second image; the second image is subjected to median filtering to obtain the third image; and the third image is binarized to obtain the screen image of the power secondary equipment.
[0154] In one example, Figure 6 This is a schematic diagram of a text area in an exemplary screen image. Figure 6 As shown, multiple text regions are identified in the screen image, each containing text information. It should be noted that the text information includes device information as well as invalid information (…). Figure 6 (Not shown within the dashed box).
[0155] As can be seen from the foregoing embodiments, both the first model and the second model can be pre-trained. Based on any of the foregoing embodiments, this embodiment explains how to train the first model and the second model.
[0156] In one example, the method also includes:
[0157] Construct the first training set and the second training set.
[0158] The first training set includes multiple first training images and the position information of screen regions in the first training images; the second training set includes multiple second training images and the text information in the second training images.
[0159] For example, a first training set is used to train a first model, and a second training set is used to train a second model. The first training set includes multiple first training images and the location information of screen regions within those first training images. It can be understood that in each first training image within the first training set, a screen region is determined through pre-annotation, and based on this determined screen region, its location information can be obtained. The first training images are used as input data, i.e., features, and the location information of their corresponding screen regions is used as target data, i.e., labels.
[0160] The second training set includes multiple second training images and text information within those images. Each second training image in the second training set contains corresponding text information. The second training images are used as input data (features), and their corresponding text information is used as target data (labels).
[0161] The first training set is input into the first initial model for training to obtain the trained first model; and the second training set is input into the second initial model for training to obtain the trained second model.
[0162] For example, a first training set is input into a first initial model for training, the process is iterated, and the parameters in the first initial model are adjusted to obtain a trained first model. A second training set is input into a second initial model for training, the process is iterated, and the parameters in the second initial model are adjusted to obtain a trained second model.
[0163] Optionally, the architecture of the first initial model can be a Convolutional Neural Network (CNN), a Transformer architecture, etc. This embodiment does not specifically limit the architecture of the first initial model.
[0164] Optionally, the architecture of the second initial model can be a PaddleOCR model. PaddleOCR refers to the PaddlePaddle-based Optical Character Recognition (OCR) tool. In the PaddleOCR model, text region detection and / or text recognition can be achieved through the Efficient and Accurate Scene Text Detector (EAST) model.
[0165] Furthermore, to improve the computational speed of the PaddleOCR model, a Math Kernel Library (MKL library) can be configured during the application of the second model. Based on the second model configured with the MKL library, the recognition speed of device information in device images can be further improved.
[0166] It should be noted that this embodiment does not specifically limit the architecture of the second initial model.
[0167] In one example, a pre-trained second model configured with the MKL library was used to perform text recognition on screen images of secondary power equipment. The accuracy reached 98.86%, and the time taken was only 2.11 seconds. Compared to manual recognition of equipment information, automating the recognition process significantly improves recognition speed and thus increases the efficiency of equipment information recognition.
[0168] The device information recognition method based on multiple models provided in this application acquires device images of secondary power equipment, determines the area containing the screen in the device image using a pre-trained first model, preprocesses the device image based on the screen area to obtain the screen image of the secondary power equipment, and determines the device information contained in the screen image using a pre-trained second model. This technical means realizes automated recognition of device information of secondary power equipment. On the one hand, it shortens the time spent recognizing device information of secondary power equipment and improves the efficiency of recognizing device information of secondary power equipment; on the other hand, it eliminates the need for manual recognition and recording, reducing labor costs.
[0169] Figure 7 A schematic diagram of the structure of the device information recognition device based on multiple models provided in this application is shown below. Figure 7As shown, the device information identification device 70 based on multiple models provided in this embodiment includes:
[0170] The first processing module 701 is used to acquire the equipment image of the secondary power equipment, input the equipment image into the first model for processing, and determine the screen area in the equipment image that includes the equipment screen; wherein, the screen area includes equipment information;
[0171] The second processing module 702 is used to preprocess the device image according to the screen area to obtain the screen image of the power secondary equipment.
[0172] The third processing module 703 is used to input the screen image into the second model for recognition to obtain the equipment information of the power secondary equipment.
[0173] In one possible implementation, the second model is a pre-trained text recognition model; the screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment; the third processing module 703 is used for:
[0174] The screen image is input into the second model for recognition to obtain the text information in the screen image; the text information includes device information and invalid information.
[0175] The text information is matched with a preset list of invalid information to obtain invalid information in the text information; and the invalid information in the text information is removed to obtain the equipment information of the secondary power equipment.
[0176] In one possible implementation, the screen image is input to the second model for recognition to obtain the text information in the screen image, and the third processing module 703 is used for:
[0177] From the screen image, at least one text region is identified; wherein the text region includes text information;
[0178] Character recognition is performed on each text region to obtain the text information in the screen image.
[0179] In one possible implementation, at least one text region is determined from the screen image, and the third processing module 703 is used to:
[0180] Determine the probability that each pixel in the screen image belongs to the text region to obtain a probability map;
[0181] The positional information of the text region is determined from the probability map, and at least one text region is determined based on the positional information; wherein, the positional information includes at least one of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
[0182] In one possible implementation, the device image is preprocessed based on the screen area to obtain the screen image of the secondary power equipment. The second processing module 702 is used for:
[0183] The device image is resized according to the screen area to obtain a first image; wherein the size of the first image is the same as the size of the screen area.
[0184] The first image is converted to grayscale to obtain the second image; wherein the second image is a grayscale image.
[0185] The second image is then subjected to median filtering to obtain the third image;
[0186] The third image is binarized to obtain the screen image of the power secondary equipment.
[0187] In one possible implementation, the third image is binarized to obtain the screen image of the secondary power equipment, and the second processing module 702 is used for:
[0188] The third image is divided into multiple computation regions, and the gray value threshold of the computation region is determined based on the gray value of the pixels in the computation region.
[0189] Within each calculation area, if the gray value of a pixel is determined to be greater than or equal to the gray value threshold, the gray value of the pixel is set to 255; otherwise, the gray value of the pixel is set to 0 to obtain the screen image of the secondary power equipment.
[0190] In one possible implementation, the first processing module 701 is further configured to:
[0191] Construct a first training set and a second training set; wherein the first training set includes multiple first training images and position information of screen regions in the first training images, and the second training set includes multiple second training images and text information in the second training images;
[0192] The first training set is input into the first initial model for training to obtain the trained first model; and the second training set is input into the second initial model for training to obtain the trained second model.
[0193] The device information identification device based on multiple models provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.
[0194] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Figure 8As shown, the electronic device 80 provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the device 80 further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.
[0195] In a specific implementation, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to perform the above-described method.
[0196] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.
[0197] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.
[0198] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.
[0199] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.
[0200] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.
[0201] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.
[0202] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.
[0203] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.
[0204] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.
[0205] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0206] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0207] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0208] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.
[0209] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. A method for identifying device information based on multiple models, characterized in that, include: A device image of a secondary power equipment is acquired, and the device image is input into a first model for processing to determine the screen area in the device image that includes the device screen; wherein, the screen area includes device information; The device image is preprocessed according to the screen area to obtain the screen image of the secondary power equipment. The screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment.
2. The method according to claim 1, characterized in that, The second model is a pre-trained text recognition model; the screen image is input into the second model for recognition to obtain the equipment information of the secondary power equipment, including: The screen image is input into the second model for recognition to obtain the text information in the screen image; wherein, the text information includes device information and invalid information; The text information is matched with a preset list of invalid information to obtain invalid information in the text information; and the invalid information in the text information is removed to obtain the equipment information of the secondary power equipment.
3. The method according to claim 2, characterized in that, The screen image is input into the second model for recognition to obtain the text information in the screen image, including: At least one text region is determined from the screen image; wherein the text region includes text information; Character recognition is performed on each of the text regions to obtain the text information in the screen image.
4. The method according to claim 3, characterized in that, Determining at least one text region from the screen image includes: Determine the probability that each pixel in the screen image belongs to a text region to obtain a probability map; The position information of the text region is determined from the probability map, and at least one text region is determined based on the position information; wherein the position information includes at least one of the following: vertex coordinates of the text region, center coordinates of the text region, width of the text region, height of the text region, and rotation angle of the text region.
5. The method according to claim 1, characterized in that, Preprocessing the device image based on the screen area to obtain the screen image of the secondary power equipment includes: The device image is resized according to the screen area to obtain a first image; wherein the size of the first image is the same as the size of the screen area. The first image is converted to grayscale to obtain the second image; wherein the second image is a grayscale image; The second image is subjected to median filtering to obtain the third image; The third image is binarized to obtain the screen image of the secondary power equipment.
6. The method according to claim 5, characterized in that, The third image is binarized to obtain the screen image of the secondary power equipment, including: The third image is divided into multiple calculation regions, and a gray value threshold for each calculation region is determined based on the gray values of the pixels within the calculation regions. Within each of the calculation regions, if the grayscale value of a pixel is determined to be greater than or equal to the grayscale value threshold, then the grayscale value of the pixel is set to 255; otherwise, the grayscale value of the pixel is set to 0, so as to obtain the screen image of the secondary power equipment.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: Construct a first training set and a second training set; wherein the first training set includes multiple first training images and position information of screen regions in the first training images, and the second training set includes multiple second training images and text information in the second training images; The first training set is input into the first initial model for training to obtain the trained first model; and the second training set is input into the second initial model for training to obtain the trained second model.
8. A device for identifying device information based on multiple models, characterized in that, include: The first processing module is used to acquire an image of a secondary power equipment, input the image into a first model for processing, and determine the screen area in the image that includes the equipment screen; wherein, the screen area includes equipment information. The second processing module is used to preprocess the device image according to the screen area to obtain the screen image of the power secondary equipment. The third processing module is used to input the screen image into the second model for recognition to obtain the equipment information of the power secondary equipment.
9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.
11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-7.