Casing oil level gauge identification method and device, computer equipment and storage medium

By combining the trained identification model and multi-category labeling technology, the casing oil level meter of power equipment is identified, which solves the problems of low recognition efficiency and low accuracy of casing oil level meter, and achieves higher recognition accuracy and simplification Labeling process.

CN119091104APending Publication Date: 2024-12-06CHINA SOUTHERN POWER GRID ARTIFICIAL INTELLIGENCE TECHNOLOGY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411191904.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-28
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art has problems such as low recognition efficiency, error-prone and poor real-time performance in the identification of casing oil level meters of power equipment. In particular, the development time cost of deep learning models and the iteration speed are high, resulting in low recognition accuracy.

Method used

Using the first and second recognition models trained in combination, the casing marking box is used to cut and multi-category annotation to obtain target training data, which is used to identify the oil dial and liquid level line of the casing oil level meter to improve identification Accuracy.

Benefits of technology

The unified rectangular box labeling method reduces mislabeling, simplifies the labeling process, and improves the recognition accuracy of the casing oil level meter image and the accuracy of the recognition results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091104A_ABST
    Figure CN119091104A_ABST
Patent Text Reader

Abstract

The invention relates to a casing oil level gauge recognition method, device and equipment and a storage medium, and the method comprises the steps: obtaining a to-be-recognized casing oil level gauge image, carrying out the casing frame prediction of the casing oil level gauge image according to an oil level gauge recognition model, obtaining a casing prediction frame corresponding to the casing oil level gauge image, and carrying out the recognition of the casing prediction frame. And performing oil level meter plate frame identification and liquid level line frame identification on the casing prediction frame to obtain an oil level meter plate prediction frame and a liquid level line prediction frame, and determining an oil level identification result corresponding to the casing oil level meter image based on the oil level meter plate prediction frame and the liquid level line prediction frame. The method comprises the following steps: training according to an oil level gauge sample image carrying a casing marking frame to obtain a first recognition model, cutting and multi-class marking the casing marking frame to obtain target training data, training according to the target training data to obtain a second recognition model, and combining the first recognition model and the second recognition model to obtain an oil level gauge recognition model. By adopting the method, the accuracy of the obtained oil level identification result can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of electric power equipment, and in particular to a casing oil level gauge identification method, device, computer equipment, computer-readable storage medium and computer program product. Background Art

[0002] With the development of power equipment technology and the promotion and use of various types of power equipment, during the operation of power equipment, it is necessary to pay attention to the different parameters of the power equipment, so as to determine whether the operating status of the power equipment is stable according to the different parameters of the power equipment, so as to handle and repair it in time when a fault occurs.

[0003] Traditionally, it is usually necessary to monitor the oil level gauge count of power equipment to ensure the stable operation of the power equipment. When identifying the oil level gauge count of power equipment, manual on-site reading is usually used, which still has problems such as low efficiency, prone to errors, and poor real-time performance.

[0004] In order to improve the recognition accuracy of oil level gauge counting, there is a method that uses drone photography technology and patrol robot photography technology to automatically collect data on oil level gauge images of power equipment, and then uses deep learning models to automatically identify oil level gauges from oil level gauge images. However, there are many types of casing oil level gauges for substation equipment, and there is little actual counting data and hidden danger data for different types of casing oil level gauges, which leads to high development time cost of deep learning models and slow model iteration speed. There are still problems such as uneven recognition effects and low recognition accuracy for oil level gauges of different equipment. Summary of the invention

[0005] Based on this, it is necessary to provide a casing oil level gauge identification method, device, computer equipment, computer readable storage medium and computer program product that can improve the counting and identification accuracy of casing oil level gauges for different power equipment in response to the above technical problems.

[0006] In a first aspect, the present application provides a casing oil level gauge identification method, comprising:

[0007] Acquire a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0008] According to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image;

[0009] Based on the oil gauge prediction frame and the liquid level line prediction frame, an oil level recognition result corresponding to the casing oil level gauge image is determined.

[0010] In an exemplary embodiment, the initial recognition model is trained according to the target training data, and when a model training end condition is met, a trained second recognition model is obtained, including:

[0011] Performing model training on the initial recognition model according to the target training data, and determining target confidence loss, classification loss and positioning loss during the model training process;

[0012] Determine the model training loss according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss, respectively, and the target confidence loss, classification loss and positioning loss;

[0013] When the model training loss meets the model training end condition, the training is stopped to obtain a trained second recognition model.

[0014] In a second aspect, the present application also provides a casing oil level gauge identification device, comprising:

[0015] The first prediction frame acquisition module is used to obtain a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0016] A second prediction frame obtaining module is used to perform oil gauge frame recognition and liquid level line frame recognition on the casing prediction frame according to the oil level gauge recognition model, and obtain an oil gauge frame and a liquid level line prediction frame corresponding to the casing oil level gauge image;

[0017] The oil level recognition result determination module is used to determine the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and the liquid level line prediction frame.

[0018] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0019] Acquire a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0020] According to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image;

[0021] Based on the oil gauge prediction frame and the liquid level line prediction frame, an oil level recognition result corresponding to the casing oil level gauge image is determined.

[0022] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0023] Acquire a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0024] According to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image;

[0025] Based on the oil gauge prediction frame and the liquid level line prediction frame, an oil level recognition result corresponding to the casing oil level gauge image is determined.

[0026] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0027] Acquire a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0028] According to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image;

[0029] Based on the oil gauge prediction frame and the liquid level line prediction frame, an oil level recognition result corresponding to the casing oil level gauge image is determined.

[0030] In the above-mentioned casing oil level gauge recognition method, device, computer equipment, computer-readable storage medium and computer program product, a casing oil level gauge image to be recognized is obtained, and a casing frame prediction is performed on the casing oil level gauge image according to a trained oil level gauge recognition model, so as to obtain a casing prediction frame corresponding to the casing oil level gauge image, wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, and specifically, the first recognition model is obtained by training according to the sample image of the oil level gauge carrying the casing annotation frame, and the target training data can be obtained by cropping and multi-category labeling of the casing annotation frame, including oil gauge plate labeling and liquid level line labeling, so as to train the second recognition model according to the target training data. Among them, due to the use of a unified rectangular frame labeling method, it is not necessary to accurately divide each pixel, which simplifies the labeling process while reducing the mislabeling caused by inconsistent standards, and can simultaneously obtain different labeling categories for the casing oil level gauge image, reducing the probability of mislabeling caused by a single labeling method. Furthermore, according to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge disk frame recognition and liquid level line frame recognition, and the oil gauge disk prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge image can be quickly obtained. Based on the oil gauge disk prediction frame and the liquid level line prediction frame, the oil level recognition result corresponding to the casing oil level gauge image can be accurately obtained, thereby improving the accuracy of the obtained oil level recognition result through multi-category labeling and recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] FIG1 is an application environment diagram of a casing oil level meter identification method according to an embodiment;

[0033] Figure 2 A schematic diagram of a flow chart of a casing oil level gauge identification method in one embodiment;

[0034] Figure 3 A schematic diagram of a process for determining an oil level recognition result corresponding to a casing oil level gauge image in one embodiment;

[0035] Figure 4 A schematic diagram of an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to a casing oil level gauge image in an embodiment;

[0036] Figure 5 A schematic diagram of a process of training a second recognition model in one embodiment;

[0037] Figure 6 is a flow chart of a first recognition model obtained by training in one embodiment;

[0038] Figure 7 A schematic diagram of a flow chart of a casing oil level gauge identification method in another embodiment;

[0039] Figure 8 It is a schematic diagram of the overall process of a casing oil level gauge identification method in one embodiment;

[0040] Fig. 9 is a structural block diagram of a casing oil level gauge identification device in one embodiment;

[0041] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The casing oil level gauge identification method provided in the embodiment of the present application can be applied in the application environment shown in FIG1. ​​Among them, the terminal device 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal device 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, portable wearable devices, and aircraft, etc. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, and projection devices, etc. Portable wearable devices can be smart watches, smart bracelets, and head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 may be an independent physical broadcast control server, or a broadcast control server cluster composed of multiple physical broadcast control servers, or a cloud broadcast control server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Among them, the terminal device 102 and the server 104 can be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiments of the present application.

[0044] Among them, the terminal 102 and the server 104 can be used alone to execute the casing oil level gauge recognition method provided in the embodiment of the present application, and the terminal 102 and the server 104 can also cooperate to execute the casing oil level gauge recognition method provided in the embodiment of the present application. For example, taking the case where the terminal 102 and the server 104 cooperate to execute the casing oil level gauge recognition method provided in the embodiment of the present application as an example, the server 104 obtains the casing oil level gauge image to be identified collected by it from the terminal device 102, and performs casing frame prediction on the casing oil level gauge image according to the trained oil level gauge recognition model, and obtains the casing prediction frame corresponding to the casing oil level gauge image. Among them, the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, and the server 104 specifically trains the first recognition model according to the sample image of the oil level gauge carrying the casing annotation frame, and by cutting and multi-category annotation of the casing annotation frame, the target training data can be obtained, so as to train the second recognition model according to the target training data, wherein the multi-category annotation includes oil gauge plate annotation and liquid level line annotation. Furthermore, the server 104 performs oil gauge plate frame recognition and liquid level line frame recognition on the casing oil level gauge image according to the trained oil level gauge recognition model, so as to obtain the oil gauge plate prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge image, thereby determining the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and liquid level line prediction frame. The server 104 may display the obtained oil level recognition result on the terminal device 102 for the user to view, and may also provide an oil level abnormality prompt when the oil level recognition result is abnormal, and may specifically send an abnormality prompt information to the terminal device 102.

[0045] In an exemplary embodiment, Figure 2 As shown, a casing oil level meter identification method is provided, and the method is applied to the server 104 in FIG. 1 as an example for description, including the following steps S202 to S206. Among them:

[0046] Step S202, obtaining a casing oil level gauge image to be identified, and performing casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model, to obtain a casing prediction frame corresponding to the casing oil level gauge image, wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, and the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame, and the target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train the second recognition model according to the target training data, and the multi-category annotation includes oil gauge plate annotation and liquid level line annotation.

[0047] Specifically, the terminal device can detect the oil level gauge identification request triggered by the user and obtain the casing oil level gauge image to be identified corresponding to the oil level gauge identification request, so as to feed back the oil level gauge identification request carrying the casing oil level gauge image to be identified to the server, and after receiving the oil level gauge identification request, the server parses the oil level gauge identification request to obtain the casing oil level gauge image to be identified carried by the oil level gauge identification request. The casing oil level gauge image to be identified can specifically be an image of multiple categories of casing components collected from a substation, such as a casing oil level gauge image of a substation equipment.

[0048] Furthermore, the server obtains a trained oil level gauge recognition model to perform casing frame prediction on the casing oil level gauge image according to the trained oil level gauge recognition model, and obtains a casing prediction frame corresponding to the casing oil level gauge image. Specifically, the casing oil level gauge image is input into the trained oil level gauge recognition model to perform casing frame prediction on the casing oil level gauge image according to the trained oil level gauge recognition model, that is, casing components that may exist in the casing oil level gauge image are predicted to obtain a casing prediction frame corresponding to the casing oil level gauge image, so that the casing oil level gauge in the casing oil level gauge image can be framed by a rectangular frame.

[0049] The trained oil level gauge recognition model is obtained by combining the trained first recognition model and the trained second recognition model. The trained first recognition model is obtained by marking the casing on the initial image of the oil level gauge with casing with a rectangular frame, obtaining a sample image of the oil level gauge with the casing marked frame, and then training the initial recognition model according to the sample image of the oil level gauge with the casing marked frame.

[0050] In an exemplary embodiment, after obtaining a sample image of an oil level gauge with a casing annotation frame, the server obtains target training data by cropping the casing annotation frame on the sample image of the oil level gauge and performing multi-category annotation, including oil gauge plate annotation and liquid level line annotation, and performs model training on the initial recognition model based on the target training data to obtain a trained second recognition model.

[0051] Among them, the oil gauge plate marking and the liquid level line marking mean that on the basis of the casing marking frame on the casing oil level gauge image, the casing oil level gauge in the casing oil level gauge image is marked in the form of a rectangular frame, and then the casing oil level gauge in the casing oil level gauge image is marked again, including marking the oil gauge plate of the casing oil level gauge in the form of a rectangular frame, and marking the liquid level line in the oil gauge plate of the casing oil level gauge in the form of a rectangular frame. Among them, the rectangular frame corresponding to the liquid level line in the oil gauge plate is located in the rectangular frame corresponding to the oil gauge plate of the casing oil level gauge (that is, the range of the rectangular frame corresponding to the oil gauge plate is larger than the rectangular frame corresponding to the liquid level line), and the liquid level line in the oil gauge plate is located in the middle position of its corresponding rectangular frame.

[0052] Step S204, performing oil gauge panel frame recognition and liquid level line frame recognition on the casing prediction frame according to the oil level gauge recognition model, and obtaining an oil gauge panel prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image.

[0053] Specifically, the server inputs the casing oil level gauge image carrying the casing prediction frame into the trained oil level gauge recognition model, and uses the oil level gauge recognition model to perform oil gauge plate frame recognition and liquid level line frame recognition on the casing prediction frame in the casing oil level gauge image. Specifically, after using the oil level gauge recognition model to predict the rectangular frame corresponding to the casing oil level gauge in the casing oil level gauge image, the server further performs secondary recognition and prediction on the casing oil level gauge defined by the rectangular frame, that is, performs oil gauge plate frame recognition and liquid level line frame recognition on the casing oil level gauge, and obtains the oil gauge plate prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge in the casing oil level gauge image.

[0054] Step S206: determining the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and the liquid level line prediction frame.

[0055] Specifically, the server further determines the height of the oil gauge prediction frame, the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame, and the coordinates of the midpoint of the oil gauge prediction frame for the identified oil gauge prediction frame and the liquid level line prediction frame, so as to determine the proportion of the oil level height corresponding to the casing oil level gauge image based on the coordinates of the midpoint of the oil gauge prediction frame, the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame, and the height of the oil gauge prediction frame, and use the proportion of the oil level height as the oil level recognition result corresponding to the casing oil level gauge image.

[0056] In an exemplary embodiment, after determining the oil level recognition result corresponding to the casing oil level gauge image, the method further includes:

[0057] The oil level recognition results corresponding to the casing oil level gauge image are filtered and screened to obtain the target oil level recognition results that meet the preset display conditions; the preset display conditions include that the resolution of the casing oil level gauge image is greater than the preset resolution threshold, or the oil gauge prediction box is located at the center of the casing oil level gauge image; if it is determined that the target oil level recognition result is less than the preset oil level height, an oil level abnormality prompt is given.

[0058] Specifically, the server filters and selects the oil level recognition results corresponding to the casing oil level gauge image according to preset display conditions, including the preset display conditions that the resolution of the casing oil level gauge image is greater than the preset resolution threshold, or the oil gauge disk prediction frame is located at the center of the casing oil level gauge image. Specifically, the oil level recognition results whose resolution of the casing oil level gauge image is less than or equal to the preset resolution threshold are eliminated and are not output and displayed, or the oil level recognition results whose oil gauge disk prediction frame of the casing oil level gauge is located at the edge of the casing oil level gauge image are eliminated and are not output and displayed, thereby retaining the target oil level recognition results that meet the preset display conditions, that is, retaining the target oil level recognition results whose resolution of the casing oil level gauge image is greater than the preset resolution threshold, or retaining the target oil level recognition results whose oil gauge disk prediction frame is located at the center of the casing oil level gauge image.

[0059] Exemplarily, the oil level recognition results whose resolution of the casing oil level gauge image is less than or equal to a preset resolution threshold (such as 20*20) are discarded and not output or displayed, or the oil level recognition results whose oil gauge disk prediction box of the casing oil level gauge is located at the edge of the casing oil level gauge image (that is, it is not a shooting target) are discarded and not output or displayed.

[0060] Furthermore, the server obtains a pre-set abnormal prompt condition and a preset oil level height corresponding to the abnormal prompt condition to compare the target oil level identification result with the preset oil level height. If it is determined that the target oil level identification result is less than the preset oil level height, it indicates that the abnormal prompt condition is currently met, and the oil level abnormality prompt is performed based on the current target oil level identification result.

[0061] Among them, the abnormal oil level prompt can be specifically provided by displaying abnormal warning information or playing abnormal alarm voice, so that the staff can promptly handle the abnormality and carry out follow-up work such as maintenance. For example, if the preset oil level height is 10% of the oil level, if it is determined that the target oil level recognition result is less than 10% of the preset oil level height, the abnormal oil level prompt is provided, including displaying abnormal warning information or playing abnormal alarm voice. Similarly, if the target oil level recognition result is 10%~100% of the oil level, there is no need to provide an abnormal prompt.

[0062] In the above-mentioned casing oil level gauge recognition method, the casing oil level gauge image to be recognized is obtained, and the casing frame prediction is performed on the casing oil level gauge image according to the trained oil level gauge recognition model, so as to obtain the casing prediction frame corresponding to the casing oil level gauge image, wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, and specifically, the first recognition model is obtained by training the sample image of the oil level gauge carrying the casing annotation frame, and the target training data can be obtained by cropping and multi-category labeling of the casing annotation frame, including oil gauge plate labeling and liquid level line labeling, so as to train the second recognition model according to the target training data. Among them, since a unified rectangular frame labeling method is used, it is not necessary to accurately divide each pixel, which simplifies the labeling process while reducing the mislabeling caused by inconsistent standards, and different labeling categories for the casing oil level gauge image can be obtained at the same time, reducing the probability of mislabeling caused by a single labeling method. Furthermore, according to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge disk frame recognition and liquid level line frame recognition, and the oil gauge disk prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge image can be quickly obtained. Based on the oil gauge disk prediction frame and the liquid level line prediction frame, the oil level recognition result corresponding to the casing oil level gauge image can be accurately obtained, thereby improving the accuracy of the obtained oil level recognition result through multi-category labeling and recognition.

[0063] In an exemplary embodiment, Figure 3 As shown, the step of determining the oil level recognition result corresponding to the casing oil level gauge image, that is, the step of determining the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge disk prediction frame and the liquid level line prediction frame, specifically includes the following steps S302 to S306. Among them:

[0064] Step S302: Determine the height of the fuel gauge prediction frame and the coordinates of the lower boundary midpoint of the fuel gauge prediction frame according to the fuel gauge prediction frame.

[0065] Specifically, the server obtains the height of the oil gauge prediction frame according to the oil gauge prediction frame corresponding to the casing oil level gauge image, that is, specifically obtains the height of the oil gauge of the casing oil level gauge framed by the rectangular frame, and obtains the lower boundary of the oil gauge prediction frame, takes the leftmost side of the lower boundary of the oil gauge prediction frame as the origin of the coordinate system, establishes a rectangular coordinate system, and obtains the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame, thereby obtaining the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame.

[0066] Step S304, determining the midpoint coordinates of the liquid level line prediction frame according to the liquid level line prediction frame.

[0067] Specifically, the server determines the midpoint of the liquid level line prediction box according to the liquid level line prediction box corresponding to the casing oil level gauge image, and establishes a rectangular coordinate system by taking the leftmost side of the lower boundary of the oil gauge prediction box as the origin of the coordinate system, and determines the midpoint coordinates of the liquid level line prediction box based on the established rectangular coordinate system.

[0068] Among them, since the liquid level line in the oil gauge is located in the middle position of its corresponding rectangular frame, that is, the liquid level line in the oil gauge is located in the middle position of the liquid level line prediction frame, the midpoint coordinates of the liquid level line prediction frame can be understood as the midpoint of the liquid level line in the oil gauge.

[0069] Step S306, based on the midpoint coordinates of the liquid level line prediction box, the midpoint coordinates of the lower boundary of the oil gauge prediction box, and the height of the oil gauge prediction box, determine the oil level height ratio corresponding to the casing oil level gauge image, and use the oil level height ratio as the oil level recognition result corresponding to the casing oil level gauge image.

[0070] Specifically, based on the midpoint coordinates of the liquid level line prediction box, the midpoint coordinates of the lower boundary of the oil gauge prediction box, and the height of the oil gauge prediction box, the oil level height ratio is calculated to obtain the oil level height ratio corresponding to the casing oil level gauge image, and the determined oil level height ratio is used as the oil level recognition result corresponding to the casing oil level gauge image.

[0071] For example, the oil level height ratio corresponding to the casing oil level gauge image is determined by the following formula (1):

[0072] ;Formula (1)

[0073] Among them, referring to formula (1), it can be known that the absolute difference between the midpoint coordinates of the liquid level line prediction frame and the midpoint coordinates of the lower boundary of the oil gauge prediction frame is calculated, and the ratio between the absolute difference and the height of the oil gauge prediction frame is calculated, and then the percentage conversion is performed to obtain the oil level height ratio corresponding to the casing oil level gauge image.

[0074] In an exemplary embodiment, Figure 4 As shown, a schematic diagram of the oil gauge prediction frame and the liquid level line prediction frame corresponding to the casing oil level gauge image is provided. Figure 4 It can be seen that after the casing oil level gauge on the casing oil level gauge image is framed, secondary recognition and prediction are performed based on the framed casing oil level gauge, and the oil gauge disk prediction frame indicated by rectangular frame 1 and the liquid level line prediction frame indicated by rectangular frame 2 can be obtained. Among them, rectangular frame 2 and rectangular frame 1 do not overlap, the liquid level line prediction frame indicated by rectangular frame 2 is located in the oil gauge disk prediction frame indicated by rectangular frame 1, the range of the oil gauge disk prediction frame indicated by rectangular frame 1 is larger than the liquid level line prediction frame indicated by rectangular frame 2, and the liquid level line in the oil gauge disk is located in the middle of the liquid level line prediction frame indicated by rectangular frame 2.

[0075] Furthermore, the lower boundary of the oil gauge prediction box indicated by the rectangular frame 1 can be obtained, and the leftmost side of the lower boundary of the oil gauge prediction box is used as the origin of the coordinate system to establish a rectangular coordinate system (xy coordinate system), so that the coordinates of the midpoint of the lower boundary of the oil gauge prediction box can be obtained, so as to obtain the coordinates of the midpoint of the lower boundary of the oil gauge prediction box, such as h1 (x1, y1).

[0076] Similarly, on the basis of establishing a rectangular coordinate system by taking the leftmost side of the lower boundary of the liquid level line prediction box indicated by the rectangular frame 2 as the origin of the coordinate system, the midpoint coordinates of the liquid level line prediction box, such as h2 (x2, y2), are determined based on the established rectangular coordinate system. After obtaining the height H of the oil gauge plate of the casing oil level gauge framed by the rectangular frame 1, the oil level height ratio is calculated based on the midpoint coordinates h1 (x1, y1) of the liquid level line prediction box, the midpoint coordinates h2 (x2, y2) of the lower boundary of the oil gauge plate prediction box, and the height H of the oil gauge plate prediction box, to obtain the oil level height ratio corresponding to the casing oil level gauge image, and the determined oil level height ratio is used as the oil level recognition result corresponding to the casing oil level gauge image.

[0077] Exemplarily, by calculating the absolute difference between the midpoint coordinates h2 (x2, y2) of the liquid level line prediction box and the midpoint coordinates h1 (x1, y1) of the lower boundary of the oil gauge prediction box, and calculating the ratio between the absolute difference and the height H of the oil gauge prediction box, percentage conversion is performed to obtain the oil level height ratio corresponding to the casing oil level gauge image.

[0078] In this embodiment, the height of the oil gauge prediction frame and the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame are determined according to the oil gauge prediction frame, and the midpoint coordinates of the liquid level line prediction frame are determined according to the liquid level line prediction frame. Based on the midpoint coordinates of the liquid level line prediction frame, the midpoint coordinates of the lower boundary of the oil gauge prediction frame, and the height of the oil gauge prediction frame, the oil level height ratio corresponding to the casing oil level gauge image is determined, and the oil level height ratio is used as the oil level recognition result corresponding to the casing oil level gauge image, so that different annotation categories for the casing oil level gauge image can be obtained at the same time, reducing the probability of mis-annotation caused by a single annotation method, and determining the oil level height ratio based on the annotation and recognition results of multiple categories, thereby improving the accuracy of the obtained oil level recognition result.

[0079] In an exemplary embodiment, Figure 5 As shown, the method of training to obtain the second recognition model specifically includes the following steps S502 to S506. Among them:

[0080] Step S502: enlarge and crop the casing marking frame on the casing oil level gauge image to obtain an intermediate casing oil level gauge image.

[0081] Specifically, after the server marks the casing on the initial image with a rectangular frame and obtains a sample image of the oil level gauge with the casing mark frame, the casing mark frame on the sample image of the oil level gauge is enlarged and cropped, for example, the casing mark frame is enlarged 1.5 times and cropped to obtain an intermediate casing oil level gauge image with the enlarged casing mark frame. Among them, the server obtains multiple power equipment images collected by drone inspections, screens multiple power equipment images, and obtains an initial image of the oil level gauge with the casing, and by marking the casing on the initial image with a rectangular frame, a sample image of the oil level gauge with the casing mark frame can be obtained.

[0082] The enlargement multiple of the casing marking frame can be set and adjusted according to actual needs, such as 1.8 times, 2 times, or 2.5 times, etc., without specific limitation, to obtain the intermediate casing oil level gauge image carrying the enlarged casing marking frame. Step S504: Based on the intermediate casing oil level gauge image, the oil gauge plate and the liquid level line are marked to obtain the target casing oil level gauge image carrying the oil gauge plate marking frame and the liquid level line marking frame.

[0083] Specifically, the server performs multi-category annotation on the intermediate casing oil level gauge image carrying the expanded casing annotation frame, including oil gauge plate annotation and liquid level line annotation, that is, the casing oil level gauge in the casing oil level gauge image is annotated in the form of a rectangular frame, and after expansion and cropping, the intermediate casing oil level gauge image is obtained, and secondary annotation is performed on the casing oil level gauge in the intermediate casing oil level gauge image, including annotating the oil gauge plate of the casing oil level gauge in the form of a rectangular frame, and annotating the liquid level line in the oil gauge plate of the casing oil level gauge in the form of a rectangular frame, to obtain a target casing oil level gauge image carrying the oil gauge plate annotation frame and the liquid level line annotation frame.

[0084] Step S506, performing data enhancement processing on the target casing oil level gauge image to obtain target training data, and performing model training on the initial recognition model based on the target training data, and obtaining a trained second recognition model when the model training end condition is met.

[0085] Specifically, data enhancement processing is performed on the target casing oil level gauge image, including image flipping, random image cropping, random contrast, random scaling and then random distribution for splicing, etc., to obtain target training data. Among them, the random scaling and then random distribution for splicing can be specifically to randomly use N pictures, first perform random scaling and then use random distribution for splicing, and N can be set and adjusted according to actual needs, such as N is 5, 9, 12, and other different values.

[0086] Furthermore, the server performs model training on the initial recognition model according to the target training data, and determines the target confidence loss, classification loss and positioning loss in the model training process, and obtains the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss respectively, and determines the model training loss according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss respectively, the target confidence loss, classification loss and positioning loss.

[0087] Among them, the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss respectively include the first weight coefficient corresponding to the target confidence loss, the second weight coefficient corresponding to the classification loss, and the third weight coefficient corresponding to the positioning loss, and specifically, the weighted sum is performed according to the target confidence loss, the first weight coefficient, the classification loss, the second weight coefficient, the positioning loss and the third weight coefficient to determine the model training loss. Among them, the first weight coefficient, the second weight coefficient and the third weight coefficient can be set according to actual needs, and the sum of the first weight coefficient, the second weight coefficient and the third weight coefficient is 1.

[0088] Furthermore, the model training loss Total Loss can be determined using the following formula (2):

[0089] Total Loss=λ1* Objectness Loss+λ2* Classification Loss+λ3*Localization Loss; Formula (2)

[0090] Among them, Total Loss is the model training loss, Objectness Loss is the target confidence loss, λ1 is the first weight coefficient, Classification Loss is the classification loss, λ2 is the second weight coefficient, Localization Loss is the positioning loss, and λ3 is the third weight coefficient.

[0091] For example, if the first weight coefficient is 0.2, the second weight coefficient is 0.5, and the third weight coefficient is 0.3, then the model training loss Total Loss is determined by weighted summation of the object confidence loss Objectness Loss, the first weight coefficient 0.2, the classification loss Classification Loss, the second weight coefficient 0.5, the positioning loss Localization Loss, and the third weight coefficient 0.3.

[0092] Furthermore, when it is determined that the model training loss meets the model training end condition, the training is stopped to obtain a trained second recognition model. In the model training process, specifically, by obtaining a training loss threshold corresponding to the model training end condition, and comparing the model training loss with the training loss threshold, when it is determined that the model training loss reaches the training loss threshold, it is determined that the model training end condition is currently met, then the training is stopped to obtain a trained second recognition model.

[0093] In an exemplary embodiment, a corresponding model training iteration number threshold can be set according to the model training end condition, and during the model training process, the model training iteration number is counted, and the training iteration number is compared with the model training iteration number threshold, with the goal of minimizing the value of the loss function. When it is determined that the number of training iterations reaches the model training iteration number threshold, it means that the model training end condition is currently met, then the training is stopped to obtain a trained second recognition model.

[0094] In this embodiment, the casing oil level gauge image is obtained by enlarging and cropping the casing annotation frame on the casing oil level gauge image, and the oil gauge plate and the liquid level line are annotated based on the intermediate casing oil level gauge image to obtain a target casing oil level gauge image carrying the oil gauge plate annotation frame and the liquid level line annotation frame, so that the target casing oil level gauge image can be data enhanced to obtain target training data, so as to perform model training on the initial recognition model according to the target training data, and obtain a trained second recognition model when the model training end condition is met, so as to realize the method of enlarging and cropping the casing annotation frame on the casing oil level gauge image, intercepting the effective area on the casing oil level gauge image, focusing on the casing oil level gauge itself, reducing background interference, and providing more pure and targeted training data for model training, thereby improving the recognition accuracy and generalization ability of the obtained trained second recognition model, and reducing the false detection rate of the oil level proportion height recognition of the oil level gauge using the second recognition model.

[0095] In an exemplary embodiment, Figure 6 As shown, the method of training to obtain the first recognition model specifically includes the following steps S602 to S608. Among them:

[0096] Step S602, acquiring multiple power equipment images, screening the multiple power equipment images, and obtaining an initial image of the oil level gauge of the casing.

[0097] Specifically, the server obtains substation images collected by drone inspections, extracts multiple power equipment images involved in the substation images, and filters the multiple power equipment images to select an initial image with a casing oil level gauge.

[0098] Step S604: annotate the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge carrying the casing annotation frame.

[0099] Specifically, the server uses an image annotation tool (such as LabelImg software, i.e., a target detection and annotation tool) to annotate the casing on the initial image with a rectangular frame for the selected initial image with the casing oil level gauge, and obtains a sample image of the oil level gauge with the casing annotation frame. Among them, by extracting the outer contours of each casing component (such as the casing oil level gauge) in the high-resolution image, the amount of information of the model in the feature extraction stage can be increased, thereby effectively narrowing the recognition range of the casing oil level gauge and improving the recognition accuracy.

[0100] Step S606: perform data enhancement processing on the oil level gauge sample image to obtain an oil level gauge sample image training set.

[0101] Specifically, the server performs data enhancement processing on the sample images of the oil level gauge, including image flipping, random image cropping, random contrast, and random scaling followed by random distribution for splicing, to obtain a training set of sample images of the oil level gauge. Among them, the random scaling followed by random distribution for splicing may specifically be to randomly use N pictures, first perform random scaling and then use random distribution for splicing, and N may be set and adjusted according to actual needs, such as N being 5, 9, 12, or other different values.

[0102] The oil level gauge sample images are subjected to data enhancement processing to obtain the oil level gauge sample image training set, and the oil level gauge image test set is obtained at the same time, which is used to test and verify the trained first recognition model to determine whether it meets the actual needs. The specific division ratio of the oil level gauge sample image training set and the oil level gauge image test set can be 8:2 or 7:3, and the division ratio can be set and adjusted according to actual needs.

[0103] Step S608, training the initial recognition model according to the oil level gauge sample image training set, and obtaining a trained first recognition model when the model training end condition is met.

[0104] Specifically, the initial recognition model is trained according to the oil level gauge sample image training set, and the target confidence loss, classification loss and positioning loss in the model training process are determined. The model training loss is determined according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss, as well as the target confidence loss, classification loss and positioning loss.

[0105] Among them, the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss respectively include the first weight coefficient corresponding to the target confidence loss, the second weight coefficient corresponding to the classification loss, and the third weight coefficient corresponding to the positioning loss, and specifically, the weighted sum is performed according to the target confidence loss, the first weight coefficient, the classification loss, the second weight coefficient, the positioning loss and the third weight coefficient to determine the model training loss. Among them, the first weight coefficient, the second weight coefficient and the third weight coefficient can be set according to actual needs, and the sum of the first weight coefficient, the second weight coefficient and the third weight coefficient is 1.

[0106] Furthermore, when it is determined that the model training loss meets the model training end condition, the training is stopped to obtain the trained first recognition model. In the model training process, specifically, by obtaining the training loss threshold corresponding to the model training end condition, and comparing the model training loss with the training loss threshold, when it is determined that the model training loss reaches the training loss threshold, it is determined that the model training end condition is currently met, then the training is stopped to obtain the trained first recognition model.

[0107] In an exemplary embodiment, a corresponding model training iteration number threshold can be set according to the model training end condition, and during the model training process, the model training iteration number is counted, and the training iteration number is compared with the model training iteration number threshold, with the goal of minimizing the value of the loss function. When it is determined that the number of training iterations reaches the model training iteration number threshold, it means that the model training end condition is currently met, then the training is stopped to obtain the trained first recognition model.

[0108] In an exemplary embodiment, the initial recognition model may be a target detection model YOLOv8, wherein the initial recognition model may be a target detection model YOLOv8 constructed based on a target detection network and an instance segmentation model with different resolutions (including resolutions of 640 and 1280). The target detection model YOLOv8 provides models of different scales such as N, S, M, L, and XL based on a scaling factor, which can meet the needs of different deployment platforms and application scenarios.

[0109] Specifically, the Backbone (i.e., backbone network) of the target detection model YOLOv8 replaces the C3 module in the target detection model Yolov5 with the C2f module, achieving further lightweighting, while continuing to use the SPPF module in the target detection model Yolov5, and fine-tuning the models of different scales, greatly improving the model performance. The Neck (i.e., neck network) of the target detection model YOLOv8 is based on the target detection model Yolov5, but removes the 1*1 downsampling layer. The Head part (i.e., the head network) of the target detection model YOLOv8 is a decoupled head structure (i.e., Decoupled-Head), which separates the classification and detection heads, and also changes from Anchor-Based to Anchor-Free (anchor-free and anchor-based represent different target detection methods, the difference lies in whether a predefined anchor box is used to match the real target box. The anchor-based method uses anchor boxes of different sizes and shapes to regress and classify targets, while the anchor-free method does not require preset anchors, but only needs to regress the target center point and width and height of feature maps of different scales). Among them, the target detection model Yolov8 does not use the traditional IoU allocation or unilateral ratio allocation method, but uses the Task-Aligned Assigner positive and negative sample allocation strategy for label allocation.

[0110] In this embodiment, by acquiring multiple power equipment images, screening the multiple power equipment images, obtaining an initial image of an oil level gauge with a casing, and marking the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge with a casing marking frame, and further performing data enhancement processing on the sample image of the oil level gauge, a training set of sample images of the oil level gauge can be obtained, and the initial recognition model can be trained according to the training set of sample images of the oil level gauge. When the model training end condition is met, a trained first recognition model is obtained, and by using a unified rectangular frame marking method without the need to accurately divide each pixel, while reducing the mis-marking caused by inconsistent standards, the marking process is simplified, and the quality and consistency of the marked data are ensured. At the same time, the use of a unified marking standard helps to enhance the recognition effect of the prediction frame of the small sample category, and can better learn and generalize, and can maintain a high recognition accuracy even when the sample size is small, thereby improving the recognition accuracy and generalization ability of the obtained trained first recognition model.

[0111] In an exemplary embodiment, Figure 7 As shown, a casing oil level meter identification method is provided, and the method is applied to the server 104 in FIG. 1 as an example for description, including the following steps S701 to S718. Among them:

[0112] Step S701, acquiring multiple power equipment images, screening the multiple power equipment images, and obtaining an initial image of the oil level gauge of the casing.

[0113] Step S702 , annotate the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge carrying the casing annotation frame.

[0114] Step S703: perform data enhancement processing on the oil level gauge sample image to obtain an oil level gauge sample image training set.

[0115] Step S704: train the initial recognition model according to the oil level gauge sample image training set, and determine the target confidence loss, classification loss and positioning loss in the model training process.

[0116] Step S705, determining the model training loss according to the weight coefficients corresponding to the target confidence loss, the classification loss and the positioning loss, respectively, and the target confidence loss, the classification loss and the positioning loss.

[0117] Step S706, when the model training loss meets the model training end condition, stop the training and obtain the trained first recognition model.

[0118] Step S707, enlarging and cropping the casing marking frame on the casing oil level gauge image to obtain an intermediate casing oil level gauge image.

[0119] Step S708: Based on the intermediate casing oil level gauge image, the oil gauge plate and the liquid level line are marked to obtain the target casing oil level gauge image with the oil gauge plate marking frame and the liquid level line marking frame.

[0120] Step S709, performing data enhancement processing on the target casing oil level gauge image to obtain target training data, and performing model training on the initial model according to the target training data, and obtaining a trained second recognition model when the model training end condition is met.

[0121] Step S710, an oil level gauge recognition model is obtained according to the combination of the trained first recognition model and the second recognition model, a casing oil level gauge image to be recognized is obtained, and a casing frame prediction is performed on the casing oil level gauge image according to the trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image.

[0122] Step S711, according to the oil level gauge recognition model, the oil gauge disk frame recognition and the liquid level line frame recognition are performed on the casing oil level gauge image to obtain the oil gauge disk prediction frame and the liquid level line prediction frame corresponding to the casing oil level gauge image.

[0123] Step S712, determining the height of the fuel gauge prediction frame and the coordinates of the lower boundary midpoint of the fuel gauge prediction frame according to the fuel gauge prediction frame.

[0124] Step S713, determining the midpoint coordinates of the liquid level line prediction frame according to the liquid level line prediction frame.

[0125] Step S714, based on the midpoint coordinates of the liquid level line prediction box, the midpoint coordinates of the lower boundary of the oil gauge prediction box, and the height of the oil gauge prediction box, determine the oil level height ratio corresponding to the casing oil level gauge image, and use the oil level height ratio as the oil level recognition result corresponding to the casing oil level gauge image.

[0126] Step S715, filtering and selecting the oil level recognition results corresponding to the casing oil level gauge image to obtain target oil level recognition results that meet preset display conditions, wherein the preset display conditions include that the resolution of the casing oil level gauge image is greater than a preset resolution threshold, or the oil gauge prediction box is located at the center of the casing oil level gauge image.

[0127] Step S716: If it is determined that the target oil level identification result is less than the preset oil level height, an oil level abnormality prompt is issued.

[0128] In an exemplary embodiment, Figure 8 As shown, an overall processing flow of a casing oil level meter identification method is provided. Figure 8 It can be seen that the casing oil level gauge identification method specifically includes the following processing steps:

[0129] 1) Obtain the substation image inspected by the drone, extract multiple power equipment images involved in the substation image, filter the multiple power equipment images, and filter out the initial image with the casing oil level gauge.

[0130] 2) For the selected initial images with casing oil level gauges, use the image annotation tool to annotate the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge with a casing annotation frame.

[0131] 3) The oil level gauge sample images are augmented by image flipping, random image cropping, random contrast, and random scaling followed by random distribution and splicing to obtain the oil level gauge sample image training set.

[0132] 4) The initial recognition model is trained according to the oil level gauge sample image training set, and when the model training end condition is met, a trained first recognition model is obtained.

[0133] 5) The casing marking frame on the casing oil level gauge image is enlarged and cropped to obtain an intermediate casing oil level gauge image. Based on the intermediate casing oil level gauge image, the oil gauge plate and the liquid level line are marked to obtain a target casing oil level gauge image with the oil gauge plate marking frame and the liquid level line marking frame.

[0134] 6) Performing data enhancement processing on the target casing oil level gauge image to obtain target training data, and performing model training on the initial recognition model based on the target training data. When the model training end condition is met, a trained second recognition model is obtained.

[0135] 7) An oil level gauge recognition model is obtained by combining the first recognition model and the second recognition model. According to the oil level gauge recognition model, the casing oil level gauge image is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image.

[0136] 8) Based on the oil gauge prediction frame and the liquid level line prediction frame, determine the oil level recognition result corresponding to the casing oil level gauge image.

[0137] 9) Filter and screen the oil level recognition results corresponding to the casing oil level gauge image to obtain the target oil level recognition results that meet the preset display conditions. If it is determined that the target oil level recognition result is less than the preset oil level height, an oil level abnormality prompt is given.

[0138] In the above-mentioned casing oil level gauge recognition method, the casing oil level gauge image to be recognized is obtained, and the casing frame prediction is performed on the casing oil level gauge image according to the trained oil level gauge recognition model, so as to obtain the casing prediction frame corresponding to the casing oil level gauge image, wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, and specifically, the first recognition model is obtained by training the sample image of the oil level gauge carrying the casing annotation frame, and the target training data can be obtained by cropping and multi-category labeling of the casing annotation frame, including oil gauge plate labeling and liquid level line labeling, so as to train the second recognition model according to the target training data. Among them, since a unified rectangular frame labeling method is used, it is not necessary to accurately divide each pixel, which simplifies the labeling process while reducing the mislabeling caused by inconsistent standards, and different labeling categories for the casing oil level gauge image can be obtained at the same time, reducing the probability of mislabeling caused by a single labeling method. Furthermore, according to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge disk frame recognition and liquid level line frame recognition, and the oil gauge disk prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge image can be quickly obtained. Based on the oil gauge disk prediction frame and the liquid level line prediction frame, the oil level recognition result corresponding to the casing oil level gauge image can be accurately obtained, thereby improving the accuracy of the obtained oil level recognition result through multi-category labeling and recognition.

[0139] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0140] Based on the same inventive concept, the embodiment of the present application also provides a casing oil level gauge identification device for implementing the casing oil level gauge identification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more casing oil level gauge identification device embodiments provided below can refer to the limitations of the casing oil level gauge identification method above, and will not be repeated here.

[0141] In an exemplary embodiment, Fig. 9 As shown, a casing oil level meter identification device is provided, comprising: a first prediction frame obtaining module 902, a second prediction frame obtaining module 904, and an oil level identification result determination module 906, wherein:

[0142] The first prediction frame acquisition module 902 is used to obtain a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to the trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training the sample image of the oil level gauge carrying the casing annotation frame; the target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train the second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation;

[0143] The second prediction frame obtaining module 904 is used to perform oil gauge frame recognition and liquid level line frame recognition on the casing prediction frame according to the oil level gauge recognition model, and obtain the oil gauge frame and liquid level line prediction frame corresponding to the casing oil level gauge image;

[0144] The oil level recognition result determination module 906 is used to determine the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and the liquid level line prediction frame.

[0145] In the above-mentioned casing oil level gauge recognition device, by obtaining the casing oil level gauge image to be recognized, and predicting the casing frame of the casing oil level gauge image according to the trained oil level gauge recognition model, a casing prediction frame corresponding to the casing oil level gauge image is obtained, wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model, specifically, the first recognition model is trained according to the sample image of the oil level gauge carrying the casing annotation frame, and the target training data can be obtained by cropping and multi-category labeling of the casing annotation frame, including oil gauge plate labeling and liquid level line labeling, so as to train the second recognition model according to the target training data. Among them, due to the use of a unified rectangular frame labeling method, there is no need to accurately divide each pixel, which simplifies the labeling process while reducing the mislabeling caused by inconsistent standards, and can simultaneously obtain different labeling categories for the casing oil level gauge image, reducing the probability of mislabeling caused by a single labeling method. Furthermore, according to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge disk frame recognition and liquid level line frame recognition, and the oil gauge disk prediction frame and liquid level line prediction frame corresponding to the casing oil level gauge image can be quickly obtained. Based on the oil gauge disk prediction frame and the liquid level line prediction frame, the oil level recognition result corresponding to the casing oil level gauge image can be accurately obtained, thereby improving the accuracy of the obtained oil level recognition result through multi-category labeling and recognition.

[0146] In an exemplary embodiment, the oil level identification result determination module is also used to: determine the height of the oil gauge plate prediction frame and the coordinates of the lower boundary midpoint of the oil gauge plate prediction frame according to the oil gauge plate prediction frame; determine the midpoint coordinates of the liquid level line prediction frame according to the liquid level line prediction frame; determine the oil level height ratio corresponding to the casing oil level gauge image based on the midpoint coordinates of the liquid level line prediction frame, the midpoint coordinates of the lower boundary midpoint of the oil gauge plate prediction frame, and the height of the oil gauge plate prediction frame, and use the oil level height ratio as the oil level identification result corresponding to the casing oil level gauge image.

[0147] In an exemplary embodiment, a casing oil level gauge recognition device is provided, which also includes a first recognition model acquisition module, which is used to: acquire multiple power equipment images, screen the multiple power equipment images, and obtain an initial image carrying a casing oil level gauge; mark the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge carrying the casing marking frame; perform data enhancement processing on the oil level gauge sample image to obtain a training set of oil level gauge sample images; train the initial recognition model based on the oil level gauge sample image training set, and obtain a trained first recognition model when the model training end condition is met.

[0148] In an exemplary embodiment, the first recognition model acquisition module is also used to: train the initial recognition model according to the oil level gauge sample image training set, and determine the target confidence loss, classification loss and positioning loss during the model training process; determine the model training loss according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss, as well as the target confidence loss, classification loss and positioning loss; stop training when the model training loss meets the model training end condition to obtain the trained first recognition model.

[0149] In an exemplary embodiment, a casing oil level gauge recognition device is provided, further comprising a second recognition model acquisition module, which is used to: expand and crop a casing annotation frame on a casing oil level gauge image to obtain an intermediate casing oil level gauge image; perform oil gauge plate annotation and liquid level line annotation based on the intermediate casing oil level gauge image to obtain a target casing oil level gauge image carrying an oil gauge plate annotation frame and a liquid level line annotation frame; perform data enhancement processing on the target casing oil level gauge image to obtain target training data, and perform model training on an initial model based on the target training data, and obtain a trained second recognition model when a model training end condition is met.

[0150] In an exemplary embodiment, the second recognition model acquisition module is also used to: perform model training on the initial recognition model according to the target training data, and determine the target confidence loss, classification loss and positioning loss during the model training process; determine the model training loss according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss, as well as the target confidence loss, classification loss and positioning loss; when the model training loss meets the model training end condition, stop the training and obtain the trained second recognition model.

[0151] In an exemplary embodiment, a casing oil level gauge identification device is provided, further comprising an abnormal prompt module, which is used to: filter and screen the oil level identification results corresponding to the casing oil level gauge image to obtain a target oil level identification result that meets a preset display condition; the preset display condition includes that the resolution of the casing oil level gauge image is greater than a preset resolution threshold, or the oil gauge disk prediction box is located at the center of the casing oil level gauge image; if it is determined that the target oil level identification result is less than a preset oil level height, an oil level abnormality prompt is issued.

[0152] Each module in the casing oil level gauge identification device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0153] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as a casing oil level gauge image, a trained oil level gauge recognition model, a casing prediction frame, an oil level gauge sample image carrying a casing annotation frame, a trained first recognition model, target training data, a trained second recognition model, an oil gauge plate prediction frame, a liquid level line prediction frame, and an oil level recognition result. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a casing oil level gauge recognition method is implemented.

[0154] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0156] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0157] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0159] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A casing oil level gauge identification method, It is characterized in that The method comprises: Acquire a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation; According to the oil level gauge recognition model, the casing prediction frame is subjected to oil gauge plate frame recognition and liquid level line frame recognition to obtain an oil gauge plate prediction frame and a liquid level line prediction frame corresponding to the casing oil level gauge image; Based on the oil gauge prediction frame and the liquid level line prediction frame, an oil level recognition result corresponding to the casing oil level gauge image is determined.

2. The method according to claim 1, It is characterized in that The step of determining the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and the liquid level line prediction frame includes: According to the oil gauge prediction frame, determining the height of the oil gauge prediction frame and the coordinates of the midpoint of the lower boundary of the oil gauge prediction frame; According to the liquid level line prediction frame, determining the midpoint coordinates of the liquid level line prediction frame; Based on the midpoint coordinates of the liquid level line prediction box, the midpoint coordinates of the lower boundary of the oil gauge prediction box, and the height of the oil gauge prediction box, the oil level height ratio corresponding to the casing oil level gauge image is determined, and the oil level height ratio is used as the oil level recognition result corresponding to the casing oil level gauge image.

3. The method according to claim 1, It is characterized in that The first recognition model obtained by training includes: Acquire multiple power equipment images, filter the multiple power equipment images, and obtain an initial image of the oil level gauge of the casing; Marking the casing on the initial image with a rectangular frame to obtain a sample image of the oil level gauge carrying the casing marking frame; Performing data enhancement processing on the oil level gauge sample image to obtain an oil level gauge sample image training set; The initial recognition model is trained according to the oil level gauge sample image training set, and when the model training end condition is met, a trained first recognition model is obtained.

4. The method according to claim 3, It is characterized in that The initial recognition model is trained according to the oil level gauge sample image training set, and when the model training end condition is met, a trained first recognition model is obtained, including: According to the oil level gauge sample image training set, the initial recognition model is trained to determine the target confidence loss, classification loss and positioning loss in the model training process; Determine the model training loss according to the weight coefficients corresponding to the target confidence loss, classification loss and positioning loss, respectively, and the target confidence loss, classification loss and positioning loss; When the model training loss meets the model training end condition, the training is stopped to obtain the trained first recognition model.

5. The method according to claim 3 or 4, It is characterized in that Methods for training the second recognition model include: Expanding and cropping the casing marking frame on the casing oil level gauge image to obtain an intermediate casing oil level gauge image; Based on the intermediate casing oil level gauge image, an oil gauge plate and a liquid level line are marked to obtain a target casing oil level gauge image with an oil gauge plate marking frame and a liquid level line marking frame; Data enhancement processing is performed on the target casing oil level gauge image to obtain target training data, and model training is performed on the initial model based on the target training data. When a model training end condition is met, a trained second recognition model is obtained.

6. The method according to any one of claims 1 to 3, It is characterized in that The method further comprises: Filtering and screening the oil level recognition results corresponding to the casing oil level gauge image to obtain a target oil level recognition result that meets a preset display condition; the preset display condition includes that the resolution of the casing oil level gauge image is greater than a preset resolution threshold, or the oil gauge disk prediction frame is located at the center of the casing oil level gauge image; If it is determined that the target oil level identification result is less than the preset oil level height, an oil level abnormality prompt is given.

7. A casing oil level gauge identification device, It is characterized in that The device comprises: The first prediction frame acquisition module is used to obtain a casing oil level gauge image to be identified, and perform casing frame prediction on the casing oil level gauge image according to a trained oil level gauge recognition model to obtain a casing prediction frame corresponding to the casing oil level gauge image; wherein the oil level gauge recognition model is obtained by combining the trained first recognition model and the second recognition model; the first recognition model is obtained by training according to a sample image of the oil level gauge carrying a casing annotation frame; target training data is obtained by cropping and multi-category annotating the casing annotation frame, so as to train a second recognition model according to the target training data; the multi-category annotation includes oil gauge plate annotation and liquid level line annotation; A second prediction frame obtaining module is used to perform oil gauge frame recognition and liquid level line frame recognition on the casing prediction frame according to the oil level gauge recognition model, and obtain an oil gauge frame and a liquid level line prediction frame corresponding to the casing oil level gauge image; The oil level recognition result determination module is used to determine the oil level recognition result corresponding to the casing oil level gauge image based on the oil gauge plate prediction frame and the liquid level line prediction frame.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Key point detection method, neural network training method, devices and electronic apparatus

    CN108229490A

  • Power transformer bushing intelligent oil level identification method based on infrared image

    CN111639579A

  • Target detection and model training method and device, computer equipment and storage medium

    CN111950329A

  • Method and system implemented based on neutral network

    CN113705585A

  • Intelligent identification method, system and device for liquid level instrument and storage medium

    CN118429974A