Power grid special equipment operation detection method, computer equipment, readable storage medium and program product

Abstract

The invention relates to a power grid special equipment operation detection method, computer equipment, a readable storage medium and a program product, and is applied to the technical field of big data, and the method comprises the steps: obtaining an equipment image, and carrying out the target detection of the equipment image, and obtaining a plurality of local frame selection images; according to the image classification result of each local frame selection image, determining a target detection frame from each local frame selection image; according to the target detection frame, positioning an actual space environment position, and identifying the equipment type of the power grid special equipment; identifying the equipment identifier to obtain a special equipment identifier, and identifying the face of the operator in the equipment image according to the target detection frame to obtain a face identification result; according to the face recognition result, the equipment type of the power grid special equipment, the special equipment identifier of the power grid special equipment and the actual space environment position of the power grid special equipment, operation specification detection is carried out, and an operation specification detection result is obtained. The method can improve the detection efficiency of the operation of the special equipment of the power grid.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and particularly to a method for detecting the operation of special equipment in a power grid, a computer device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Special equipment in a power grid refers to equipment that involves life safety and has relatively high risks in the power grid system. For example, boilers, pressure vessels, pressure pipelines, elevators, cranes, passenger ropeways, large-scale amusement facilities, and special-purpose motor vehicles in factories (plants), etc. To ensure the safety of using special equipment in a power grid, relatively strict requirements are also imposed on the special equipment in the power grid.

[0003] Currently, manual inspection is relied on to detect the operation of special equipment in a power grid. However, this method has a large workload and low efficiency. In addition, the time for manual inspection is relatively fixed, and due to the limitations of human physiology, there may be a situation where operation problems of special equipment in a power grid cannot be detected in a timely manner during times such as early morning when the intensity of manual inspection is relatively loose, resulting in a low detection efficiency for the operation of special equipment in a power grid. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting the operation of special equipment in a power grid that can improve the detection efficiency of the operation of special equipment in a power grid.

[0005] In a first aspect, the present application provides a method for detecting the operation of special equipment in a power grid, including:

[0006] Obtaining an equipment image of special equipment in a power grid, and performing object detection on the equipment image to obtain a plurality of locally framed images;

[0007] Determining a target detection frame from each of the locally framed images according to the image classification result of each of the locally framed images;

[0008] Locating the actual spatial environment position of the special equipment in a power grid according to the target detection frame, and identifying the equipment type of the special equipment in a power grid;

[0009] Identifying the equipment identifier in the target detection frame to obtain a special equipment identifier, and, according to the target detection frame, identifying the face of the operator in the equipment image to obtain a face recognition result;

[0010] Performing operation specification detection on the special equipment in a power grid according to the face recognition result, the equipment type of the special equipment in a power grid, the special equipment identifier of the special equipment in a power grid, and the actual spatial environment position of the special equipment in a power grid to obtain an operation specification detection result.

[0011] In a second aspect, the present application also provides an operating detection device for special equipment of a power grid, including:

[0012] An acquisition module, configured to acquire an equipment image of special equipment of a power grid, and perform object detection on the equipment image to obtain a plurality of locally framed images;

[0013] A determination module, configured to determine a target detection frame from each of the locally framed images according to the image classification results of each of the locally framed images;

[0014] An identification module, configured to locate the actual spatial environment position of the special equipment of the power grid and identify the equipment type of the special equipment of the power grid according to the target detection frame; identify the equipment identifier in the target detection frame to obtain a special equipment identifier, and, according to the target detection frame, identify the face of the operator in the equipment image to obtain a face recognition result;

[0015] A detection module, configured to perform an operation specification detection on the special equipment of the power grid according to the face recognition result, the equipment type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid to obtain an operation specification detection result.

[0016] In a third aspect, the present application also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] Acquire an equipment image of special equipment of a power grid, and perform object detection on the equipment image to obtain a plurality of locally framed images;

[0018] Determine a target detection frame from each of the locally framed images according to the image classification results of each of the locally framed images;

[0019] Locate the actual spatial environment position of the special equipment of the power grid and identify the equipment type of the special equipment of the power grid according to the target detection frame;

[0020] Identify the equipment identifier in the target detection frame to obtain a special equipment identifier, and, according to the target detection frame, identify the face of the operator in the equipment image to obtain a face recognition result;

[0021] Perform an operation specification detection on the special equipment of the power grid according to the face recognition result, the equipment type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid to obtain an operation specification detection result.

[0022] Fourthly, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0023] Obtain the device image of the special equipment of the power grid, and perform object detection on the device image to obtain a plurality of locally framed images;

[0024] Determine the object detection frame from each of the locally framed images according to the image classification results of each of the locally framed images;

[0025] Locate the actual spatial environment position of the special equipment of the power grid according to the object detection frame, and identify the device type of the special equipment of the power grid;

[0026] Identify the equipment identification in the object detection frame to obtain the special equipment identification, and, according to the object detection frame, identify the face of the operator in the device image to obtain the face recognition result;

[0027] Perform operation specification detection on the special equipment of the power grid according to the face recognition result, the device type of the special equipment of the power grid, the special equipment identification of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid to obtain the operation specification detection result.

[0028] Fifthly, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0029] Obtain the device image of the special equipment of the power grid, and perform object detection on the device image to obtain a plurality of locally framed images;

[0030] Determine the object detection frame from each of the locally framed images according to the image classification results of each of the locally framed images;

[0031] Locate the actual spatial environment position of the special equipment of the power grid according to the object detection frame, and identify the device type of the special equipment of the power grid;

[0032] Identify the equipment identification in the object detection frame to obtain the special equipment identification, and, according to the object detection frame, identify the face of the operator in the device image to obtain the face recognition result;

[0033] Perform operation specification detection on the special equipment of the power grid according to the face recognition result, the device type of the special equipment of the power grid, the special equipment identification of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid to obtain the operation specification detection result.

[0034] The above-mentioned power grid special equipment operation detection method, device, computer equipment, computer-readable storage medium and computer program product obtain the device image of the power grid special equipment, perform target detection on the device image to obtain a plurality of locally framed images; determine the target detection frame from each of the locally framed images according to the image classification result of each locally framed image; locate the actual spatial environment position of the power grid special equipment and identify the device type of the power grid special equipment according to the target detection frame; identify the equipment identifier in the target detection frame to obtain the special equipment identifier, and, according to the target detection frame, identify the face of the operator in the device image to obtain the face recognition result; perform operation specification detection on the power grid special equipment according to the face recognition result, the device type of the power grid special equipment, the special equipment identifier of the power grid special equipment and the actual spatial environment position of the power grid special equipment to obtain the operation specification detection result.

[0035] In this way, target detection is first performed to achieve the preliminary detection of the power grid special equipment. Determining the target detection frame from multiple locally framed images makes the target detection frame closest to the power grid characteristic equipment, so as to locate the actual spatial environment position of the power grid special equipment, identify the device type, identify the special equipment identifier and perform face recognition on the operator. Furthermore, according to the face recognition result, device type, special equipment identifier and actual spatial environment position, operation specification detection can be performed on the power grid special equipment to obtain the operation specification detection result. Using multi-dimensional information as the basis for operation specification detection can timely detect the abnormal operation conditions of the power grid special equipment. Therefore, the detection efficiency of the power grid special equipment operation is improved. Description of the Drawings

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0037] Figure 1 It is an application environment diagram of the power grid special equipment operation detection method in an embodiment;

[0038] Figure 2 It is a flow diagram of the power grid special equipment operation detection method in an embodiment;

[0039] Figure 3 It is a flow diagram of obtaining the device fusion feature by performing feature extraction and feature fusion on the device image in an embodiment;

[0040] Figure 4 A flowchart showing the steps of performing an operation specification detection on a special equipment of the power grid according to the face recognition result, the equipment type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment location of the special equipment of the power grid in an embodiment to obtain an operation specification detection result;

[0041] Figure 5 A flowchart showing the model training process of a face recognition model in an embodiment;

[0042] Figure 6 A structural block diagram of an operation detection device for special equipment of the power grid in an embodiment;

[0043] Figure 7 An internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0044] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] It should be noted that the information (such as device images, training samples, preset operators, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by users or fully authorized by all parties, and the acquisition, transmission, storage, use and processing of relevant data all comply with the relevant provisions of national laws and regulations. For the content pushed to users (such as operation specification detection results, partial boxed images, target detection frames, actual spatial environment locations, device types, face recognition results, etc.), users can reject or can conveniently reject content push, etc. In the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned, and they should be considered exemplary. The purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0046] The operation detection method for special equipment of the power grid provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the image acquisition component 102 communicates with the server 104 through the network. The image acquisition component 102 is used to acquire device images of special equipment in the power grid. The terminal 106 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers. Obtain the device images of special equipment in the power grid through the server 104, and perform object detection on the device images to obtain multiple locally framed images; determine the object detection frames from the locally framed images according to the image classification results of the locally framed images; according to the object detection frames, locate the actual spatial environment position of the special equipment in the power grid, and identify the device type of the special equipment in the power grid; identify the device identifier in the object detection frame to obtain the special equipment identifier, and, according to the object detection frame, identify the face of the operator in the device image to obtain the face recognition result; perform operation specification detection on the special equipment in the power grid according to the face recognition result, the device type of the special equipment in the power grid, the special equipment identifier of the special equipment in the power grid, and the actual spatial environment position of the special equipment in the power grid to obtain the operation specification detection result. The server 104 can push at least one of the operation specification detection result, the locally framed image, the object detection frame, the actual spatial environment position, the device type, and the face recognition result to the terminal 106. Among them, the terminal 106 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, a smart glasses, etc. The server 104 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0047] In an exemplary embodiment, as Figure 2 shown, a method for detecting the operation of special equipment in the power grid is provided. Taking the server 104 in Figure 1 as an example and described in an abbreviated form of the main body, it includes the following steps 202 to step 210. Among them:

[0048] Step 202, obtain the device images of special equipment in the power grid, and perform object detection on the device images to obtain multiple locally framed images.

[0049] Among them, the number of device images in step 202 can be single or multiple, and can be images collected from different angles of special power grid equipment. The device image is preferably an image representing the front view angle characteristics of the special power grid equipment.

[0050] As an embodiment, obtaining the device image of the special power grid equipment includes: obtaining the device image of the special power grid equipment collected by the image acquisition component.

[0051] As an embodiment, performing object detection on the device image to obtain a plurality of locally boxed images includes: boxing the regions in the device image where there is a possibility of being recognized as special power grid equipment to obtain a plurality of locally boxed images.

[0052] As an embodiment, boxing the regions in the device image where there is a possibility of being recognized as special power grid equipment to obtain a plurality of locally boxed images includes: extracting features of the device image at multiple different scales to obtain device image features; performing multi-scale feature enhancement on the device image features to obtain multi-scale device enhanced features; performing semantic feature transfer and fusion on the multi-scale device enhanced features to obtain device fusion features; and boxing the regions in the device image where there is a possibility of being recognized as special power grid equipment according to the device fusion features to obtain locally boxed images.

[0053] Among them, the process of extracting features of the device image at multiple different scales to obtain device image features can be executed by a deep convolutional layer, and the deep convolutional layer can be CSPDarknet (formed by introducing the CSPNet (Cross Stage Partial Networks) concept on the basis of the traditional Darknet architecture).

[0054] Among them, the process of performing multi-scale feature enhancement on the device image features to obtain multi-scale device enhanced features; and performing semantic feature transfer and fusion on the multi-scale device enhanced features to obtain device fusion features can be executed by a fusion network combining FPN (Feature Pyramid Network) and PANet (Path Aggregation Network).

[0055] Optionally, referring to Figure 3, the device image (Image shown in the figure) is input into PANet, and feature extraction of the device image is performed through step-by-step path enhancement from bottom to top by multiple feature layers of PANet. The features output by multiple feature layers of PANet are transmitted to FPN, so that multiple feature layers of FPN can perform step-by-step feature expansion from top to bottom to perform semantic feature fusion and obtain device fusion features (FPN Feature shown in the figure).

[0056] Step 204: Determine the target detection box from each locally boxed image according to the image classification result of each locally boxed image.

[0057] Among them, the image classification result in step 204 can represent the category of the target corresponding to the locally boxed image, and the image classification result can also represent the predicted probability that the target corresponding to the locally boxed image belongs to special equipment of the power grid, which is not limited here.

[0058] Exemplarily, step 204 includes: selecting each power grid special equipment boxed image from each locally boxed image according to the image classification result of each locally boxed image; determining the target detection box from each power grid special equipment boxed image.

[0059] In this way, before the process of determining the target detection box, the boxed images of special equipment of the power grid are screened first, which improves the accuracy of determining the target detection box.

[0060] As an embodiment, selecting each power grid special equipment boxed image from each locally boxed image according to the image classification result of each locally boxed image includes: selecting each power grid special equipment boxed image whose corresponding boxed target category belongs to special equipment of the power grid from each locally boxed image according to the image classification result of each locally boxed image.

[0061] As another embodiment, selecting each power grid special equipment boxed image from each locally boxed image according to the image classification result of each locally boxed image includes: selecting each power grid special equipment boxed image whose predicted probability that the corresponding boxed target belongs to special equipment of the power grid is greater than the first preset probability threshold from each locally boxed image.

[0062] As an embodiment, determining the target detection box from each power grid special equipment boxed image includes: selecting the target detection box from each power grid special equipment box, specifically: selecting the target detection box with the highest predicted probability that the corresponding boxed target belongs to special equipment of the power grid from each locally boxed image.

[0063] As another embodiment, determining a target detection box from the selected images of each special equipment of the power grid includes: selecting, from each partial selected image, each screened selected image for which the predicted probability that the corresponding selected target belongs to the special equipment of the power grid is greater than a second preset probability threshold, where the second preset probability threshold is greater than the first preset probability threshold; determining the target detection box according to the selected ranges corresponding to the screened images. Specifically, the selected range of the target detection box is the average of the selected ranges corresponding to the screened images, and the selected range corresponding to the target detection box includes the selected range corresponding to one of the screened images.

[0064] In this way, the selected ranges of multiple screened images can be fused, and finally a relatively average target detection box can be obtained, so that the target detection box is obtained through the decision-making of multiple screened images.

[0065] As an embodiment, determining a target detection box from each partial selected image according to the image classification result of each partial selected image includes: determining the predicted probability that each partial selected image respectively belongs to the special equipment of the power grid according to the image classification result of each partial selected image; for each partial selected image, evaluating the classification reliability of the partial selected image according to the confidence level and predicted probability of the partial selected image to obtain the reliability evaluation result of the partial selected image; and selecting the target detection box from each partial selected image according to the reliability evaluation results respectively corresponding to the partial selected images.

[0066] As an embodiment, evaluating the classification reliability of a partial selected image according to the confidence level and predicted probability of the partial selected image to obtain the reliability evaluation result of the partial selected image includes: performing weighted fusion on the confidence level and predicted probability of the partial selected image to obtain the reliability evaluation result of the partial selected image, where the higher the confidence level, the higher the reliability represented by the reliability evaluation result of the partial selected image, and the higher the predicted probability, the higher the reliability represented by the reliability evaluation result of the partial selected image.

[0067] As an embodiment, selecting the target detection box from each partial selected image according to the reliability evaluation results respectively corresponding to the partial selected images includes: selecting the target detection box with the highest reliability represented by the corresponding reliability evaluation result from each partial selected image.

[0068] Optionally, the method further includes: for each partial selected image, identifying the actual size of the target corresponding to the partial selected image in the actual space environment according to the partial selected image to obtain the actual size information; and performing image classification on the partial selected image according to the actual size information and the partial selected image to obtain the image classification result.

[0069] In this way, the actual size information and the locally cropped image are jointly used as the basis for image classification of the locally cropped image, thereby improving the accuracy of image classification of the locally cropped image.

[0070] Further, based on the locally cropped image, the size of the target corresponding to the locally cropped image in the actual space environment is recognized to obtain the actual size information, including: obtaining the acquisition parameters corresponding to the image acquisition component, where the acquisition parameters include at least one of focal length, sensor size, and image acquisition distance; according to the acquisition parameters corresponding to the image acquisition component, converting the relative size of the target corresponding to the locally cropped image in the locally cropped image into the size of the target corresponding to the locally cropped image in the actual space environment to obtain the actual size information.

[0071] Step 206, according to the target detection frame, locate the actual space environment position of the special equipment of the power grid and identify the equipment type of the special equipment of the power grid.

[0072] Among them, the actual space environment position in step 206 can represent the relative position between the special equipment of the power grid and the fixed components (such as walls, fixing racks, etc.) in the actual space environment, or can represent the coordinate positions of each point on the special equipment of the power grid in a preset space coordinate system, and the preset space coordinate system is a coordinate system constructed with any point in the actual space environment where the special equipment of the power grid is located as the origin.

[0073] Exemplarily, according to the target detection frame, locating the actual space environment position of the special equipment of the power grid includes: respectively converting the image coordinates of each point on the special equipment of the power grid in the target detection frame into the actual coordinates of each point on the special equipment of the power grid in the actual space environment; according to the actual coordinates of each point on the special equipment of the power grid in the actual space environment, locate the actual space environment position of the special equipment of the power grid.

[0074] As an embodiment, respectively converting the image coordinates of each point on the special equipment of the power grid in the target detection frame into the actual coordinates of each point on the special equipment of the power grid in the actual space environment includes: determining the coordinate conversion coefficient corresponding to the target detection frame according to the acquisition parameters corresponding to the image acquisition component; according to the coordinate conversion coefficient, respectively converting the image coordinates of each point on the special equipment of the power grid in the target detection frame into the actual coordinates of each point on the special equipment of the power grid in the actual space environment.

[0075] Exemplarily, according to the target detection frame, identifying the equipment type of the special equipment of the power grid includes: obtaining the actual size information of the special equipment of the power grid, and extracting the equipment features of the special equipment of the power grid from the target detection frame, where the equipment features are used to represent at least one of the equipment shape and equipment structure of the special equipment of the power grid; according to the actual size information and the equipment features of the special equipment of the power grid, identify the equipment type of the special equipment of the power grid.

[0076] Optionally, for the specific implementation steps of obtaining the actual size information of the special equipment of the power grid, reference may be made to the above-mentioned specific implementation content of identifying the size of the target corresponding to the locally framed image in the actual space environment based on the locally framed image to obtain the actual size information, which will not be elaborated here.

[0077] Step 208: Identify the equipment identifier in the target detection box to obtain the special equipment identifier, and, based on the target detection box, identify the face of the operator in the equipment image to obtain the face recognition result.

[0078] Among them, the special equipment identifier in step 208 may include at least one of the license plate and the nameplate identifier. The operator may be a person driving the special equipment of the power grid, or a person contacting the special equipment of the power grid, or a person controlling the special equipment of the power grid, which is not limited here.

[0079] Exemplarily, identifying the equipment identifier in the target detection box to obtain the special equipment identifier includes: performing image processing on the target detection box to obtain a processed detection box, where the image processing methods include at least one of grayscale conversion, noise reduction, binarization, contrast adjustment, and skew correction; locating the text area in the processed detection box and extracting the text features of the equipment identifier in the text area to obtain the identifier features; identifying the equipment identifier based on the identifier features to obtain the special equipment identifier.

[0080] Optionally, after identifying the equipment identifier based on the identifier features to obtain the special equipment identifier, the method further includes: performing a readable processing on the special equipment identifier, where the readable processing methods include at least one of correction, patching, and filtering, to improve the readability and accuracy of the special equipment identifier.

[0081] Among them, the process of identifying the equipment identifier in the target detection box to obtain the special equipment identifier can be executed by a text recognition model, and the text recognition model can be Paddle OCR (a high-performance optical character recognition tool based on deep learning).

[0082] As an embodiment, based on the target detection box, identifying the face of the operator in the equipment image to obtain the face recognition result includes: obtaining a preset feature vector set, where the preset feature vector set includes the correspondence between the personnel identity information and the preset feature vectors, and the personnel identity information includes at least one of the personnel name and the personnel work number; extracting the face feature vector of the operator from the equipment image based on the target detection box; identifying the face of the operator in the equipment image based on the distances between the face feature vector and each preset feature vector in the preset feature vector set to obtain the face recognition result.

[0083] Further, based on the distances between the face feature vector and each preset feature vector in the preset feature vector set, the face of the operator in the device image is recognized to obtain a face recognition result, including: if there is a target feature vector in the preset feature vector set whose corresponding distance is greater than the preset distance threshold, the personnel identity information corresponding to the target feature vector in the preset feature vector set is determined as the face recognition result; if there is no target feature vector in the preset feature vector set whose corresponding distance is greater than the preset distance threshold, the face recognition abnormal result is determined as the face recognition result.

[0084] Specifically, the preset feature vector set includes the correspondence between the personnel identity information of the personnel permitted to operate the special equipment of the power grid belonging to each equipment type and the preset feature vector.

[0085] In this way, the operator in the target detection frame is specifically compared with the personnel permitted to operate each special equipment of the power grid, rather than performing face recognition on the operator in the target detection frame. Therefore, the workload of face recognition is reduced and the face recognition efficiency is improved.

[0086] Among them, the process of recognizing the face of the operator in the device image based on the target detection frame to obtain a face recognition result can be executed by a face recognition algorithm, and the face recognition algorithm can be FaceNet (face recognition network).

[0087] Step 210, perform an operation specification detection on the special equipment of the power grid according to the face recognition result, the equipment type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid, to obtain an operation specification detection result.

[0088] Exemplarily, step 210 includes: if the face recognition result includes a face recognition abnormal result, the face recognition abnormal result is determined as the operation specification detection result; if the face recognition result does not include a face recognition abnormal result, perform an operation specification detection on the special equipment of the power grid according to the equipment type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid, to obtain an operation specification detection result.

[0089] In this way, considering that the face recognition result is obtained by comparing with the preset feature vector set, and the face recognition abnormal result indicates that the recognized operator does not belong to the personnel in the preset feature vector set. Therefore, it means that the operator may be a person who does not have permission to operate the special equipment of the power grid. Then, directly determining the face recognition abnormal result as the operation specification detection result can provide timely feedback to the manual end and improve the detection efficiency of the operation of the special equipment of the power grid.

[0090] In the above method for detecting the operation of special equipment in the power grid, target detection is first performed to achieve the preliminary detection of the special equipment in the power grid. The target detection frame is determined from multiple locally selected images, so that the target detection frame is closest to the characteristic equipment of the power grid, thereby positioning the actual spatial environment location of the special equipment in the power grid, identifying the equipment type, identifying the special equipment identification, and performing face recognition on the operator. Furthermore, according to the face recognition result, equipment type, special equipment identification, and actual spatial environment location, the operation specification detection of the special equipment in the power grid is carried out to obtain the operation specification detection result. Using multi-dimensional information as the basis for the operation specification detection can timely detect the abnormal operation conditions of the special equipment in the power grid. Therefore, the detection efficiency of the operation of the special equipment in the power grid is improved.

[0091] In an exemplary embodiment, as Figure 4 shown, a method for accurately detecting the operation specifications of special equipment in the power grid is provided. According to the face recognition result, the equipment type of the special equipment in the power grid, the special equipment identification of the special equipment in the power grid, and the actual spatial environment location of the special equipment in the power grid, the operation specification detection of the special equipment in the power grid is carried out to obtain the operation specification detection result, which includes steps 302 to 304. Among them:

[0092] Step 302, if the face recognition result matches the special equipment identification, then when the equipment type matches the actual spatial environment location, the operation specification detection of the special equipment in the power grid is carried out according to the special equipment identification and the actual spatial environment location to obtain the operation specification detection result. When the equipment type does not match the actual spatial environment location, the equipment position abnormal result is determined as the operation specification detection result.

[0093] As an embodiment, the method further includes: obtaining the identity information of the permitted personnel corresponding to the special equipment identification, where the identity information of the permitted personnel includes the identity information of the personnel permitted to operate the special equipment in the power grid corresponding to the special equipment identification; when the identity information of the permitted personnel includes the same identity information as the face recognition result, it is determined that the face recognition result matches the special equipment identification. When the identity information of the permitted personnel does not include the same identity information as the face recognition result, it is determined that the face recognition result does not match the special equipment identification.

[0094] It is understandable that there may be a situation where grid special equipment belonging to the same equipment type but containing different special equipment identifiers have different equipment functions. For example, forklift A is responsible for cargo handling in factory area a, and forklift B is responsible for cargo handling in factory area b. At this time, forklift A and forklift B belong to the same equipment type but contain different special equipment identifiers and are responsible for different equipment functions. To ensure that grid special equipment in the grid area can be accurately, quickly, and safely controlled, generally, grid special equipment with different equipment functions is not controlled by mixing equipment. For example, controlling forklift B to handle the cargo in factory area a and controlling forklift A to handle the cargo in factory area b. Since the control data of different forklifts may not be exchanged and interconnected, there may be a situation of performing redundant control operations. For example, not knowing that forklift A has completed the cargo handling task in factory area b and still controlling forklift B to go to factory area b for cargo handling, etc. Therefore, it is necessary to add whether the special equipment identifier matches the actual spatial environment position to the detection process of grid special equipment operations to ensure accurate, quick, and safe control of grid special equipment.

[0095] As an embodiment, according to the special equipment identifier and the actual spatial environment position, the operation specification of the grid special equipment is detected to obtain the operation specification detection result, including: if the special equipment identifier matches the actual spatial environment position, according to the actual spatial environment position, the operation video stream data during the operator's operation of the grid special equipment is collected, and according to the operation video stream data, the operation specification of the grid special equipment is detected to obtain the operation specification detection result; if the special equipment identifier does not match the actual spatial environment position, the equipment operation position abnormal result is determined as the operation specification detection result.

[0096] Optionally, the method further includes: obtaining the equipment operation position corresponding to the special equipment identifier, and determining that the special equipment identifier matches the actual spatial environment position when the actual spatial environment position belongs to the equipment operation position; determining that the special equipment identifier does not match the actual spatial environment position when the actual spatial environment position does not belong to the equipment operation position.

[0097] Through the above process, it can only be ensured that the grid special equipment is operated by the personnel permitted to operate and is operated at the correct position. It is understandable that since most grid special equipment involves life safety and is of great danger, then, when operating the grid special equipment, the correctness and safety of the operation need to be considered. Therefore, it is necessary to add the corresponding operation process of the grid special equipment to the detection process of grid special equipment operations to ensure accurate and safe control of the grid special equipment.

[0098] Further, based on the operation video stream data, the operation specification of the special equipment of the power grid is detected to obtain the operation specification detection result, including: identifying the relative position between the person and the special equipment of the power grid according to the operation video stream data to obtain the relative position change information; if the relative position change information matches the special equipment identifier, then according to the operation video stream data, identifying the process of interaction between the operator and the special equipment of the power grid to obtain the human-machine interaction recognition result, and according to the human-machine interaction recognition result, detecting the operation specification of the special equipment of the power grid to obtain the operation specification detection result; if the relative position change information does not match the special equipment identifier, then determining the result of potential operation safety hazard as the operation specification detection result.

[0099] As an embodiment, the method further includes: obtaining the relative position information of potential hazard corresponding to the special equipment identifier, where the relative position information of potential hazard is used to represent the relative position information between the person with potential safety hazard and the special equipment of the power grid; if at least one relative position information in the relative position change information is the same as the relative position information of potential hazard, then determining that the relative position change information does not match the special equipment identifier, and if there is no at least one relative position information in the relative position change information that is the same as the relative position information of potential hazard, then determining that the relative position change information does not match the special equipment identifier.

[0100] As an embodiment, according to the operation video stream data, identifying the process of interaction between the operator and the special equipment of the power grid to obtain the human-machine interaction recognition result, including: identifying the operation process of the operator according to the operation video stream data to obtain the operator recognition result; and identifying the equipment control process of the special equipment of the power grid according to the operation video stream data to obtain the equipment recognition result; identifying the process of interaction between the operator and the special equipment of the power grid according to the operator recognition result and the equipment recognition result to obtain the human-machine interaction recognition result.

[0101] As an embodiment, according to the human-machine interaction recognition result, detecting the operation specification of the special equipment of the power grid to obtain the operation specification detection result, including: obtaining the equipment function information corresponding to the special equipment identifier, and according to the equipment function information, determining the standard interaction process information of the special equipment of the power grid; if the human-machine interaction recognition result is consistent with the standard interaction process information, then determining the result of correct operation as the operation specification detection result; if the human-machine interaction recognition result is inconsistent with the standard interaction process information, then determining the result of incorrect operation as the operation specification detection result.

[0102] Step 304, if the face recognition result does not match the special equipment identifier, then determining the result of operator anomaly as the operation specification detection result.

[0103] As an embodiment, the method further includes: generating an operator work order corresponding to the special equipment of the power grid according to the face recognition result and the operation specification detection result, and pushing and visually displaying the operator work order.

[0104] In this way, in order to monitor the operation of the special equipment of the power grid in real time, when the operation specification of the special equipment of the power grid is abnormal, manual intervention can be carried out in time to avoid serious consequences (for example, safety problems caused by non-standard operation, low task completion efficiency, and tasks not completed for a long time, etc.).

[0105] In this embodiment, by matching the face recognition result with the special equipment identifier, when the face recognition result does not match the special equipment identifier, the operator abnormality result is directly determined as the operation specification detection result. When the face recognition result matches the special equipment identifier, the operation specification of the special equipment of the power grid is detected according to the special equipment identifier and the actual spatial environment position only when the equipment type matches the actual spatial environment position, and the operation specification detection result is obtained. When the equipment type does not match the actual spatial environment position, the equipment position abnormality result is determined as the operation specification detection result. That is to say, it belongs to a progressive recognition process. In the case of non-conformity in each process, the operation specification detection result is directly determined. Therefore, the detection efficiency of the operation of the special equipment of the power grid is improved.

[0106] In an exemplary embodiment, as Figure 5 shown, a method for accurately training a face recognition model is provided. According to the target detection frame, the face of the operator in the equipment image is recognized, and the process of obtaining the face recognition result is executed by the face recognition model. The model training process of the face recognition model includes steps 402 to 406. Among them:

[0107] Step 402, obtaining multiple training samples, where each training sample includes a training face image and a face recognition label corresponding to the training face image.

[0108] Among them, the training face images in step 402 include face images of multiple personnel who are allowed to operate special equipment of the power grid belonging to multiple equipment types, collected in multiple environments. The training personnel images can be images when the personnel are operating the special equipment of the power grid, or images when the personnel are not operating the special equipment of the power grid (for example, centralized portrait collection, ID photos, etc.). The training personnel images can be images of the personnel collected from various acquisition angles, and the training personnel images can be images of the personnel with various facial expressions. The face recognition label includes at least one of personnel identity information and personnel operation. The personnel operation includes the personnel operating the special equipment of the power grid or the personnel not operating the special equipment of the power grid.

[0109] Step 404: For each training sample, when the face recognition label in the training sample represents both the preset operator and the special equipment of the power grid at the same time, determine the sample type of the training sample as the anchor sample type; when the face recognition label in the training sample does not represent the preset operator, determine the sample type of the training sample as the negative sample type; when the face recognition label in the training sample only represents the preset personnel, determine the sample type of the training sample as the positive sample type, where the preset operator is the person permitted to operate the special equipment of the power grid.

[0110] As an embodiment, the method further includes: if the personnel identity information included in the face recognition label of the training sample belongs to the preset operator, determine that the face recognition label of the training sample represents the preset operator; if the personnel operation included in the face recognition label of the training sample includes the operation of the special equipment of the power grid by the personnel, determine that the face recognition label of the training sample represents the special equipment of the power grid.

[0111] Step 406: According to the sample types corresponding to multiple training samples, construct the loss functions corresponding to the multiple training samples, and train a face recognition model based on the loss functions and the multiple training samples.

[0112] Exemplarily, constructing the loss functions corresponding to multiple training samples according to the sample types corresponding to the multiple training samples includes: constructing the loss functions corresponding to the multiple training samples according to the first sample distance between the training samples belonging to the anchor sample type and the training samples belonging to the positive sample type, and the second sample distance between the training samples belonging to the anchor sample type and the training samples belonging to the negative sample type.

[0113] As an embodiment, constructing the loss functions corresponding to multiple training samples according to the first sample distance between the training samples belonging to the anchor sample type and the training samples belonging to the positive sample type, and the second sample distance between the training samples belonging to the anchor sample type and the training samples belonging to the negative sample type includes: correcting the first sample distance through a hyperparameter to obtain a corrected sample distance, where the correction method includes but is not limited to aggregation correction (addition and subtraction correction) and fusion correction (multiplication and division correction); constructing the loss functions corresponding to the multiple training samples according to the magnitude relationship between the corrected sample distance and the second sample distance.

[0114] Further, correcting the first sample distance through a hyperparameter to obtain a corrected sample distance includes: determining the sum of the hyperparameter and the first sample distance as the corrected sample distance.

[0115] As an embodiment, according to the magnitude relationship between the corrected sample distance and the second sample distance, loss functions corresponding to multiple training samples are constructed, including: constructing loss functions corresponding to multiple training samples when the corrected sample distance is greater than the second sample distance.

[0116] Optionally, when the corrected sample distance is greater than the second sample distance, the loss function corresponding to multiple training samples can be expressed by the formula:

[0117]

[0118] where is the first sample distance, is a hyperparameter, is the second sample distance.

[0119] In this way, by setting the first sample distance to be less than the second sample distance, it can be ensured that the face recognition model trained based on the loss function can better recognize the faces of operators who operate on special equipment of the power grid, improving the accuracy of face recognition.

[0120] Furthermore, based on the loss function and multiple training samples, a face recognition model is trained, including: performing image unification processing on the training face images in multiple training samples, where the image unification processing methods include normalization, cropping, and scaling, so that the training face images in multiple training samples have the same format and size; for each training sample, detecting a face bounding box from the training face image in the training sample through the trained face recognition model; extracting features from the face bounding box through the trained face recognition model to obtain a training face feature vector; determining the training sample distance between the training face feature vectors of each training sample through the trained face recognition model; determining the loss function value corresponding to each pair of training samples through the trained face recognition model according to the sample type, training sample distance, and loss function corresponding to each training sample. If the loss function value corresponding to the training sample indicates loss convergence, the current trained face recognition model is determined as the face recognition model. If the loss function value corresponding to the training sample indicates loss non-convergence, the model parameters of the trained face recognition model are adjusted, and the process returns to the step of detecting the face bounding box from the training face image in the training sample through the trained face recognition model and subsequent steps until the loss function value corresponding to the training sample indicates loss convergence.

[0121] Optionally, the method further includes: installing the PyTorch system using the pip command and building the foundation of the algorithm framework based on the PyTorch system; importing basic libraries such as torch, torch.nn as nn, torch.optim as optim, and torchvision to provide data support for system development; loading the MNIST dataset using torchvision.datasets, performing batch processing and distribution of the data through DataLoader, and performing data preprocessing such as normalization, cropping, and data type conversion using transforms; creating a class that inherits from nn.Module to define the neural network structure, overriding the init() initialization function and the forward() forward propagation function. Define the convolutional layer, pooling layer, and fully connected layer of the network in init(); describe the flow order of the data in the network in forward(), that is, the connection method of each layer and the application of the activation function; define the loss function based on different task types, where the CrossEntropyLoss function is selected for classification tasks and the MSELoss function is selected for regression tasks. Initialize the SGD optimizer using network parameters and hyperparameters such as learning rate; train the model by iterating over the dataset multiple times, clearing the gradients in each iteration; perform forward propagation to obtain the output and calculate the loss; perform backward propagation to calculate the gradients; update the network parameters using the optimizer; save the parameters of the trained model or the entire model object using the torch.save() function.

[0122] In this way, the system framework algorithm of the model can be built and applied to the above-mentioned model.

[0123] In this embodiment, by using whether the person is permitted to operate the special equipment of the power grid as the basis, the training samples are classified, and the loss function is constructed based on the classified sample types, so that the face recognition model trained based on the loss function can more accurately identify the face of the operator who operates the special equipment of the power grid, improving the recognition accuracy of the face recognition model.

[0124] As a detailed embodiment, obtain the device image of the special equipment of the power grid, perform feature extraction on the device image at multiple different scales to obtain the device image features; perform multi-scale feature enhancement on the device image features to obtain multi-scale device enhancement features; perform semantic feature transfer and fusion on the multi-scale device enhancement features to obtain device fusion features; perform object detection on the device image according to the device fusion features to obtain a locally framed image; determine the object detection frame from each locally framed image according to the image classification result of each locally framed image; locate the actual spatial environment position of the special equipment of the power grid and identify the device type of the special equipment of the power grid according to the object detection frame; identify the device identifier in the object detection frame to obtain the special equipment identifier, and, according to the object detection frame, identify the face of the operator in the device image to obtain the face recognition result; if the face recognition result matches the special equipment identifier, then when the device type matches the actual spatial environment position, if the special equipment identifier matches the actual spatial environment position, collect the operation video stream data during the operator's operation of the special equipment of the power grid according to the actual spatial environment position, and identify the relative position between the person and the special equipment of the power grid according to the operation video stream data to obtain the relative position change information; if the relative position change information matches the special equipment identifier, then identify the process of interaction between the operator and the special equipment of the power grid according to the operation video stream data to obtain the human-computer interaction recognition result, and perform operation specification detection on the special equipment of the power grid according to the human-computer interaction recognition result to obtain the operation specification detection result; if the relative position change information does not match the special equipment identifier, then determine the result of potential operation safety hazards as the operation specification detection result; if the special equipment identifier does not match the actual spatial environment position, then determine the result of abnormal device operation position as the operation specification detection result, and when the device type does not match the actual spatial environment position, determine the result of abnormal device position as the operation specification detection result; if the face recognition result does not match the special equipment identifier, then determine the result of operator abnormality as the operation specification detection result.

[0125] Further, according to the image classification results of each locally cropped image, determine the predicted probability that each locally cropped image respectively belongs to a special equipment of the power grid; for each locally cropped image, evaluate the classification reliability of the locally cropped image according to the confidence level and the predicted probability of the locally cropped image, and obtain the reliability evaluation result of the locally cropped image; according to the reliability evaluation results respectively corresponding to each locally cropped image, select the target detection box from each locally cropped image; according to the target detection box, identify the face of the operator in the equipment image to obtain the face recognition result, and the process is executed by the face recognition model. The model training process of the face recognition model includes: obtaining multiple training samples, where each training sample includes a training face image and a face recognition label corresponding to the training face image; for each training sample, when the face recognition label in the training sample represents both a preset operator and a special equipment of the power grid, determine the sample type of the training sample as the anchor sample type, when the face recognition label in the training sample does not represent the preset operator, determine the sample type of the training sample as the negative sample type, and when the face recognition label in the training sample only represents the preset personnel, determine the sample type of the training sample as the positive sample type, where the preset operator is the person permitted to operate the special equipment of the power grid; according to the sample types corresponding to the multiple training samples, construct the loss functions corresponding to the multiple training samples, and train the face recognition model based on the loss functions and the multiple training samples.

[0126] In this way, first perform target detection to achieve the preliminary detection of the special equipment of the power grid, and determine the target detection box from multiple locally cropped images, so that the target detection box is closest to the power grid feature equipment, thereby positioning the actual spatial environment position of the special equipment of the power grid, performing equipment type recognition, performing special equipment identification recognition, and operator face recognition. Furthermore, according to the face recognition result, equipment type, special equipment identification, and actual spatial environment position, perform operation specification detection on the special equipment of the power grid to obtain the operation specification detection result. Using multi-dimensional information as the basis for operation specification detection can timely detect the abnormal operation conditions of the special equipment of the power grid. Therefore, the detection efficiency of the operation of the special equipment of the power grid is improved.

[0127] Further, by matching the face recognition result with the special equipment identification, when the face recognition result does not match the special equipment identification, the operator anomaly result is directly determined as the operation specification detection result. When the face recognition result matches the special equipment identification, only when the equipment type matches the actual spatial environment location, the operation specification of the power grid special equipment is detected according to the special equipment identification and the actual spatial environment location to obtain the operation specification detection result. When the equipment type does not match the actual spatial environment location, the equipment location anomaly result is determined as the operation specification detection result. That is, it belongs to a progressive recognition process. And when each process does not conform, the operation specification detection result is directly determined. Therefore, the detection efficiency of the operation of the power grid special equipment is improved; and, by using whether the person is permitted to operate the power grid special equipment as a basis, the training samples are classified, and based on the classified sample types, a loss function is constructed, so that the face recognition model trained based on the loss function can more accurately recognize the face of the operator who operates the power grid special equipment, improving the recognition accuracy of the face recognition model.

[0128] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0129] Based on the same inventive concept, an embodiment of the present application further provides a power grid special equipment operation detection device for implementing the power grid special equipment operation detection method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the power grid special equipment operation detection device provided below can refer to the limitations on the power grid special equipment operation detection method in the above text, and will not be repeated here.

[0130] In an exemplary embodiment, as Figure 6 shown, a power grid special equipment operation detection device 600 is provided, including: an acquisition module 602, a determination module 604, an identification module 606, and a detection module 608, where:

[0131] An acquisition module 602 acquires device images of special equipment of the power grid, and performs object detection on the device images to obtain a plurality of locally cropped images;

[0132] A determination module 604 is configured to determine an object detection frame from each of the locally cropped images according to the image classification results of the locally cropped images;

[0133] An identification module 606 is configured to locate the actual spatial environment position of the special equipment of the power grid and identify the device type of the special equipment of the power grid according to the object detection frame; identify the device identifier in the object detection frame to obtain a special equipment identifier, and, according to the object detection frame, identify the face of the operator in the device image to obtain a face recognition result;

[0134] A detection module 608 is configured to perform an operation specification detection on the special equipment of the power grid according to the face recognition result, the device type of the special equipment of the power grid, the special equipment identifier of the special equipment of the power grid, and the actual spatial environment position of the special equipment of the power grid, to obtain an operation specification detection result.

[0135] In one embodiment, the detection module 608 is further configured to, if the face recognition result matches the special equipment identifier, perform an operation specification detection on the special equipment of the power grid according to the special equipment identifier and the actual spatial environment position when the device type matches the actual spatial environment position, to obtain an operation specification detection result, and determine an abnormal device position result as the operation specification detection result when the device type does not match the actual spatial environment position; if the face recognition result does not match the special equipment identifier, determine an abnormal operator result as the operation specification detection result.

[0136] In one embodiment, the detection module 608 is further configured to, if the special equipment identifier matches the actual spatial environment position, collect operation video stream data during the operation of the operator on the special equipment of the power grid according to the actual spatial environment position, and perform an operation specification detection on the special equipment of the power grid according to the operation video stream data, to obtain an operation specification detection result; if the special equipment identifier does not match the actual spatial environment position, determine an abnormal device operation position result as the operation specification detection result.

[0137] In one embodiment, the detection module 608 is further configured to identify the relative position between the person and the special equipment of the power grid according to the operation video stream data, so as to obtain the relative position change information; if the relative position change information matches the special equipment identifier, then according to the operation video stream data, identify the process of interaction between the operator and the special equipment of the power grid, obtain the human-computer interaction recognition result, and according to the human-computer interaction recognition result, detect the operation specification of the special equipment of the power grid to obtain the operation specification detection result; if the relative position change information does not match the special equipment identifier, then determine the result of potential operation safety hazard as the operation specification detection result.

[0138] In one embodiment, the determination module 604 is further configured to determine the prediction probability that each locally boxed image respectively belongs to the special equipment of the power grid according to the image classification result of each locally boxed image; for each locally boxed image, evaluate the classification reliability of the locally boxed image according to the confidence level and prediction probability of the locally boxed image, so as to obtain the reliability evaluation result of the locally boxed image; according to the reliability evaluation results respectively corresponding to each locally boxed image, select the target detection box from each locally boxed image.

[0139] In one embodiment, the acquisition module 602 is further configured to extract features of the equipment image at multiple different scales to obtain equipment image features; perform multi-scale feature enhancement on the equipment image features to obtain multi-scale equipment enhanced features; perform semantic feature transfer and fusion on the multi-scale equipment enhanced features to obtain equipment fusion features; perform target detection on the equipment image according to the equipment fusion features to obtain locally boxed images.

[0140] In one embodiment, the process of identifying the face of the operator in the equipment image according to the target detection box to obtain the face recognition result is executed by a face recognition model, and the device further includes: a training module, configured to obtain multiple training samples, where each training sample includes a training face image and a face recognition label corresponding to the training face image; for each training sample, when the face recognition label in the training sample represents both the preset operator and the special equipment of the power grid, determine the sample type of the training sample as the anchor sample type, when the face recognition label in the training sample does not represent the preset operator, determine the sample type of the training sample as the negative sample type, and when the face recognition label in the training sample only represents the preset person, determine the sample type of the training sample as the positive sample type, where the preset operator is the person permitted to operate the special equipment of the power grid; construct loss functions corresponding to the multiple training samples according to the sample types corresponding to the multiple training samples, and train the face recognition model based on the loss functions and the multiple training samples.

[0141] Each module in the above-mentioned power grid special equipment operation detection device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor in the computer device in hardware form or be independent of the processor, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0142] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communicating with external terminals in a wired or wireless manner. The wireless manner can be achieved through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a power grid special equipment operation detection method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0143] Those skilled in the art can understand that Figure 7 the structure shown in

[0144] is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0145] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0146] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0147] Those of ordinary skill in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0148] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0149] The above embodiments only express several implementation manners of this application, and the description thereof is relatively specific and detailed. However, it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several deformations and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A method for detecting operation of special equipment in a power grid, characterized in that: The method comprises: Acquire an equipment image of a special equipment of a power grid, and perform target detection on the equipment image to obtain a plurality of local frame selection images; Determining a target detection frame from each of the local framed images according to an image classification result of each of the local framed images; According to the target detection frame, locate the actual spatial environment position of the power grid special equipment, and identify the equipment type of the power grid special equipment; Identify the equipment identification in the target detection frame to obtain the special equipment identification, and, based on the target detection frame, identify the face of the operator in the equipment image to obtain a face recognition result; Based on the face recognition result, the equipment type of the power grid special equipment, the special equipment identification of the power grid special equipment and the actual spatial environment position of the power grid special equipment, the operation specification detection is performed on the power grid special equipment to obtain the operation specification detection result.

2. The method according to claim 1, characterized in that According to the face recognition result, the equipment type of the power grid special equipment, the special equipment identification of the power grid special equipment and the actual spatial environment position of the power grid special equipment, the operation specification detection of the power grid special equipment includes: If the face recognition result matches the special equipment identification, then when the equipment type matches the actual space environment position, the power grid special equipment is subjected to an operation specification detection according to the special equipment identification and the actual space environment position to obtain an operation specification detection result; when the equipment type does not match the actual space environment position, the abnormal equipment position result is determined as the operation specification detection result; If the face recognition result does not match the special equipment identification, the operator abnormal result is determined as the operation specification detection result.

3. The method according to claim 2, characterized in that The step of performing an operation specification detection on the power grid special equipment according to the special equipment identifier and the actual space environment position to obtain an operation specification detection result includes: If the special equipment identifier matches the actual space environment position, then according to the actual space environment position, the operation video stream data of the operator operating the power grid special equipment is collected, and according to the operation video stream data, the operation specification detection is performed on the power grid special equipment to obtain the operation specification detection result; If the special equipment identification does not match the actual spatial environment position, the abnormal result of the equipment operation position is determined as the operation specification detection result.

4. The method according to claim 3, characterized in that The step of performing an operation specification detection on the power grid special equipment according to the operation video stream data to obtain an operation specification detection result includes: According to the operation video stream data, the relative position between the personnel and the special equipment of the power grid is identified to obtain relative position change information; If the relative position change information matches the special equipment identifier, the process of the operator interacting with the power grid special equipment is identified according to the operation video stream data to obtain a human-computer interaction identification result, and the power grid special equipment is subjected to an operation specification detection according to the human-computer interaction identification result to obtain an operation specification detection result; If the relative position change information does not match the special equipment identification, the result of the existence of an operational safety hazard is determined as an operational specification detection result.

5. The method according to claim 1, characterized in that The determining the target detection frame from each of the local frame selected images according to the image classification result of each of the local frame selected images includes: Determine, according to the image classification results of each of the local framed images, the predicted probability that each of the local framed images corresponds to the special equipment of the power grid; For each of the partially framed images, evaluating the classification reliability of the partially framed image according to the confidence and prediction probability of the partially framed image, and obtaining a reliability evaluation result of the partially framed image; According to the reliability evaluation results respectively corresponding to each of the local framed images, a target detection frame is selected from each of the local framed images.

6. The method according to claim 1, characterized in that The performing target detection on the device image to obtain a plurality of local frame-selected images includes: Extracting features of the device image at multiple different scales to obtain device image features; Performing multi-scale feature enhancement on the device image features to obtain multi-scale device enhancement features; Performing semantic feature transfer and fusion on the multi-scale device enhancement features to obtain device fusion features; According to the device fusion feature, target detection is performed on the device image to obtain a local frame selection image.

7. The method according to any one of claims 1 to 6, characterized in that The process of recognizing the face of the operator in the device image according to the target detection frame and obtaining the face recognition result is performed by a face recognition model, and the model training process of the face recognition model includes: Acquire multiple training samples, wherein each of the training samples includes a training face image and a face recognition label corresponding to the training face image; For each of the training samples, when the face recognition label in the training sample represents both the preset operator and the power grid special equipment, the sample type of the training sample is determined to be the anchor sample type; when the face recognition label in the training sample does not represent the preset operator, the sample type of the training sample is determined to be the negative sample type; when the face recognition label in the training sample only represents the preset person, the sample type of the training sample is determined to be the positive sample type, wherein the preset operator is a person who is allowed to operate the power grid special equipment; According to the sample types corresponding to the multiple training samples, loss functions corresponding to the multiple training samples are constructed, and based on the loss function and the multiple training samples, a face recognition model is trained to obtain the face recognition model.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images