Model training method of power face protection appliance based on deep learning

Abstract

The invention relates to a deep learning-based model training method and device for a power face protection appliance, computer equipment, a computer readable storage medium and a computer program product, and can be used in the technical field of artificial intelligence. The method comprises the following steps: acquiring sample image data of a power operation site through an image acquisition device; performing multi-dimensional classification processing on the sample image data to obtain a classification result of the sample image data; according to a classification result, labeling the sample image data to obtain labeled data of the sample image data; iteratively training a to-be-trained power face protection appliance recognition model by using the labeled data to obtain a target power face protection appliance recognition model; the target electric power face protection tool recognition model is used for recognizing whether an operator in an electric power operation site correctly wears an electric power face protection tool or not. By adopting the method, the detection efficiency of the power face protection tool worn by the operator can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model training method, apparatus, computer equipment, computer-readable storage medium, and computer program product for electric facial protection equipment based on deep learning. Background Art

[0002] With the development of power technology, power face protection equipment plays a vital role in the power industry. Properly wearing and using these protective gear is key to ensuring worker safety and improving work efficiency. Therefore, effectively testing workers wearing power face protection equipment has become an important research topic.

[0003] Traditional technology usually uses manual inspections to detect whether workers are wearing electric face protection equipment; however, this method requires a lot of manual processing time, resulting in low efficiency in detecting whether workers are wearing electric face protection equipment. Summary of the Invention

[0004] Based on this, it is necessary to provide a model training method, device, computer equipment, computer-readable storage medium and computer program product for electric facial protection based on deep learning, which can improve the efficiency of detection of workers wearing electric facial protection.

[0005] In a first aspect, the present application provides a model training method for electric face protection based on deep learning. The method comprises:

[0006] Acquire sample image data of the power operation site through an image acquisition device;

[0007] Performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0008] performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data;

[0009] The labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

[0010] In one embodiment, performing multi-dimensional classification processing on the sample image data to obtain the classification result of the sample image data includes:

[0011] Performing the job type classification process on the sample image data to obtain a job type classification result of the sample image data;

[0012] Performing the personnel type classification process on the sample image data to obtain a personnel type classification result of the sample image data;

[0013] Performing the protective gear wearing mode classification processing on the sample image data to obtain a protective gear wearing mode classification result of the sample image data;

[0014] The classification result of the sample image data is determined according to the classification result of the operation type, the classification result of the personnel type and the classification result of the protective equipment wearing method.

[0015] In one embodiment, performing the job type classification process on the sample image data to obtain the job type classification result of the sample image data includes:

[0016] Performing the operation type classification processing on the sample image data to obtain mask operation type image data, goggles operation type image data and irrelevant operation type image data;

[0017] The operation type classification result is determined based on the mask operation type image data, the goggles operation type image data and the irrelevant operation type image data.

[0018] In one embodiment, performing the personnel type classification process on the sample image data to obtain the personnel type classification result of the sample image data includes:

[0019] Performing personnel type classification processing on the sample image data to obtain mask operator type image data, goggles operator type image data, non-operator type image data, and invalid person type image data;

[0020] The personnel type classification result is determined based on the mask worker type image data, the goggles worker type image data, the non-worker type image data and the invalid worker type image data.

[0021] In one embodiment, after iteratively training the electric facial protection device recognition model to be trained using the labeled data to obtain the target electric facial protection device recognition model, the method further includes:

[0022] Acquiring current image data of the power operation site;

[0023] Inputting the current image data into the target power facial protection equipment recognition model to perform power facial protection equipment recognition processing to obtain the power facial protection equipment recognition result at the power operation site;

[0024] Determining whether the operator at the power operation site is wearing the power face protection device correctly based on the recognition result of the power face protection device;

[0025] When it is determined that the worker at the power operation site is not wearing the power face protector correctly, abnormal wearing warning information of the power face protector is generated.

[0026] In one embodiment, the method further comprises:

[0027] Obtaining historical power face protection equipment identification results at the power operation site;

[0028] Performing data analysis and processing on the historical power face protection equipment recognition results to obtain historical abnormal wearing type information, historical abnormal wearing time information, and historical abnormal wearing area information of the power face protection equipment at the power operation site;

[0029] Based on the historical wearing abnormality type information, the historical wearing abnormality time information and the historical wearing abnormality area information, future wearing abnormality information of the electric power face protection device at the electric power operation site is predicted.

[0030] In a second aspect, the present application also provides a model training device for electric facial protection based on deep learning. The device comprises:

[0031] A data acquisition module is used to acquire sample image data of the power operation site through an image acquisition device;

[0032] a data classification module, configured to perform multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of operation type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0033] a data annotation module, configured to perform annotation processing on the sample image data according to the classification result to obtain annotation data of the sample image data;

[0034] The model training module is used to use the labeled data to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are performed:

[0036] Acquire sample image data of the power operation site through an image acquisition device;

[0037] Performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0038] performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data;

[0039] The labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0041] Acquire sample image data of the power operation site through an image acquisition device;

[0042] Performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0043] performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data;

[0044] The labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

[0045] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0046] Acquire sample image data of the power operation site through an image acquisition device;

[0047] Performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0048] performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data;

[0049] The labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

[0050] The above-mentioned deep learning-based model training method, device, computer equipment, computer-readable storage medium and computer program product for electric facial protective equipment obtain sample image data of the electric power operation site through an image acquisition device; perform multi-dimensional classification processing on the sample image data to obtain the classification results of the sample image data; the multi-dimensional classification processing includes operation type classification processing, personnel type classification processing and protective equipment wearing method classification processing; based on the classification results, the sample image data is labeled to obtain labeled data of the sample image data; using the labeled data, the electric facial protective equipment recognition model to be trained is iteratively trained to obtain a target electric facial protective equipment recognition model; the target electric facial protective equipment recognition model is used to identify whether the workers in the electric power operation site are wearing electric facial protective equipment correctly. This solution obtains sample image data of the power operation site through an image acquisition device, and classifies the sample image data into three dimensions: operation type, personnel type, and protective equipment wearing method, which is conducive to comprehensive and accurate identification of the use of protective equipment; then, by labeling the classified sample image data and using the labeled data to iteratively train the power facial protective equipment recognition model, it is conducive to improving the efficiency and accuracy of the model in identifying the use of protective equipment in the operation scene, thereby improving the efficiency and accuracy of the detection of workers wearing power facial protective equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 1. A flowchart of a model training method for an electric face protection device based on deep learning in one embodiment;

[0053] Figure 2 A schematic flow chart of the steps of determining the classification result of sample image data in one embodiment;

[0054] Figure 3 A flowchart illustrating steps for determining a job type classification result in one embodiment;

[0055] Figure 4 1 is a block diagram of a model training device for electric facial protection based on deep learning in one embodiment;

[0056] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0058] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0059] In an exemplary embodiment, Figure 1As shown, a model training method for electric facial protection based on deep learning is provided. This embodiment uses the method applied to a terminal as an example for illustration; it is understandable that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, etc.; the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. In this embodiment, the method includes the following steps:

[0060] Step S101: acquiring sample image data of a power operation site through an image acquisition device.

[0061] Step S102 , performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type and classification processing of protective equipment wearing mode.

[0062] Step S103 : performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data.

[0063] In step S104, the labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers at the electric power operation site are wearing the electric facial protection equipment correctly.

[0064] The image acquisition device may be a device for acquiring images of a power operation site, for example, a fixed camera deployed at the operation site.

[0065] The sample image data may be image data collected during power operation on-site, including the use of facial protective equipment, for example, video data of an operation scene collected in real time by a fixed camera.

[0066] Among them, multi-dimensional classification processing can be classification processing of sample image data from different angles, including operation type classification processing, personnel type classification processing and protective equipment wearing method classification processing.

[0067] The job type classification process may be a classification of sample image data according to job requirements, for example, the sample image data may be classified into jobs requiring a mask, jobs requiring goggles, other jobs, or irrelevant jobs.

[0068] Among them, the personnel type classification processing can be the classification of sample image data according to personnel requirements, for example, the sample image data can be divided into workers who need to wear masks, workers who need to wear goggles, non-workers or invalid personnel.

[0069] Among them, the classification processing of the wearing method of protective equipment can be the classification of sample image data according to the wearing method, for example, the sample image data can be divided into wearing goggles, wearing protective glasses, wearing a protective face shield, wearing an arc shield, wearing other glasses, wearing a welding mask, not wearing any kind of goggles or not wearing a welding mask.

[0070] The labeling process may be a process of labeling the sample image data, for example, a process of labeling the sample image data as labels 0 to 10.

[0071] The labeled data may be sample image data that has been labeled, for example, data including labels 0 (mask_must_work, work requiring a mask) to 10 (no_glasses_or_no_welding_mask, not wearing any type of goggles or not wearing a welding mask).

[0072] Among them, the electric facial protection equipment recognition model to be trained can be a deep learning model used to identify the use of facial protection equipment by workers at power operation sites, and the target electric facial protection equipment recognition model can be a model that can accurately identify whether workers are wearing facial protection equipment correctly after training.

[0073] The operator may be a worker who performs work at an electric power operation site, for example, a worker who performs welding work or a worker who needs to wear goggles.

[0074] Among them, the power face protection device can be protective equipment used to protect the face safety of power workers, such as a protective mask, goggles, protective glasses, a protective face shield, an arc protection mask or a welding mask.

[0075] Optionally, the terminal collects sample image data of the power operation site through a fixed camera deployed at the power operation site, and performs multi-dimensional classification processing on the collected sample image data, including dividing the sample image data into operations requiring masks, operations requiring goggles and other operations according to operation requirements, and dividing them into operators who need to wear masks, operators who need to wear goggles and non-operators according to personnel requirements, and dividing them into wearing goggles / protective glasses / protective face shields / arc-proof masks, wearing other glasses and wearing welding masks according to wearing methods. Then, the sample image data is labeled according to the classification results, and the sample image data is labeled with labels 0 to 10. The labeled data is then used to iteratively train the power facial protective equipment recognition model to be trained, so that the power facial protective equipment recognition model can accurately identify whether the operators at the power operation site are wearing power facial protective equipment correctly, and obtain the target power facial protective equipment recognition model.

[0076] For example, the terminal collects video data of welding operation scenes through fixed cameras as sample image data, and performs multi-dimensional classification processing on the sample image data. It classifies scenes with welding equipment, strong light (light) and sparks generated by welding as operations requiring masks, and classifies live operations in the power distribution link as operations requiring goggles. Personnel who perform welding operations and are equipped with welding masks and generate flying sparks are classified as workers who need to wear masks. Personnel who perform operations such as hammering, cutting, and meter repair are classified as workers who need to wear goggles. The sample image data is then labeled. Processing, for example, labeling welding operations that require wearing a mask as label 0 (mask_must_work), and labeling operations that require wearing goggles as label 1 (protective_glasses_must_work, operations that require wearing goggles). Then, the electric facial protective equipment recognition model is trained to enable the electric facial protective equipment recognition model to monitor the work site in real time and identify whether the workers are wearing electric facial protective equipment correctly. When it is found that the workers are not wearing or wearing the wrong facial protective equipment, audio or visual prompts are issued through fixed cameras.

[0077] In the above-mentioned deep learning-based model training method for electric facial protective equipment, sample image data of the electric operation site is obtained through an image acquisition device; the sample image data is subjected to multi-dimensional classification processing to obtain the classification results of the sample image data; the multi-dimensional classification processing includes operation type classification processing, personnel type classification processing and protective equipment wearing method classification processing; according to the classification results, the sample image data is labeled to obtain labeled data of the sample image data; using the labeled data, the electric facial protective equipment recognition model to be trained is iteratively trained to obtain the target electric facial protective equipment recognition model; the target electric facial protective equipment recognition model is used to identify whether the workers at the electric operation site are wearing electric facial protective equipment correctly. This solution obtains sample image data of the power operation site through an image acquisition device, and classifies the sample image data into three dimensions: operation type, personnel type, and protective equipment wearing method, which is conducive to comprehensive and accurate identification of the use of protective equipment; then, by labeling the classified sample image data and using the labeled data to iteratively train the power facial protective equipment recognition model, it is conducive to improving the efficiency and accuracy of the model in identifying the use of protective equipment in the operation scene, thereby improving the efficiency and accuracy of the detection of workers wearing power facial protective equipment.

[0078] In an exemplary embodiment, referring to Figure 2 , perform multi-dimensional classification processing on the sample image data to obtain the classification results of the sample image data, which specifically include the following:

[0079] Step S201, performing job type classification processing on the sample image data to obtain a job type classification result of the sample image data;

[0080] Step S202, performing personnel type classification processing on the sample image data to obtain a personnel type classification result of the sample image data;

[0081] Step S203, performing protective gear wearing mode classification processing on the sample image data to obtain a protective gear wearing mode classification result of the sample image data;

[0082] Step S204 : determining the classification result of the sample image data according to the classification result of the operation type, the classification result of the personnel type and the classification result of the protective equipment wearing mode.

[0083] Among them, the job type classification result can be the data obtained after performing job type classification processing on the sample image data, for example, it can be the data obtained after classifying the sample image data into label 0 (mask_must_work, job requiring wearing a mask), label 1 (protective_glasses_must_work, job requiring wearing goggles) and label 2 (other_work, unrelated job).

[0084] Among them, the personnel type classification result can be the data obtained after performing personnel type classification processing on the sample image data, for example, it can be the data obtained after classifying the sample image data into label 3 (mask_must_worker, workers who need to wear masks), label 4 (protective_glasses_must_worker, workers who need to wear goggles), label 5 (person, non-workers) and label 6 (invalid_person, invalid personnel).

[0085] Among them, the classification result of the wearing method of protective equipment can be the data obtained after the sample image data is classified into the wearing method of protective equipment. For example, it can be the data obtained after the sample image data is classified into label 7 (protective_glasses, wearing goggles / protective glasses / protective face shield / arc shield), label 8 (other_glasses, other glasses), label 9 (welding_mask, welding mask) and label 10 (no_glasses_or_no_welding_mask, not wearing any kind of goggles or not wearing a welding mask).

[0086] Optionally, when the terminal performs multi-dimensional classification processing on the sample image data, the terminal first performs job classification processing on the sample image data, and divides the sample image data into jobs that require wearing masks (label 0, mask_must_work), jobs that require wearing goggles (label 1, protective_glasses_must_work), and irrelevant jobs (label 2, other_work), and obtains the job type classification result of the sample image data; then, the terminal performs personnel type classification processing on the sample image data, and divides the sample image data into workers who need to wear masks (label 3, mask_must_worker), workers who need to wear goggles (label 4, protective_glasses_must_worker), and non-workers (label 5, person) and invalid personnel (label 6, invalid_person), and obtain the personnel type classification result of the sample image data; then the sample image data is classified into wearing mode of protective equipment, and the sample image data is divided into wearing goggles / protective glasses / protective face shield / arc shield (label 7, protective_glasses), wearing other glasses (label 8, other_glasses), wearing welding mask (label 9, welding_mask) and not wearing any kind of goggles or not wearing welding mask (label 10, no_glasses_or_no_welding_mask), and obtain the protective equipment wearing mode classification result of the sample image data; finally, the final classification result of the sample image data is determined based on these three classification results.

[0087] The technical solution provided in this embodiment classifies the sample image data according to the job type, personnel type and protective equipment wearing method, and comprehensively analyzes the sample image data from three dimensions: job scene requirements, personnel requirements and actual wearing conditions, which is conducive to accurately identifying the protection needs and actual protection conditions in different job scenes; and then determining the final classification result by integrating the classification results of these three dimensions, which is conducive to improving the classification accuracy of the use of facial protective equipment in power operation sites, thereby improving the effect of subsequent model training.

[0088] In an exemplary embodiment, referring to Figure 3 , perform job type classification processing on the sample image data to obtain the job type classification results of the sample image data, which specifically include the following contents:

[0089] Step S301, performing operation type classification processing on the sample image data to obtain mask operation type image data, goggles operation type image data and irrelevant operation type image data;

[0090] Step S302: Determine the operation type classification result based on the mask operation type image data, the goggles operation type image data, and the irrelevant operation type image data.

[0091] Among them, the mask operation type image data can be image data of operations requiring wearing a mask obtained after performing operation type classification processing on the sample image data. For example, it can be image data containing welding operation scenes (with welding equipment, strong light and sparks generated by welding, or welding equipment and fire extinguishers).

[0092] The goggles operation type image data may be image data of an operation requiring goggles obtained by performing operation type classification processing on sample image data, for example, image data of a live operation process in a power distribution link.

[0093] Among them, the irrelevant job type image data can be image data that is irrelevant to jobs requiring wearing a mask and jobs requiring wearing goggles, which is obtained after job type classification processing is performed on the sample image data. For example, it can be image data that is irrelevant to label 0 (mask_must_work, jobs requiring wearing a mask) and label 1 (protective_glasses_must_work, jobs requiring wearing goggles).

[0094] Optionally, when the terminal performs job type classification processing on the sample image data, it first identifies whether the sample image data contains a welding operation scene. When it is detected that the sample image data contains welding equipment, strong light (light) and sparks generated by welding, or contains welding equipment and a fire extinguisher, the sample image data is classified as mask operation type image data; when it is detected that the sample image data contains a live operation process in the power distribution link, the sample image data is classified as goggles operation type image data; when it is detected that the sample image data is irrelevant to operations requiring wearing a mask and operations requiring wearing goggles, the sample image data is classified as irrelevant operation type image data; finally, the operation type classification result of the sample image data is determined based on the mask operation type image data, the goggles operation type image data and the irrelevant operation type image data.

[0095] The technical solution provided in this embodiment is conducive to accurately distinguishing the specific requirements of different work scenarios for facial protection equipment by dividing the sample image data into mask work type image data, goggles work type image data and irrelevant work type image data; and then determining the work type classification results based on these three types of work type image data, which is conducive to improving the recognition accuracy of different types of work scenes in power work sites, thereby facilitating the subsequent accurate judgment of whether the workers are wearing facial protection equipment correctly.

[0096] In an exemplary embodiment, personnel type classification processing is performed on sample image data to obtain personnel type classification results of the sample image data, which specifically includes the following contents: personnel type classification processing is performed on the sample image data to obtain mask working personnel type image data, goggles working personnel type image data, non-working personnel type image data and invalid personnel type image data; the personnel type classification results are determined based on the mask working personnel type image data, goggles working personnel type image data, non-working personnel type image data and invalid personnel type image data.

[0097] Among them, the mask operator type image data can be the image data of the operator who needs to wear a mask obtained after the personnel type classification processing of the sample image data, for example, it can be the image data of the personnel who actually perform the welding operation (in addition to the personnel, the image data also needs to include welding-related content, such as welding masks, flying sparks, strong light and smoke generated by welding, welding tools, etc.).

[0098] Among them, the image data of the goggles worker type can be the image data of the worker who needs to wear goggles obtained after the sample image data is classified into the worker type. For example, it can be the image data of the worker who actually performs hammering, cutting operations, installing electricity meters or repairing or dismantling electricity meters, and roadside pole-related operations.

[0099] Among them, the non-working personnel type image data can be the image data of non-working personnel obtained after the sample image data is classified into personnel types. For example, it can be image data containing working personnel, non-working personnel and passers-by that are not related to label 3 (mask_must_worker, working personnel who need to wear masks) and label 4 (protective_glasses_must_worker, working personnel who need to wear goggles), or it can be image data containing half-body of a person but it is uncertain what behavior he is doing.

[0100] Among them, the invalid person type image data can be the image data of invalid persons obtained after the sample image data is subjected to person type classification processing, for example, it can be the image data of persons whose body parts are more than 80% blocked, or only one body part (such as feet, hands, head, waist) appears.

[0101] Optionally, when the terminal performs personnel type classification processing on the sample image data, it first identifies whether the sample image data contains personnel who actually perform welding operations. When it is detected that the sample image data contains workers and welding-related content (including welding masks, flying sparks, strong light and smoke generated by welding, welding tools, etc.), the sample image data is classified as mask operator type image data; when it is detected that the sample image data contains operators who actually perform hammering, cutting operations, install electricity meters or repair or remove electricity meters, or perform roadside utility pole-related operations, the sample image data is classified as goggles operator type. type image data; when it is detected that the sample image data contains workers who are not related to the workers who need to wear masks and the workers who need to wear goggles, non-workers and passers-by, or a person whose half body appears but it is uncertain what behavior he is doing, the sample image data is divided into non-worker type image data; when it is detected that the sample image data contains a person who is blocked by more than 80%, or a person with only one body part (such as feet, hands, head, waist) appears, the sample image data is divided into invalid person type image data; finally, based on these four types of person type image data, the person type classification result of the sample image data is determined.

[0102] The technical solution provided in this embodiment is conducive to accurately distinguishing the specific needs of different types of personnel for facial protection equipment by dividing the sample image data into mask operator type image data, goggles operator type image data, non-operator type image data and invalid personnel type image data; and then determining the personnel type classification results based on these four types of personnel type image data is conducive to improving the recognition accuracy of different types of personnel at the power operation site, thereby facilitating the subsequent accurate judgment of whether the operators are wearing facial protection equipment correctly.

[0103] In an exemplary embodiment, after using the labeled data to iteratively train the electric facial protective equipment recognition model to be trained and obtaining the target electric facial protective equipment recognition model, it also includes the following contents: obtaining the current image data of the electric power operation site; inputting the current image data into the target electric facial protective equipment recognition model to perform electric facial protective equipment recognition processing to obtain the electric facial protective equipment recognition result of the electric power operation site; judging whether the workers at the electric power operation site are wearing the electric facial protective equipment correctly based on the electric facial protective equipment recognition result; and generating abnormal wearing warning information of the electric facial protective equipment when it is judged that the workers at the electric power operation site are not wearing the electric facial protective equipment correctly.

[0104] Among them, the current image data can be image data of the operation scene collected in real time by a fixed camera deployed at the power operation site, for example, it can be image data collected in real time including whether the operator wears facial protection equipment.

[0105] Among them, the electric facial protection equipment recognition result can be the recognition result obtained after the target electric facial protection equipment recognition model performs electric facial protection equipment recognition processing on the current image data. For example, it can be the result of identifying whether the operator is wearing goggles / protective glasses / protective face shield / arc shield, other glasses, welding mask, or not wearing any kind of goggles or welding mask.

[0106] Among them, the wearing abnormality warning information can be a warning information generated when it is determined that the workers at the power operation site are not wearing the power face protection equipment correctly.

[0107] Optionally, after completing the training of the target electric facial protection equipment recognition model, the terminal collects the current image data of the operation scene in real time through a fixed camera deployed at the electric operation site, and inputs the current image data into the target electric facial protection equipment recognition model for electric facial protection equipment recognition processing, and identifies whether the operator is wearing goggles / protective glasses / protective face shield / arc shield, other glasses, welding mask, or not wearing any kind of goggles or welding mask, to obtain the electric facial protection equipment recognition result at the electric operation site; based on the electric facial protection equipment recognition result, it is judged whether the operators at the electric operation site are wearing the electric facial protection equipment correctly; when it is judged that the operators at the electric operation site are not wearing the electric facial protection equipment correctly, an abnormal wearing warning information of the electric facial protection equipment is generated, and a warning is issued through audio or visual prompts from the on-site fixed camera, and alarm notifications from the remote monitoring center.

[0108] The technical solution provided in this embodiment, by inputting the current image data of the power operation site into a trained target power facial protection equipment recognition model for recognition processing, is conducive to realizing automated real-time monitoring of the wearing of power facial protection equipment by operators; and by timely generating wearing abnormality warning information when it is detected that the operator is not wearing the power facial protection equipment correctly, it is conducive to timely discovery of safety hazards, thereby facilitating the improvement of operation safety at the power operation site.

[0109] In an exemplary embodiment, the following contents are also included: obtaining historical electric facial protection equipment identification results at the electric power operation site; performing data analysis and processing on the historical electric facial protection equipment identification results to obtain historical wearing abnormality type information, historical wearing abnormality time information and historical wearing abnormality area information of the electric facial protection equipment at the electric power operation site; predicting future wearing abnormality information of the electric facial protection equipment at the electric power operation site based on the historical wearing abnormality type information, historical wearing abnormality time information and historical wearing abnormality area information.

[0110] Among them, the historical power facial protection equipment recognition results can be the recognition results of the use of facial protection equipment collected and stored at each power operation site, for example, it can be historical data recording whether the workers correctly wear goggles / protective glasses / protective face shields / arc masks or welding masks at different times and in different areas.

[0111] Among them, the historical wearing abnormal type information can be the violation pattern information obtained by data analysis of the historical power facial protective equipment recognition results, for example, it can be abnormal type information such as the operator not wearing any protective equipment, wearing the wrong type of protective equipment, or wearing protective equipment improperly.

[0112] Among them, the historical wearing abnormal time information can be the risk period information obtained by performing data analysis on the historical electric facial protection equipment recognition results. For example, it can be information about more wearing abnormalities in certain specific time periods (such as nighttime operations, high temperature weather operations, etc.).

[0113] Among them, the historical wearing abnormal area information can be the risk area information obtained by performing data analysis on the historical recognition results of power facial protection equipment. For example, it can be information about certain specific working areas (such as distribution rooms, etc.) where more wearing abnormalities occur.

[0114] Among them, future wearing abnormality information can be information about future protective equipment wearing problems predicted based on historical wearing abnormality type information, historical wearing abnormality time information, and historical wearing abnormality area information. For example, it can be information predicting the risk of wearing abnormality in certain specific time periods or specific areas.

[0115] Optionally, the terminal collects and stores the recognition results of electric facial protective equipment at each operation site through cameras deployed at the electric power operation site, performs data analysis and processing on the historical electric facial protective equipment recognition results, analyzes information on abnormal types such as the operator not wearing any protective equipment, wearing the wrong type of protective equipment, or wearing protective equipment improperly, and obtains historical wearing abnormal type information; analyzes information on a large number of wearing abnormalities in specific time periods such as night operations and operations in high temperature weather, and obtains historical wearing abnormal time information; analyzes information on a large number of wearing abnormalities in specific operation areas such as distribution rooms, and obtains historical wearing abnormal area information; based on these three types of historical information, big data analysis technology is used to predict future wearing abnormality information of electric facial protective equipment at the electric power operation site.

[0116] The technical solution provided in this embodiment obtains historical wearing abnormality type information, historical wearing abnormality time information and historical wearing abnormality area information by performing data analysis and processing on the historical recognition results of electric facial protection equipment at the electric power operation site, which is conducive to analyzing various problems existing in the use of electric facial protection equipment and their occurrence patterns; and then obtains future wearing abnormality information by comprehensively analyzing and predicting these three types of historical information, which is conducive to discovering potential safety hazards in advance, thereby helping to improve the operation safety at the electric power operation site.

[0117] The following application example illustrates the model training method for electric facial protection equipment based on deep learning provided by this application. This application example uses the method applied to the terminal as an example.

[0118] Power face protection plays a vital role in the power industry. Proper wearing and use of these protective gear is a key measure to ensure worker safety and improve work efficiency.

[0119] Properly wearing power work face protection, such as a face shield and goggles, can effectively block spatter generated during work. Power work, especially during repairs, welding, or cutting, can produce metal splatter. These high-speed flying metal fragments are highly dangerous and can cause serious injury if struck in the face.

[0120] Properly wearing power face protection effectively filters arc flash. In power systems, high voltage currents can generate intense arc flash. This flash is not only extremely bright, potentially damaging to the eyes, but also emits significant amounts of heat and ultraviolet radiation. The specialized materials and design of power face protection effectively filter and attenuate arc flash, protecting workers' eyes and skin from harm.

[0121] In the complex environment of power systems, a variety of environmental factors may pose a threat to operators. As a comprehensive protective equipment, protective masks can effectively reduce the adverse effects of the external environment on operators, thereby improving the safety of the entire operation process.

[0122] The specific process of this application example includes:

[0123] First, we will acquire multimodal sample data on facial protection used during power operations. This is done by using fixed cameras or operator terminals deployed at the work site to collect real-time video data of the operation scene.

[0124] Second, the collected data is categorized across multiple dimensions. Specifically, the data is categorized based on job requirements into tasks requiring masks, tasks requiring goggles, and other or unrelated tasks; based on personnel requirements, the data is categorized into workers requiring masks, workers requiring goggles, non-workers, and ineffective personnel; and based on wearing methods, the data is categorized into those wearing goggles / protective glasses / face shields / arc shields, those wearing other glasses, those wearing welding masks, and those not wearing any type of goggles or not wearing welding masks.

[0125] Third, label the facial protection equipment models according to the classification to facilitate subsequent model training. The specific labeling rules are as follows:

[0126] Tag 0 (mask_must_work, mask-required work): refers to work requiring a mask, specifically welding. Characteristics of welding work include: strong light, sparks, smoke, and dust; the presence of welding equipment, strong light, and sparks; and the availability of welding equipment and fire extinguishers.

[0127] Tag 1 (protective_glasses_must_work, work requiring goggles): refers to work requiring goggles, primarily involving live work in power distribution.

[0128] Label 2 (other_work, other jobs): refers to other job scenarios that are unrelated to label 0 and label 1.

[0129] Tag 3 (mask_must_worker): This refers to the person actually performing the welding operation. This tag should include the worker and related elements, such as the welding mask, flying sparks, strong light and smoke generated by welding, and welding tools.

[0130] Label 4 (protective_glasses_must_worker, workers who need to wear goggles): refers to workers performing the work scenarios described in label 1, including those performing hammering, cutting, meter installation, repair, and removal, and other operations related to utility poles.

[0131] Tag 5 (person, non-operating personnel): refers to other personnel unrelated to tags 3 (mask_must_worker, workers who need to wear masks) and 4 (protective_glasses_must_worker, workers who need to wear goggles), including other workers and non-operating personnel. When only half of a person's body is shown and their specific behavior cannot be determined, they are classified as non-operating personnel.

[0132] Tag 6 (invalid_person): refers to a person whose occlusion exceeds 80%, or a person whose only body part (such as feet, hands, head, or waist) is visible.

[0133] Tag 7 (protective_glasses): Indicates the wearing of goggles / safety glasses / face shields / flash shields. When marking, the head, protective gear, and helmet (if any) must be boxed together.

[0134] Tag 8 (other_glasses): refers to the wearing of non-protective glasses, such as sunglasses and myopia glasses. When marking, the head, glasses, and helmet (if any) must be selected.

[0135] Tag 9 (welding_mask): This refers to the standard welding mask worn during welding operations. When marking, only the welding mask needs to be selected.

[0136] Label 10 (no glasses or no welding mask): indicates the person is not wearing any type of glasses or face mask. The entire head and helmet (if any) must be selected.

[0137] Feature data related to facial protection equipment and feature data related to classification are extracted as sample data.

[0138] Fourth, deep learning methods are used to train with sample data, optimizing model performance by adjusting the parameters of the classification model. Specifically, the extracted features are fed into these models for training, enabling them to accurately identify whether workers are wearing facial protection.

[0139] Fifth, the acquired multimodal data is input into the trained facial protective equipment classification model to perform classification prediction of facial protective equipment.

[0140] Sixth, establish a real-time feedback and early warning mechanism. The system (terminal) monitors the work site in real time and immediately triggers an early warning signal when it detects that workers are not wearing or are wearing facial protection incorrectly. Warning methods include audio / visual prompts from fixed cameras on site or workers' equipment, as well as alarm notifications sent to a remote monitoring center, ensuring that workers make timely corrections and improving work safety.

[0141] Seventh, historical data analysis and trend forecasting are conducted. The system collects and stores information on facial protection usage at each work site and uses big data analysis to identify common violation patterns, high-risk time periods or areas, and the distribution of workers' protective awareness. Based on these data analysis results, the system can predict potential protective gear wearing issues, providing data support for the development of targeted safety training and regulatory measures.

[0142] The technical solution provided in this application example can improve the accuracy of identifying the wearing status of facial protective equipment.

[0143] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0144] Based on the same inventive concept, the embodiment of the present application also provides a deep learning-based electric facial protection model training device for implementing the deep learning-based electric facial protection model training method involved above. The implementation solution provided by the device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more embodiments of the deep learning-based electric facial protection model training device provided below can be found in the above limitations of the deep learning-based electric facial protection model training method, and will not be repeated here.

[0145] In an exemplary embodiment, Figure 4As shown, a model training device for electric facial protection based on deep learning is provided. The model training device 400 for electric facial protection based on deep learning may include:

[0146] The data acquisition module 401 is used to acquire sample image data of the power operation site through an image acquisition device;

[0147] The data classification module 402 is used to perform multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of work type, classification processing of personnel type, and classification processing of protective equipment wearing mode;

[0148] The data annotation module 403 is used to perform annotation processing on the sample image data according to the classification result to obtain the annotation data of the sample image data;

[0149] The model training module 404 is used to use the labeled data to iteratively train the electric facial protection equipment recognition model to be trained to obtain the target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers at the electric operation site are wearing the electric facial protection equipment correctly.

[0150] In an exemplary embodiment, the data classification module 402 is also used to perform job type classification processing on the sample image data to obtain the job type classification result of the sample image data; perform personnel type classification processing on the sample image data to obtain the personnel type classification result of the sample image data; perform protective equipment wearing mode classification processing on the sample image data to obtain the protective equipment wearing mode classification result of the sample image data; determine the classification result of the sample image data based on the job type classification result, the personnel type classification result and the protective equipment wearing mode classification result.

[0151] In an exemplary embodiment, the data classification module 402 is also used to perform job type classification processing on the sample image data to obtain mask job type image data, goggles job type image data and irrelevant job type image data; and determine the job type classification result based on the mask job type image data, goggles job type image data and irrelevant job type image data.

[0152] In an exemplary embodiment, the data classification module 402 is also used to perform personnel type classification processing on the sample image data to obtain mask operator type image data, goggles operator type image data, non-operating personnel type image data and invalid personnel type image data; and determine the personnel type classification result based on the mask operator type image data, goggles operator type image data, non-operating personnel type image data and invalid personnel type image data.

[0153] In an exemplary embodiment, the device 400 also includes: an information generation module for obtaining current image data of the power operation site; inputting the current image data into the target power facial protective equipment recognition model for power facial protective equipment recognition processing to obtain the power facial protective equipment recognition result of the power operation site; judging whether the workers at the power operation site are wearing the power facial protective equipment correctly based on the power facial protective equipment recognition result; and generating abnormal wearing warning information of the power facial protective equipment when it is judged that the workers at the power operation site are not wearing the power facial protective equipment correctly.

[0154] In an exemplary embodiment, the device 400 also includes: an information prediction module for obtaining historical electric facial protective equipment identification results at the electric power operation site; performing data analysis and processing on the historical electric facial protective equipment identification results to obtain historical wearing abnormal type information, historical wearing abnormal time information and historical wearing abnormal area information of the electric facial protective equipment at the electric power operation site; and predicting future wearing abnormal information of the electric facial protective equipment at the electric power operation site based on the historical wearing abnormal type information, historical wearing abnormal time information and historical wearing abnormal area information.

[0155] Each module in the deep learning-based model training device for power face protection can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in hardware form, or stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0156] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 5As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, while the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be achieved via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a model training method for electric face protection based on deep learning. The display unit of the computer device is used to produce visual images and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse.

[0157] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0158] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0159] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0160] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0161] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0162] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A model training method for electric face protection based on deep learning, characterized in that: The method comprises: Acquire sample image data of the power operation site through an image acquisition device; Performing multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of job type, classification processing of personnel type, and classification processing of protective equipment wearing mode; performing labeling processing on the sample image data according to the classification result to obtain labeling data of the sample image data; The labeled data is used to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

2. The method according to claim 1, characterized in that The performing multi-dimensional classification processing on the sample image data to obtain the classification result of the sample image data includes: performing the job type classification process on the sample image data to obtain a job type classification result of the sample image data; Performing the personnel type classification process on the sample image data to obtain a personnel type classification result of the sample image data; Performing the protective gear wearing mode classification processing on the sample image data to obtain a protective gear wearing mode classification result of the sample image data; The classification result of the sample image data is determined according to the classification result of the operation type, the classification result of the personnel type and the classification result of the protective equipment wearing method.

3. The method according to claim 2, characterized in that The performing the job type classification process on the sample image data to obtain the job type classification result of the sample image data includes: Performing the operation type classification processing on the sample image data to obtain mask operation type image data, goggles operation type image data and irrelevant operation type image data; The operation type classification result is determined based on the mask operation type image data, the goggles operation type image data and the irrelevant operation type image data.

4. The method according to claim 2, characterized in that The performing the personnel type classification process on the sample image data to obtain the personnel type classification result of the sample image data includes: Performing personnel type classification processing on the sample image data to obtain mask operator type image data, goggles operator type image data, non-operator type image data, and invalid person type image data; The personnel type classification result is determined based on the mask worker type image data, the goggles worker type image data, the non-worker type image data and the invalid worker type image data.

5. The method according to claim 1, wherein After iteratively training the electric facial protection device recognition model to be trained using the labeled data to obtain a target electric facial protection device recognition model, the method further includes: Acquiring current image data of the power operation site; Inputting the current image data into the target power facial protection equipment recognition model to perform power facial protection equipment recognition processing to obtain the power facial protection equipment recognition result at the power operation site; Determining whether the operator at the power operation site is wearing the power face protection device correctly based on the recognition result of the power face protection device; When it is determined that the worker at the power operation site is not wearing the power face protector correctly, abnormal wearing warning information of the power face protector is generated.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Obtaining historical power face protection equipment identification results at the power operation site; Performing data analysis and processing on the historical power face protection equipment recognition results to obtain historical abnormal wearing type information, historical abnormal wearing time information, and historical abnormal wearing area information of the power face protection equipment at the power operation site; Based on the historical wearing abnormality type information, the historical wearing abnormality time information and the historical wearing abnormality area information, future wearing abnormality information of the electric power face protection device at the electric power operation site is predicted.

7. A model training device for electric facial protection based on deep learning, characterized in that: The device comprises: A data acquisition module is used to acquire sample image data of the power operation site through an image acquisition device; a data classification module, configured to perform multi-dimensional classification processing on the sample image data to obtain classification results of the sample image data; the multi-dimensional classification processing includes classification processing of operation type, classification processing of personnel type, and classification processing of protective equipment wearing mode; a data annotation module, configured to perform annotation processing on the sample image data according to the classification result to obtain annotation data of the sample image data; The model training module is used to use the labeled data to iteratively train the electric facial protection equipment recognition model to be trained to obtain a target electric facial protection equipment recognition model; the target electric facial protection equipment recognition model is used to identify whether the workers in the electric operation site are wearing electric facial protection equipment correctly.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images