New energy power station inspection violation behavior early warning method and device based on image analysis

Abstract

The invention discloses a new energy power station inspection violation behavior early warning method and system based on image analysis. The method comprises the following steps: determining employee standard wearing features and evaluation image features in different scenes according to a multi-scene data model of a new energy power station; the method comprises the following steps: collecting product structure data of various types of protection products and precise image data when a user wears various types of protection products, carrying out association and labeling to generate a model training set, and training a preset deep learning model to generate a target detection model based on deep learning; performing violation evaluation on the real-time inspection images of the employees through the target detection model; and according to the real-time scene of the new energy power station, carrying out authenticity analysis and directional alarm on the evaluation result based on the employee standard wearing features and the evaluation image features. Therefore, the inspection personnel can execute related safety measures according to regulations in the inspection process, the probability of safety accidents is effectively reduced, and the safety, the stability and the inspection management efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of remote monitoring technology, and in particular to an image analysis-based new energy power station inspection violation behavior early warning method and system. Background Art

[0002] As an important part of clean energy, new energy power stations play an important role in ensuring national energy security and reducing environmental pollution. However, with the continuous expansion of the scale of new energy power stations, their operation and maintenance management has become more and more complicated. Daily inspection of equipment in new energy power stations is a daily key task. According to the characteristics of new energy power stations, there is a large inspection range and a relatively remote area, which is prone to safety hazards. Human inspection can find potential problems more quickly. Therefore, different inspection personnel are assigned to perform tasks for different inspection tasks. The personnel structure of operation and maintenance inspection is more complicated than that of traditional power stations, involving multiple personnel and uneven personnel quality. During the inspection process, personnel often have many unsafe behaviors, such as inadequate safety measures, which increases the probability of safety accidents and cannot effectively manage inspection personnel. Summary of the invention

[0003] In response to the problems shown above, the present invention provides a new energy power station inspection violation behavior warning method and system based on image analysis to solve the problem mentioned in the background technology that personnel often have many unsafe behaviors during the inspection process, which increases the probability of safety accidents and makes it impossible to effectively manage inspection personnel.

[0004] A new energy power station inspection violation behavior early warning method based on image analysis, comprising the following steps:

[0005] Build a multi-scenario data model for new energy power stations, and determine the standard wear characteristics and evaluation image features of employees in different scenarios based on the multi-scenario data model;

[0006] Collect product structure data of various types of protective products and accurate image data of users wearing various types of protective products, associate and annotate them to generate model training sets;

[0007] The preset deep learning model is trained through the model training set to generate a deep learning-based target detection model, and the target detection model is used to evaluate violations of employees' real-time inspection images;

[0008] According to the real-time scenario of the new energy power station, the authenticity of the evaluation results is analyzed and a directional alarm is issued based on the standard wearing characteristics of employees and the evaluation image characteristics.

[0009] Preferably, the multi-scenario data model of the new energy power station is constructed, and the standard wearing features and evaluation image features of employees in different scenarios are determined according to the multi-scenario data model, including:

[0010] Acquire first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions;

[0011] Collecting shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrating the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of a new energy power station;

[0012] Build a multi-scenario data model for new energy power plants based on multi-scenario data sets, and determine the dress and behavior restrictions in different scenarios by combining the multi-scenario data model with the preset inspection specification requirements;

[0013] The standard wearing characteristics of employees in different scenarios are determined based on the clothing and behavioral restrictions, and the evaluation image features are determined based on the visual clarity in different scenarios.

[0014] Preferably, the collecting of product structure data of various types of protective products and accurate image data of users wearing various types of protective products and associating and annotating them to generate a model training set includes:

[0015] Determine the wearing attributes of various types of protective products, and determine the product scale of various types of protective products based on the wearing attributes;

[0016] Determine the best shooting angle and best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance;

[0017] Determine wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of users wearing various types of protective products according to the wearing standard identification parameters, and select accurate image data of users wearing various types of protective products according to the identification results;

[0018] The product structure data of various types of protective products are associated with the precise image data of users wearing various types of protective products and data annotation is performed to generate the initial data set;

[0019] The initial data set is cleaned to obtain a model training set, where data cleaning includes removing low-resolution, blurred, and distorted data.

[0020] Preferably, the method of training a preset deep learning model through a model training set to generate a deep learning-based target detection model, and performing violation assessment on the real-time inspection images of employees through the target detection model, includes:

[0021] Extract target features and target wearing features of various types of protective products based on the model training set through a preset deep learning model;

[0022] The preset deep learning model is trained according to the target features and target wearing features of each type of protective product to generate a deep learning-based target detection model;

[0023] Through the target detection model, based on the candidate frame classification method, the frame regression is performed according to the real-time inspection image of the employee to obtain the real-time wearing features of the employee;

[0024] Compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

[0025] Preferably, the authenticity analysis and directional alarm of the evaluation results based on the real-time scene of the new energy power station and the standard wearing characteristics of the employees and the evaluation image characteristics include:

[0026] Detect the real-time weather parameters of the new energy power station, and determine the real-time weather scene of the new energy power station according to the real-time weather parameters;

[0027] Determine the standard wearing characteristics of target employees under real-time weather scenarios, and determine violation identification description parameters based on the standard wearing characteristics of target employees;

[0028] Verify the assessment results according to the violation identification description parameters, perform authenticity analysis based on the verification results, and determine the wearing details based on the assessment image features;

[0029] Determine the wearing subject and its standard description parameters based on the wearing details, and conduct a secondary authenticity analysis on the evaluation results based on the wearing subject and its standard description parameters;

[0030] Based on the two authenticity analysis data, the actual violation of the employee is determined and a targeted alarm is issued.

[0031] An early warning system for inspection violations of new energy power stations based on image analysis, the system comprising:

[0032] A determination module is used to construct a multi-scenario data model for a new energy power station, and determine the standard wearing features and evaluation image features of employees in different scenarios according to the multi-scenario data model;

[0033] A generation module is used to collect product structure data of various types of protective products and accurate image data of users wearing various types of protective products, and associate and annotate them to generate a model training set;

[0034] An evaluation module is used to train a preset deep learning model through a model training set to generate a deep learning-based target detection model, and to perform violation evaluation on employees' real-time inspection images through the target detection model;

[0035] The analysis and alarm module is used to analyze the authenticity of the evaluation results and issue directional alarms based on the real-time scene of the new energy power station and the standard wearing characteristics of employees and the evaluation image characteristics.

[0036] Preferably, the determining module includes:

[0037] The first acquisition submodule is used to acquire first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions;

[0038] A data collection and integration submodule, used to collect the shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrate the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of the new energy power station;

[0039] The first determination submodule is used to build a multi-scenario data model of a new energy power station based on a multi-scenario data set, and determine the clothing and behavior restriction factors in different scenarios through the multi-scenario data model combined with preset inspection specification requirements;

[0040] The second determination submodule is used to determine the standard wearing features of employees in different scenarios according to the wearing and behavior restriction factors, and to determine the evaluation image features based on the visual clarity in different scenarios.

[0041] Preferably, the generating module comprises:

[0042] The third determination submodule is used to determine the wearing attributes of each type of protective product, and determine the product scale of each type of protective product based on the wearing attributes;

[0043] A collection submodule, used to determine the best shooting angle and the best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance;

[0044] A screening submodule is used to determine the wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of the user wearing various types of protective products according to the wearing standard identification parameters, and screen out the accurate image data of the user wearing various types of protective products according to the identification results;

[0045] A generation submodule is used to associate and annotate the product structure data of various types of protective products with the precise image data of users wearing various types of protective products to generate an initial data set;

[0046] The data cleaning submodule is used to clean the initial data set to obtain the model training set, where data cleaning includes: removing low-resolution, blurred and distorted low-quality data.

[0047] Preferably, the evaluation module comprises:

[0048] An extraction submodule is used to extract target features and target wearing features of various types of protective products based on a model training set using a preset deep learning model;

[0049] A training submodule, used to train a preset deep learning model according to target features and target wearing features of various types of protective products to generate a deep learning-based target detection model;

[0050] The second acquisition submodule is used to obtain the real-time wearing features of the employees by performing frame regression based on the real-time inspection images of the employees through the target detection model based on the candidate frame classification method;

[0051] The comparison submodule is used to compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

[0052] Preferably, the analysis and alarm module includes:

[0053] The fourth determination submodule is used to detect the real-time weather parameters of the new energy power station and determine the real-time weather scene of the new energy power station according to the real-time weather parameters;

[0054] A fifth determination submodule is used to determine the standard wearing characteristics of the target employee under the real-time weather scenario, and determine the violation identification description parameters according to the standard wearing characteristics of the target employee;

[0055] The first analysis submodule is used to check the evaluation results according to the violation identification description parameters, perform authenticity analysis according to the check results, and determine the wearing details according to the evaluation image features;

[0056] The second analysis submodule is used to determine the wearing subject and its standard description parameters according to the wearing details, and to perform a secondary authenticity analysis on the evaluation results according to the wearing subject and its standard description parameters;

[0057] The alarm submodule is used to determine the actual violation of employees based on the two authenticity analysis data and issue a targeted alarm.

[0058] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0061] Figure 1 A workflow diagram of a new energy power station inspection violation warning method based on image analysis provided by the present invention;

[0062] Figure 2 Another work flow chart of a new energy power station inspection violation warning method based on image analysis provided by the present invention;

[0063] Figure 3 This is a structural schematic diagram of a new energy power station inspection violation warning system based on image analysis provided by the present invention;

[0064] Figure 4 This is a structural schematic diagram of a determination module in an early warning system for inspection violations of new energy power stations based on image analysis provided by the present invention. DETAILED DESCRIPTION

[0065] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0066] As an important part of clean energy, new energy power stations play an important role in ensuring national energy security and reducing environmental pollution. However, with the continuous expansion of the scale of new energy power stations, their operation and maintenance management has become more and more complicated. Daily inspection of equipment in new energy power stations is a daily key task. According to the characteristics of new energy power stations, there is a large inspection range and a relatively remote area, which is prone to safety hazards. Human inspection can discover potential problems more quickly. Therefore, different inspection personnel are assigned to perform tasks for different inspection tasks. The personnel structure of operation and maintenance inspection is more complicated than that of traditional power stations, involving multiple personnel and uneven personnel quality. During the inspection process, personnel often have many unsafe behaviors, such as inadequate safety measures, which increases the probability of safety accidents and cannot effectively manage inspection personnel. In order to solve the above problems, this embodiment discloses a method for detecting and warning violations of inspection personnel based on image analysis.

[0067] A new energy power station inspection violation behavior early warning method based on image analysis, such as Figure 1 As shown, the following steps are included:

[0068] Step S101: construct a multi-scenario data model for a new energy power station, and determine standard wearing features and evaluation image features of employees in different scenarios according to the multi-scenario data model;

[0069] Step S102: collecting product structure data of various types of protective products and accurate image data of users wearing various types of protective products, and performing association and annotation to generate a model training set;

[0070] Step S103: training a preset deep learning model through a model training set to generate a deep learning-based target detection model, and performing violation assessment on the real-time inspection images of employees through the target detection model;

[0071] Step S104: Analyze the authenticity of the evaluation results and issue a directional alarm based on the real-time scene of the new energy power station, the standard wearing features of the employees and the evaluation image features.

[0072] In this embodiment, the multi-scenario data model is represented as a statistical model of field data under different working scenarios of a new energy power station;

[0073] In this embodiment, the evaluation image features are represented as image clarity and resolution features for performing wearing feature evaluation;

[0074] In this embodiment, the product structure data is represented as appearance structure data of various types of protective products.

[0075] The working principle of the above technical solution is: construct a multi-scenario data model of a new energy power station, and determine the standard wearing characteristics and evaluation image characteristics of employees in different scenarios according to the multi-scenario data model; collect product structure data of various types of protective products and accurate image data of users when wearing various types of protective products, and associate and annotate them to generate a model training set; train a preset deep learning model with the model training set to generate a deep learning-based target detection model, and perform violation assessment on the real-time inspection images of employees through the target detection model; perform authenticity analysis and directional alarm on the assessment results based on the standard wearing characteristics and evaluation image characteristics of employees according to the real-time scenarios of the new energy power station.

[0076] The beneficial effects of the above technical solution are: by determining the standard wearing features and evaluation image features of employees in different scenarios, the standard wearing image reference parameters of patrol personnel can be accurately generated, thereby laying a reference condition for subsequent violation assessment. Furthermore, by constructing a target detection model based on deep learning, the violation behavior of employees can be quickly and accurately determined based on the real-time patrol images of employees through the target detection model, and then an alarm is issued, ensuring the standardization of safety measures for relevant personnel during the patrol process. Patrol personnel can implement relevant safety measures in accordance with regulations during the patrol process, effectively reducing the probability of safety accidents, improving safety and stability as well as patrol management efficiency, and solving the problem mentioned in the prior art that personnel often have many unsafe behaviors during the patrol process, which increases the probability of safety accidents and makes it impossible to effectively manage patrol personnel.

[0077] In this embodiment, the method further includes: monitoring the illegal behaviors of employees during the inspection process, specifically:

[0078] Obtain the employee's inspection tasks, determine the employee's equipment to be inspected based on the inspection tasks, and obtain potential security threat factors of the equipment to be inspected;

[0079] Determine the regional inspection violation scenarios of the equipment to be inspected based on potential security threat factors, and obtain the violation triggering behavior characteristics of the regional inspection violation scenarios;

[0080] Analyze the monitored limb objects and object operation characteristics of the violation triggering behavior characteristics, and determine the violation evidence chain of each violation triggering behavior characteristic based on the monitored limb objects and object operation characteristics;

[0081] Determine multiple violation vectors of each violation triggering behavior feature based on the violation evidence chain, and determine a mapping element of each violation vector;

[0082] determining an image acquisition factor of the mapping element, and determining an image verification matrix of the mapping element based on the image acquisition factor;

[0083] Determine the effective verification probability of the mapping element according to the image verification matrix based on the verification effective prediction model, and determine the repeated verification frequency of each mapping element according to the effective verification probability;

[0084] Determine the judgment delay of each violation vector according to the repeated verification frequency of each verification element, and determine the median judgment delay of each violation triggering behavior feature according to the judgment delay of each violation vector and a preset synchronization judgment factor;

[0085] Determine a video capture duration limit parameter or an image frame capture quantity limit parameter according to a median judgment delay of each violation triggering behavior feature;

[0086] According to the video acquisition time limit parameter or the image frame acquisition quantity limit parameter, the employee's inspection process video is collected and violation monitoring is performed.

[0087] In this embodiment, the potential safety threat factors are represented by physical factors or chemical factors that may affect the safety of the equipment to be inspected, such as open flames.

[0088] In this embodiment, the violation triggering behavior feature is represented by the triggering behavior of the inspection violation item, such as: smoking during the inspection process;

[0089] In this embodiment, the monitored limb object is represented by the limb object that performs the illegal behavior, for example: the limb object that performs smoking is the hand;

[0090] In this embodiment, the object operation feature is represented as the operation description feature of the limb object, for example: the description feature of the hand is holding a cigarette or lighting a cigarette;

[0091] In this embodiment, the violation evidence chain is represented as the evidence chain for the overall determination of the violation behavior. For example, the overall evidence chain for confirming the employee's smoking violation behavior is the employee holding the cigarette, lighting the cigarette, and holding the lit cigarette.

[0092] The beneficial effects of the above technical solution are: by synchronously monitoring the violations of employees during the inspection process, more comprehensive safety monitoring of employees other than wearable safety can be achieved, safety accidents caused by employees' violations can be avoided, and the safety and reliability of inspections can be improved. Furthermore, by determining the video acquisition time limit parameters or the image frame acquisition number limit parameters, the violations of employees can be more accurately evaluated based on the monitoring video, thereby improving the judgment accuracy and effectiveness.

[0093] In one embodiment, Figure 2 As shown, the multi-scenario data model of the new energy power station is constructed, and the standard wearing features and evaluation image features of employees in different scenarios are determined according to the multi-scenario data model, including:

[0094] Step S201, obtaining first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions;

[0095] Step S202: collecting shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrating the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of the new energy power station;

[0096] Step S203: construct a multi-scenario data model of the new energy power station according to the multi-scenario data set, and determine the clothing and behavior restriction factors in different scenarios by combining the multi-scenario data model with the preset inspection specification requirements;

[0097] Step S204: determining standard employee wearing features in different scenarios based on wearing and behavior restriction factors, and determining evaluation image features based on visual clarity in different scenarios.

[0098] The beneficial effects of the above technical solution are: generating a multi-scenario data set of a new energy power station based on environmental data under normal weather conditions, environmental data under abnormal weather conditions and shooting perspective data, generating a multi-scenario data model, determining the wearing and behavior restrictions in different scenarios and performing image feature evaluation, which can improve recognition accuracy, thereby more accurately determining the standard wearing conditions of employees in each inspection area, and laying a reference for subsequent violation evaluations.

[0099] In one embodiment, the collecting of product structure data of various types of protective products and accurate image data of users wearing various types of protective products and associating and annotating them to generate a model training set includes:

[0100] Determine the wearing attributes of various types of protective products, and determine the product scale of various types of protective products based on the wearing attributes;

[0101] Determine the best shooting angle and best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance;

[0102] Determine wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of users wearing various types of protective products according to the wearing standard identification parameters, and select accurate image data of users wearing various types of protective products according to the identification results;

[0103] The product structure data of various types of protective products are associated with the precise image data of users wearing various types of protective products and data annotation is performed to generate the initial data set;

[0104] The initial data set is cleaned to obtain a model training set, where data cleaning includes removing low-resolution, blurred, and distorted data.

[0105] The beneficial effects of the above technical solution are: according to the optimal shooting angle and the optimal shooting distance of the product scale of each type of protective product, the product structure data of each type of protective product is collected, and the initial data set is generated and cleaned in combination with the precise image data of users when wearing each type of protective product, which can ensure the quality and reliability of the data. At the same time, obtaining the model training set will provide high-quality input for model training to improve the accuracy of the model.

[0106] In one embodiment, the method of training a preset deep learning model through a model training set to generate a deep learning-based target detection model, and performing a violation assessment on the real-time inspection images of employees through the target detection model, includes:

[0107] Extract target features and target wearing features of various types of protective products based on the model training set through a preset deep learning model;

[0108] The preset deep learning model is trained according to the target features and target wearing features of each type of protective product to generate a deep learning-based target detection model;

[0109] Through the target detection model, based on the candidate frame classification method, the frame regression is performed according to the real-time inspection image of the employee to obtain the real-time wearing features of the employee;

[0110] Compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

[0111] The beneficial effects of the above technical solution are: the deep learning model is trained according to the target features and target wearing features of the protective product, the target detection model is generated, and the real-time wearing features of the employees are obtained, so as to evaluate the employees' violations, which can improve the accuracy of the evaluation, and can timely obtain and monitor the employees' wearing qualification in real time. Furthermore, by performing wearing feature detection in a frame regression manner, the employees can be quickly located, and the frame of the worn protective product can be located to accurately determine the wearing features, thereby improving the detection efficiency and accuracy.

[0112] In one embodiment, the authenticity analysis and directional alarm of the evaluation results based on the real-time scene of the new energy power station and the standard wearing features of the employees and the evaluation image features include:

[0113] Detect the real-time weather parameters of the new energy power station, and determine the real-time weather scene of the new energy power station according to the real-time weather parameters;

[0114] Determine the standard wearing characteristics of target employees under real-time weather scenarios, and determine violation identification description parameters based on the standard wearing characteristics of target employees;

[0115] Verify the assessment results according to the violation identification description parameters, perform authenticity analysis based on the verification results, and determine the wearing details based on the assessment image features;

[0116] Determine the wearing subject and its standard description parameters based on the wearing details, and conduct a secondary authenticity analysis on the evaluation results based on the wearing subject and its standard description parameters;

[0117] Based on the two authenticity analysis data, the actual violation of the employee is determined and a targeted alarm is issued.

[0118] The beneficial effects of the above technical solution are: determining the violation identification description parameters according to the wearing characteristics of employees in real-time weather scenarios, determining the wearing details and conducting authenticity analysis, determining the actual violation situation of the employees and conducting targeted alarms, being able to objectively evaluate and handle the violations of employees, make real-time corrections, improve safety and compliance, and reduce the risk of accidents caused by violations.

[0119] In one embodiment, this embodiment also discloses a new energy power station inspection violation behavior early warning system based on image analysis, such as Figure 3 As shown, the system includes:

[0120] A determination module 301 is used to construct a multi-scenario data model of a new energy power station, and determine standard wearing features and evaluation image features of employees in different scenarios according to the multi-scenario data model;

[0121] A generation module 302 is used to collect product structure data of various types of protective products and accurate image data of users wearing various types of protective products, and associate and annotate them to generate a model training set;

[0122] An evaluation module 303 is used to train a preset deep learning model through a model training set to generate a deep learning-based target detection model, and to perform violation evaluation on the real-time inspection images of employees through the target detection model;

[0123] The analysis and alarm module 304 is used to perform authenticity analysis and directional alarm on the evaluation results based on the real-time scene of the new energy power station and the standard wearing characteristics of the employees and the evaluation image characteristics.

[0124] The working principle and beneficial effects of the above technical solution have been explained in the method embodiment and will not be repeated here.

[0125] In one embodiment, Figure 4 As shown, the determination module 301 includes:

[0126] The first acquisition submodule 3011 is used to acquire first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions;

[0127] The data collection and integration submodule 3012 is used to collect the shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrate the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of the new energy power station;

[0128] The first determination submodule 3013 is used to construct a multi-scenario data model of a new energy power station according to the multi-scenario data set, and determine the clothing and behavior restriction factors in different scenarios by combining the multi-scenario data model with the preset inspection specification requirements;

[0129] The second determination submodule 3014 is used to determine the standard wearing features of employees in different scenarios according to the wearing and behavior restriction factors, and determine the evaluation image features based on the visual clarity in different scenarios.

[0130] In one embodiment, the generating module comprises:

[0131] The third determination submodule is used to determine the wearing attributes of each type of protective product, and determine the product scale of each type of protective product based on the wearing attributes;

[0132] A collection submodule, used to determine the best shooting angle and the best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance;

[0133] A screening submodule is used to determine the wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of the user wearing various types of protective products according to the wearing standard identification parameters, and screen out the accurate image data of the user wearing various types of protective products according to the identification results;

[0134] A generation submodule is used to associate and annotate the product structure data of various types of protective products with the precise image data of users wearing various types of protective products to generate an initial data set;

[0135] The data cleaning submodule is used to clean the initial data set to obtain the model training set, where data cleaning includes: removing low-resolution, blurred and distorted low-quality data.

[0136] In one embodiment, the evaluation module comprises:

[0137] An extraction submodule is used to extract target features and target wearing features of various types of protective products based on a model training set using a preset deep learning model;

[0138] A training submodule, used to train a preset deep learning model according to target features and target wearing features of various types of protective products to generate a deep learning-based target detection model;

[0139] The second acquisition submodule is used to obtain the real-time wearing features of the employees by performing frame regression based on the real-time inspection images of the employees through the target detection model based on the candidate frame classification method;

[0140] The comparison submodule is used to compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

[0141] In one embodiment, the analysis and alarm module includes:

[0142] The fourth determination submodule is used to detect the real-time weather parameters of the new energy power station and determine the real-time weather scene of the new energy power station according to the real-time weather parameters;

[0143] A fifth determination submodule is used to determine the standard wearing characteristics of the target employee under the real-time weather scenario, and determine the violation identification description parameters according to the standard wearing characteristics of the target employee;

[0144] The first analysis submodule is used to check the evaluation results according to the violation identification description parameters, perform authenticity analysis according to the check results, and determine the wearing details according to the evaluation image features;

[0145] The second analysis submodule is used to determine the wearing subject and its standard description parameters according to the wearing details, and to perform a secondary authenticity analysis on the evaluation results according to the wearing subject and its standard description parameters;

[0146] The alarm submodule is used to determine the actual violation of employees based on the two authenticity analysis data and issue a targeted alarm.

[0147] Those skilled in the art should understand that the first and second in the present invention merely refer to different application stages.

[0148] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0149] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A new energy power station inspection violation behavior early warning method based on image analysis, characterized in that: The following steps are involved: Build a multi-scenario data model for new energy power stations, and determine the standard wear characteristics and evaluation image features of employees in different scenarios based on the multi-scenario data model; Collect product structure data of various types of protective products and accurate image data of users wearing various types of protective products, associate and annotate them to generate model training sets; The preset deep learning model is trained through the model training set to generate a deep learning-based target detection model, and the target detection model is used to evaluate violations of employees' real-time inspection images; According to the real-time scenario of the new energy power station, the authenticity of the evaluation results is analyzed and a directional alarm is issued based on the standard wearing characteristics of employees and the evaluation image characteristics.

2. According to the image analysis-based early warning method for inspection violations in new energy power stations according to claim 1, it is characterized in that: The multi-scenario data model of the new energy power station is constructed, and the standard wearing features and evaluation image features of employees in different scenarios are determined according to the multi-scenario data model, including: Acquire first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions; Collecting shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrating the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of a new energy power station; Build a multi-scenario data model for new energy power plants based on multi-scenario data sets, and determine the dress and behavior restrictions in different scenarios by combining the multi-scenario data model with the preset inspection specification requirements; The standard wearing characteristics of employees in different scenarios are determined based on the clothing and behavioral restrictions, and the evaluation image features are determined based on the visual clarity in different scenarios.

3. According to the image analysis-based early warning method for inspection violations in new energy power stations according to claim 1, it is characterized in that: The collecting of product structure data of various types of protective products and accurate image data of users wearing various types of protective products and associating and annotating them to generate a model training set includes: Determine the wearing attributes of various types of protective products, and determine the product scale of various types of protective products based on the wearing attributes; Determine the best shooting angle and best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance; Determine wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of users wearing various types of protective products according to the wearing standard identification parameters, and select accurate image data of users wearing various types of protective products according to the identification results; The product structure data of various types of protective products are associated with the precise image data of users wearing various types of protective products and data annotation is performed to generate the initial data set; The initial data set is cleaned to obtain a model training set, where data cleaning includes removing low-resolution, blurred, and distorted data.

4. According to claim 1, the method for early warning of violations in inspection of new energy power stations based on image analysis is characterized in that: The method of training a preset deep learning model through a model training set to generate a deep learning-based target detection model, and performing a violation assessment on the real-time inspection images of employees through the target detection model, includes: Extract target features and target wearing features of various types of protective products based on the model training set through the preset deep learning model; The preset deep learning model is trained according to the target features and target wearing features of each type of protective product to generate a deep learning-based target detection model; Through the target detection model, based on the candidate frame classification method, the frame regression is performed according to the real-time inspection image of the employee to obtain the real-time wearing features of the employee; Compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

5. According to claim 1, the method for early warning of violations in inspection of new energy power stations based on image analysis is characterized in that: The authenticity analysis and directional alarm of the evaluation results based on the real-time scene of the new energy power station and the standard wearing characteristics of the employees and the evaluation image characteristics include: Detect the real-time weather parameters of the new energy power station, and determine the real-time weather scene of the new energy power station according to the real-time weather parameters; Determine the standard wearing characteristics of target employees under real-time weather scenarios, and determine violation identification description parameters based on the standard wearing characteristics of target employees; Verify the assessment results according to the violation identification description parameters, perform authenticity analysis based on the verification results, and determine the wearing details based on the assessment image features; Determine the wearing subject and its standard description parameters based on the wearing details, and conduct a secondary authenticity analysis on the evaluation results based on the wearing subject and its standard description parameters; Based on the two authenticity analysis data, the actual violation of the employee is determined and a targeted alarm is issued.

6. A warning system for inspection violations of new energy power stations based on image analysis, characterized in that: The system includes: A determination module is used to construct a multi-scenario data model for a new energy power station, and determine the standard wearing features and evaluation image features of employees in different scenarios according to the multi-scenario data model; A generation module is used to collect product structure data of various types of protective products and accurate image data of users wearing various types of protective products, and associate and annotate them to generate a model training set; An evaluation module is used to train a preset deep learning model through a model training set to generate a deep learning-based target detection model, and to perform violation evaluation on employees' real-time inspection images through the target detection model; The analysis and alarm module is used to analyze the authenticity of the evaluation results and issue directional alarms based on the real-time scene of the new energy power station and the standard wearing characteristics of employees and the evaluation image characteristics.

7. The new energy power station inspection violation warning system based on image analysis according to claim 6 is characterized in that: The determining module comprises: The first acquisition submodule is used to acquire first environmental data of different working areas of the new energy power station under normal weather conditions and second environmental data under abnormal weather conditions; A data collection and integration submodule, used to collect the shooting angle data of each working area at different shooting distances and different shooting resolutions, and integrate the shooting angle data with the first environment data and the second environment data to generate a multi-scenario data set of the new energy power station; The first determination submodule is used to build a multi-scenario data model of a new energy power station based on a multi-scenario data set, and determine the clothing and behavior restriction factors in different scenarios through the multi-scenario data model combined with preset inspection specification requirements; The second determination submodule is used to determine the standard wearing features of employees in different scenarios according to the wearing and behavior restriction factors, and to determine the evaluation image features based on the visual clarity in different scenarios.

8. The new energy power station inspection violation warning system based on image analysis according to claim 6 is characterized in that: The generating module comprises: The third determination submodule is used to determine the wearing attributes of each type of protective product, and determine the product scale of each type of protective product based on the wearing attributes; A collection submodule, used to determine the best shooting angle and the best shooting distance based on the product scale of each type of protective products, and collect product structure data of each type of protective products according to the best shooting angle and shooting distance; A screening submodule is used to determine the wearing standard identification parameters according to the wearing instructions of various types of protective products, identify the image data of the user wearing various types of protective products according to the wearing standard identification parameters, and screen out the accurate image data of the user wearing various types of protective products according to the identification results; A generation submodule is used to associate and annotate the product structure data of various types of protective products with the precise image data of users wearing various types of protective products to generate an initial data set; The data cleaning submodule is used to clean the initial data set to obtain the model training set, where data cleaning includes: removing low-resolution, blurred and distorted low-quality data.

9. The new energy power station inspection violation warning system based on image analysis according to claim 6 is characterized in that: The evaluation module comprises: An extraction submodule is used to extract target features and target wearing features of various types of protective products based on a model training set using a preset deep learning model; A training submodule, used to train a preset deep learning model according to target features and target wearing features of various types of protective products to generate a deep learning-based target detection model; The second acquisition submodule is used to obtain the real-time wearing features of the employees by performing frame regression based on the real-time inspection images of the employees through the target detection model based on the candidate frame classification method; The comparison submodule is used to compare the real-time wearing characteristics with the target wearing characteristics of various types of protective products, and determine the employee's violation behavior based on the comparison results.

10. The new energy power station inspection violation warning system based on image analysis according to claim 6 is characterized in that: The analysis and alarm module comprises: The fourth determination submodule is used to detect the real-time weather parameters of the new energy power station and determine the real-time weather scene of the new energy power station according to the real-time weather parameters; A fifth determination submodule is used to determine the standard wearing characteristics of the target employee under the real-time weather scenario, and determine the violation identification description parameters according to the standard wearing characteristics of the target employee; The first analysis submodule is used to check the evaluation results according to the violation identification description parameters, perform authenticity analysis according to the check results, and determine the wearing details according to the evaluation image features; The second analysis submodule is used to determine the wearing subject and its standard description parameters according to the wearing details, and to perform a secondary authenticity analysis on the evaluation results according to the wearing subject and its standard description parameters; The alarm submodule is used to determine the actual violation of employees based on the two authenticity analysis data and issue a targeted alarm.

Images