Inspection object monitoring method and device, storage medium and electronic device

Abstract

The invention discloses an inspection object monitoring method and device, a storage medium and an electronic device, and the method comprises the steps: controlling a video monitoring and collection device to obtain the real-time video data of an area where fan equipment is located under the condition that the existence of an inspection object around the fan equipment is determined, the video monitoring and collecting device is pre-configured in the fan equipment and at least has the following functions of frame extraction, face detection, information coding and time and picture data aggregation; receiving real-time video data transmitted by the video monitoring and collecting device, and analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, the analysis result comprises a first sub-result of identity information corresponding to the inspection object, a second sub-result of an inspection behavior corresponding to the inspection object, and a third sub-result of an inspection track corresponding to the inspection object; monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality.

Description

Technical Field

[0001] The present application relates to the field of wind power equipment inspection, and more specifically, to a monitoring method and device for an inspection object, a storage medium, and an electronic device. Background Art

[0002] Recently, electric power companies have been continuously innovating in technology and are building intelligent, information-based, and digital technology-assisted inspections, and making the execution of inspections effective and traceable. In the past, inspection trajectories were generated through the location information generated by the GPS (Global Positioning System, referred to as GPS) device and the time when the information was collected. However, this type of trajectory is prone to the situation where it is unclear whether the personnel can perform the inspection action even though they have arrived at the location, and it is also uncertain whether the inspection personnel are the designated inspectors, so other auxiliary means are needed for manual inspection. The method of generating operation trajectories through identity recognition technology is widely used in the security field. For example, the target person's whereabouts trajectory is generated based on face and body recognition technology, but it cannot be directly applied to wind farms. Differences in business scenarios require the addition of wind power environment samples in combination with the wind farm environment and the environment inside and outside the cabin to conduct scenario-based training on the model, and obtain a recognition model suitable for wind farm scenarios.

[0003] Currently, no effective solution has been proposed to the problem that related technologies cannot confirm the performance and identity compliance of inspectors, resulting in low inspection quality and security.

[0004] Therefore, it is necessary to improve the related technology to overcome the above-mentioned defects in the related technology. Summary of the invention

[0005] The embodiments of the present application provide a monitoring method and device for an inspection object, a storage medium, and an electronic device to at least solve the problem that it is impossible to confirm the execution status and identity compliance of inspection personnel, resulting in low inspection quality and security.

[0006] According to one aspect of an embodiment of the present application, a monitoring method for an inspection object is provided, comprising: in a case where it is determined that an inspection object exists around a wind turbine device, controlling a video monitoring and acquisition device to acquire real-time video data of an area where the wind turbine device is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine device, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation; receiving real-time video data transmitted by the video monitoring and acquisition device, and analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, wherein the analysis result includes: a first sub-result of identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality.

[0007] In an exemplary embodiment, the inspection quality of the current inspection object is monitored according to the analysis results, and before the inspection object is reminded to update the inspection operation based on the inspection quality, the above method also includes: when the first sub-result in the analysis results indicates that the inspection object is an abnormal object, the inspection object is prohibited from entering other sub-equipment areas in the fan equipment, and the alarm device in the fan equipment is instructed to send an alarm message to the inspection object, wherein the alarm message is used to indicate that the current inspection object has left the area where the fan equipment is located; when the second sub-result in the analysis results indicates that the inspection object has abnormal behavior, the target inspection content corresponding to the abnormal behavior is determined, and the inspection object is reminded to re-inspect the target inspection content; when the third sub-result in the analysis results indicates that the inspection object has an abnormal trajectory, it is determined that the inspection object has not performed the inspection of the fan equipment according to the preset plan, and a situation description needs to be reported to the management platform associated with the inspection object.

[0008] In an exemplary embodiment, real-time video data transmitted by a video monitoring and acquisition device is received, and the real-time video data is analyzed according to a preset analysis strategy, including: determining the transmission type of the video data; when it is determined that the transmission type is uploaded after acquisition and processing by the video monitoring and acquisition device, inputting the video data into a plurality of sub-models included in the preset analysis strategy for processing; when it is determined that the transmission type is uploaded directly after acquisition by the video monitoring and acquisition device, performing information processing on the video data according to a preset video processing flow.

[0009] In an exemplary embodiment, the video data is respectively input into a plurality of sub-models included in a preset analysis strategy for processing, including: inputting the video data into an identity recognition sub-model to obtain a first sub-result of identity information corresponding to the video data; wherein the identity recognition sub-model is trained using a plurality of pre-recorded historical images and historical identity information corresponding to the historical images, each historical image corresponds to a target inspection object, the input of the identity recognition sub-model is the historical image, and the output is the historical identity information; inputting the video data into an inspection behavior discrimination sub-model to obtain a second sub-result of the inspection behavior corresponding to the video data; wherein the inspection behavior discrimination sub-model is trained using a plurality of pre-recorded historical behavior images and discrimination results corresponding to the historical behavior images, each historical behavior image corresponds to an inspection operation, the input of the inspection behavior discrimination sub-model is the historical behavior image, and the output is the discrimination result.

[0010] In an exemplary embodiment, after the video data are respectively input into multiple sub-models included in the preset analysis strategy for processing, the above method also includes: determining the inspection time for different inspection objects to perform inspection behaviors, wherein the inspection time at least includes the inspection start time and the inspection end time; sorting the first sub-result and the second sub-result corresponding to the video data based on the inspection time, and generating an inspection trajectory in which the inspection object performs different inspection behaviors at different locations; determining the third sub-result based on the inspection trajectory, and determining the difference between the inspection trajectory and the planned trajectory reported in advance by the inspection object.

[0011] In an exemplary embodiment, real-time video data is analyzed according to a preset analysis strategy, and before the analysis results are obtained, it includes: identifying behavioral characteristics of the inspection object in the real-time video data, wherein the behavioral characteristics include at least one of the following: the moving direction of the inspection object, the moving speed of the inspection object, the operation judgment of the inspection object, the gesture action of the inspection object, and the stay time of the inspection object; when the behavioral characteristics meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be a valid inspection; when the behavioral characteristics do not meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be an invalid inspection.

[0012] According to another aspect of the embodiment of the present application, a monitoring device for an inspection object is also provided, including: a control module, for controlling a video monitoring and acquisition device to acquire real-time video data of an area where the wind turbine equipment is located when it is determined that an inspection object exists around the wind turbine equipment, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation; an analysis module, for receiving real-time video data transmitted by the video monitoring and acquisition device, and analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, wherein the analysis result includes: a first sub-result of identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; a monitoring module, for monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality.

[0013] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the above-mentioned monitoring method of the inspection object when running.

[0014] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the monitoring method of the inspection object through the computer program.

[0015] According to another aspect of the embodiments of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above-mentioned method for monitoring the inspection object is implemented.

[0016] Through this application, when it is determined that there is an inspection object around the wind turbine equipment, the video monitoring and acquisition device is controlled to obtain real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device at least includes the following functions: frame extraction, face detection, information encoding, time and image data aggregation; receiving real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy, and obtaining an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality. The above technical solution solves the problem that the execution status and identity compliance of the inspection personnel cannot be confirmed, resulting in low inspection quality and safety. Furthermore, the video monitoring and acquisition device is used to obtain video data around the wind turbine equipment in real time, analyze the identity, behavior and trajectory of the inspection object, monitor the inspection quality and remind to update the inspection operation, thereby improving the inspection quality and inspection safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 It is a hardware structure block diagram of a computer terminal of a method for monitoring an inspection object according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for monitoring an inspection object according to an embodiment of the present application;

[0021] Figure 3 It is a structural schematic diagram of a wind farm inspection trajectory monitoring system based on identity recognition technology according to an embodiment of the present application;

[0022] Figure 4 It is a structural block diagram of monitoring of an inspection object according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] The method embodiments provided in the embodiments of the present application can be executed in a computer terminal, a mobile terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 1 is a hardware structure block diagram of a computer terminal of a method for monitoring an inspection object according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 The computer terminal includes a processor 102 (only one is shown in the figure) (the processor 102 may include but is not limited to a processing device such as a microprocessor (Central Processing Unit, CPU) or a programmable logic device (Field Programmable Gate Array, FPGA)) and a memory 104 for storing data, wherein the computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0026] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the monitoring method of the inspection object in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0027] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0028] In this embodiment, a monitoring method for an inspection object is provided. Figure 2 is a flow chart of a method for monitoring an inspection object according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps S202-S206:

[0029] Step S202: when it is determined that there is an inspection object around the wind turbine equipment, control the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation;

[0030] Step S204: receiving the real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object;

[0031] Step S206: monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality.

[0032] In the above steps, when it is determined that there is an inspection object around the wind turbine equipment, the video monitoring and acquisition device is controlled to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device at least includes the following functions: frame extraction, face detection, information coding, time and image data aggregation; receiving real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy, and obtaining an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality. The above technical solution solves the problem of being unable to confirm the execution status and identity compliance of the inspection personnel, resulting in low inspection quality and safety. Furthermore, the video monitoring and acquisition device is used to obtain video data around the wind turbine equipment in real time, analyze the identity, behavior and trajectory of the inspection object, monitor the inspection quality and remind to update the inspection operation, thereby improving the inspection quality and inspection safety.

[0033] In an exemplary embodiment, the inspection quality of the current inspection object is monitored according to the analysis results, and before the inspection object is reminded to update the inspection operation based on the inspection quality, the above method also includes: when the first sub-result in the analysis results indicates that the inspection object is an abnormal object, the inspection object is prohibited from entering other sub-equipment areas in the fan equipment, and the alarm device in the fan equipment is instructed to send an alarm message to the inspection object, wherein the alarm message is used to indicate that the current inspection object has left the area where the fan equipment is located; when the second sub-result in the analysis results indicates that the inspection object has abnormal behavior, the target inspection content corresponding to the abnormal behavior is determined, and the inspection object is reminded to re-inspect the target inspection content; when the third sub-result in the analysis results indicates that the inspection object has an abnormal trajectory, it is determined that the inspection object has not performed the inspection of the fan equipment according to the preset plan, and a situation description needs to be reported to the management platform associated with the inspection object.

[0034] It is understandable that if the analysis results show that the inspection object is an abnormal object, the system will immediately prohibit it from entering other sub-equipment areas and issue an alarm message to ensure the safety of equipment and personnel; if there is abnormal behavior, the system will remind the inspection object to re-inspect the target content to improve the inspection quality; if there is an abnormal trajectory, the system will require reporting the situation to the management platform to enhance security and traceability. Through these measures, the inspection process can be effectively managed and monitored to improve overall operational efficiency and safety.

[0035] In an exemplary embodiment, real-time video data transmitted by a video monitoring and acquisition device is received, and the real-time video data is analyzed according to a preset analysis strategy, including: determining the transmission type of the video data; when it is determined that the transmission type is uploaded after acquisition and processing by the video monitoring and acquisition device, inputting the video data into a plurality of sub-models included in the preset analysis strategy for processing; when it is determined that the transmission type is uploaded directly after acquisition by the video monitoring and acquisition device, performing information processing on the video data according to a preset video processing flow.

[0036] Exemplarily, a video monitoring acquisition device transmits real-time video data to a monitoring center. The monitoring center first determines the transmission type of the video data, and then takes the corresponding processing method according to the transmission type: if the transmission type is that the video monitoring acquisition device collects and processes the data and then uploads it, the monitoring center inputs the video data into multiple preset sub-models for processing. For example, one sub-model is used for pedestrian detection, one for vehicle detection, and one for abnormal behavior detection. Through the collaborative work of multiple sub-models, the video data can be analyzed more comprehensively and the monitoring effect can be improved. If the transmission type is that the video monitoring acquisition device directly uploads the data after collection, the monitoring center processes the video data according to the preset video processing flow. First, the video frame data is parsed, and then pre-processing, image processing, information proofreading, information merging, and finally the information is stored. Through this series of information processing steps, the monitoring center can effectively extract useful information from the video data and provide support for subsequent analysis and application.

[0037] In an exemplary embodiment, the video data is respectively input into a plurality of sub-models included in a preset analysis strategy for processing, including: inputting the video data into an identity recognition sub-model to obtain a first sub-result of identity information corresponding to the video data; wherein the identity recognition sub-model is trained using a plurality of pre-recorded historical images and historical identity information corresponding to the historical images, each historical image corresponds to a target inspection object, the input of the identity recognition sub-model is the historical image, and the output is the historical identity information; inputting the video data into an inspection behavior discrimination sub-model to obtain a second sub-result of the inspection behavior corresponding to the video data; wherein the inspection behavior discrimination sub-model is trained using a plurality of pre-recorded historical behavior images and discrimination results corresponding to the historical behavior images, each historical behavior image corresponds to an inspection operation, the input of the inspection behavior discrimination sub-model is the historical behavior image, and the output is the discrimination result.

[0038] For example, an inspection system needs to analyze surveillance videos to ensure that the identity and behavior of the inspectors comply with regulations. The system first inputs the video data into the identity recognition sub-model, which is trained using facial photos of qualified inspectors in the wind power inspection personnel database to obtain the identity information corresponding to the video data. At the same time, the system also inputs the video data into the inspection behavior discrimination sub-model, which is trained using a collection of video and picture samples of personnel inspecting equipment in multiple wind turbines in the wind power equipment inspection behavior sample library to obtain the inspection behavior corresponding to the video data. By combining the results of the identity recognition sub-model and the inspection behavior discrimination sub-model, the system can accurately identify the identity and behavior of the inspectors to ensure that they comply with regulations. At the same time, training with historical data can improve the accuracy and stability of the model, making the system more reliable and efficient in practical applications.

[0039] Optional, identity recognition is the process of confirming the identity of the operator. During the inspection process, professionals with response skills will be designated to conduct on-site inspections, checks or operations of wind farm equipment. There are three common methods of identity recognition: face recognition, fingerprint recognition, and pupil recognition.

[0040] Optionally, the face recognition model is a computer technology that uses advanced biometric technology in the field of artificial intelligence to identify people by analyzing and comparing visual feature information.

[0041] Optionally, the inspection behavior discrimination model is a model that determines whether to conduct an inspection based on the behavior of personnel in front of the equipment.

[0042] In an exemplary embodiment, after the video data are respectively input into multiple sub-models included in the preset analysis strategy for processing, the above method also includes: determining the inspection time for different inspection objects to perform inspection behaviors, wherein the inspection time at least includes the inspection start time and the inspection end time; sorting the first sub-result and the second sub-result corresponding to the video data based on the inspection time, and generating an inspection trajectory in which the inspection object performs different inspection behaviors at different locations; determining the third sub-result based on the inspection trajectory, and determining the difference between the inspection trajectory and the planned trajectory reported in advance by the inspection object.

[0043] It is understandable that different devices and different inspection objects are involved in the process of equipment inspection. First, determine the inspection time of each device. For example, the inspection time of device A is from 9 to 10 am, and the inspection time of device B is from 2 to 3 pm. Then sort the conditions of device A and device B according to the video data to generate an inspection trajectory. The inspection trajectory of device A includes the inspection start time of 9 o'clock and the inspection end time of 10 o'clock, and the inspection trajectory of device B includes the inspection start time of 2 o'clock and the inspection end time of 3 o'clock. Determine the third sub-result of each device based on the inspection trajectory, and compare it with the planned trajectory reported in advance by the inspection object, find differences and adjust the inspection plan in time. This can improve the efficiency and quality of equipment inspection.

[0044] Optionally, the inspection trajectory is the driving trajectory of the inspection personnel during the inspection task, which can reflect the actual working point and working status of the inspection personnel during the actual execution process.

[0045] In an exemplary embodiment, real-time video data is analyzed according to a preset analysis strategy, and before the analysis results are obtained, it includes: identifying behavioral characteristics of the inspection object in the real-time video data, wherein the behavioral characteristics include at least one of the following: the moving direction of the inspection object, the moving speed of the inspection object, the operation judgment of the inspection object, the gesture action of the inspection object, and the stay time of the inspection object; when the behavioral characteristics meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be a valid inspection; when the behavioral characteristics do not meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be an invalid inspection.

[0046] It can be understood that, according to the above embodiment, if the characteristics of the wind turbine equipment that need to be inspected are the moving speed and the length of stay of the inspection object. Through real-time video data analysis, it is identified that a staff member moves around the wind turbine equipment at a fast speed and stays for a short time. According to the preset inspection characteristics, the inspection can be judged as an invalid inspection. Another staff member moves around the wind turbine equipment at a moderate speed and stays for a long time, which meets the preset inspection characteristics, and the inspection can be judged as a valid inspection. In this way, effective and invalid inspections can be quickly and effectively identified in real-time monitoring, thereby improving inspection efficiency and accuracy.

[0047] Obviously, the embodiments described above are only some embodiments of the present application, not all embodiments. In order to better understand the above method, the above process is described below in conjunction with the embodiments, but it is not intended to limit the technical solutions of the embodiments of the present application, specifically:

[0048] The optional embodiment of the present application provides a method for monitoring wind farm inspection trajectories based on identity recognition technology to solve the problem of being unable to confirm the execution status and identity compliance of inspection personnel, resulting in low inspection quality and safety. The method is mainly divided into three layers. The acquisition layer mainly uses video monitoring and acquisition devices to process, integrate and transmit video information; the processing layer mainly performs data analysis, image processing and identity recognition, and analyzes the behavior of inspection personnel; the application layer mainly generates inspection trajectories by processing and collecting information, compares inspection plans, and provides missed inspection prompts.

[0049] Optional, Figure 3 It is a structural diagram of a wind farm inspection trajectory monitoring system based on identity recognition technology according to an embodiment of the present application; the above system at least includes: a collection layer 32, a processing layer 34, and an application layer 36.

[0050] Optionally, the collection layer 32 configures a video monitoring collection device inside the inspection object to process, integrate and transmit the video information. For example, the video stream inside the wind turbine is connected to the video monitoring device. When the video monitoring device detects that someone has entered the tower, the device collects the video frame by frame and transmits it to the processing layer together with the wind turbine code, camera code and time information for processing. The video data of multiple cameras in the wind turbine can be processed and transmitted through the video monitoring collection device.

[0051] Optionally, the video monitoring and acquisition device includes two implementation modes, which are as follows:

[0052] Method 1: The video monitoring acquisition device includes frame extraction, face detection, encoding and numbering, and time data and image data aggregation functions. Its function is to extract frames, encode information, and transmit the video stream that recognizes the face. The detection model is quantized, pruned, distilled, and other compression technologies are used to form a small model, which can achieve a response of hundreds of milliseconds.

[0053] Method 2: The video monitoring and acquisition device transmits the video data directly to the processing layer for processing and analysis.

[0054] Optionally, the processing layer 34 includes three modules: video information processing, video model analysis, and inspection situation calculation. The specific processing process is that after receiving the video acquisition data from each point, the video processing module of the processing layer will perform data analysis and image processing on the video information; the processed image information will enter the model analysis, and the identity recognition model will use the processed data to perform face recognition, thereby verifying the identity of the inspection personnel; the behavior analysis model will use the processed data to analyze whether the inspection personnel have conducted an inspection.

[0055] Optionally, video information processing includes video frame data analysis, preprocessing, image processing, information proofreading, information merging, and information storage.

[0056] Optional, video model analysis, specifically including two types of model analysis, as follows:

[0057] (1) Identity recognition model: face recognition, key point correction, and face M:N comparison (supporting multi-person recognition).

[0058] (2) Inspection behavior discrimination model: The discrimination model is unique to wind power. It can identify whether the inspector has performed an inspection based on information such as moving direction, moving speed, behavior judgment, gestures, and length of stay.

[0059] Optionally, the inspection situation calculation is to summarize and calculate the information after video processing and model analysis. The main capabilities include authenticity judgment of inspectors, real-time calculation of inspection trajectory summary, inspection behavior judgment calculation, inspection task completion status calculation, and alarm calculation management of deviation from inspection targets.

[0060] Optionally, the application layer 36 includes inspection trajectory generation, trajectory comparison, inspection item supervision, and inspection missed inspection prompts. Specifically, by processing and collecting information such as time, coding, identity, behavior, and task defects at the processing layer, the real inspection trajectory and inspection process image tracing are generated, and the inspection trajectory is compared with the inspection plan to see if it deviates, forming a comparison trajectory; the inspection system is used to automatically generate inspection supervision situation analysis and provide possible missed inspection prompts, that is, when the inspection trajectory deviates from the planned route, it can be notified through information technology.

[0061] Optionally, the inspection trajectory is generated through face recognition, personnel identity authentication, inspection start time and end time. At the same time, since the inspection points of the wind farm are far away by car, the system can use the distance calculation algorithm to determine the expected time to reach the next site. If it exceeds the configured threshold, the system automatically analyzes the risk points on the road and gives system prompts.

[0062] Optionally, the trajectory comparison is formed by comparing the planned trajectory with the actual inspection trajectory by comparing it with the inspection plan, marking the unplanned trajectory, and the unplanned trajectory needs to report the inspection situation and explanation.

[0063] Optionally, the inspection item supervised by the inspection item is to determine whether the inspection personnel have performed inspection behavior through human body recognition in each wind turbine.

[0064] Optionally, the missed inspection reminder is provided to the inspectors and the inspection platform through trajectory comparison. If the inspection quality is not high or there are missed inspections, a missed inspection reminder message will be provided to the inspectors and the inspection platform.

[0065] It should be noted that at the acquisition layer, attention should be paid to ensuring the stability and accuracy of the video monitoring device, ensuring the real-time transmission and integrity of the video data, and protecting the security of the video data to prevent data leakage or tampering. At the processing layer, attention should be paid to ensuring the accuracy and integrity of the video data, ensuring the accuracy and timeliness of data processing, and considering the selection and optimization of the algorithm model to ensure that identity recognition and behavior analysis can be effectively performed. At the application layer, attention should be paid to ensuring the accuracy and integrity of the generated inspection tracks and image tracing, ensuring the accuracy of the comparison tracks, and timely reminding of possible missed inspections to ensure the comprehensiveness and effectiveness of the inspection work. At the same time, the friendliness and ease of operation of the user interface should also be considered to ensure that users can easily view the inspection supervision situation analysis and prompt information.

[0066] In summary, the optional embodiment of the present application processes, integrates and transmits video information through the video monitoring and acquisition device of the acquisition layer to realize the monitoring and analysis of the identity and behavior of the inspection personnel; receives and processes the video acquisition data through the processing layer, performs identity recognition and behavior analysis, and generates inspection tracks and task defect information; generates inspection track comparisons through the application layer combined with the inspection plan, and provides prompts for possible missed inspections. The entire system realizes the supervision and analysis of the inspection process. In general, after the establishment of the above technical solution, the inspection quality and whether there are missed inspections are solved. The inspection execution is supervised in an automatic and intelligent way throughout the entire process and at all times, reducing human supervision of the inspection, and reducing the occurrence of problems caused by missed inspections and inadequate inspections. Inspection tracks and track playback can be generated. The playback information not only has time and location information, but also has images and video information of the inspection. The playback information is more comprehensive, and the inspection process is automatically closed. In addition, a variety of warnings are provided based on the actual scenes of the wind farm, such as missed inspection warnings, inspection failure warnings, etc., which improve the inspection quality and inspection safety. And the same and similar inspection scenes can be quickly migrated.

[0067] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application.

[0068] In this embodiment, a monitoring device for an inspection object is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0069] Figure 4 : is a structural block diagram of a monitoring device for an inspection object according to an embodiment of the present application, the device comprising:

[0070] The control module 42 is used to control the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located when it is determined that there is an inspection object around the wind turbine equipment, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation;

[0071] The analysis module 44 is used to receive the real-time video data transmitted by the video monitoring and acquisition device, analyze the real-time video data according to a preset analysis strategy, and obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object;

[0072] The monitoring module 46 is used to monitor the inspection quality of the current inspection object according to the analysis result, and remind the inspection object to update the inspection operation based on the inspection quality.

[0073] The above device, when determining that there is an inspection object around the wind turbine equipment, controls the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device at least includes the following functions: frame extraction, face detection, information coding, time and image data aggregation; receiving real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy, and obtaining an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; monitoring the inspection quality of the current inspection object according to the analysis result, and reminding the inspection object to update the inspection operation based on the inspection quality. The above technical solution solves the problem of being unable to confirm the execution status and identity compliance of the inspection personnel, resulting in low inspection quality and safety. Furthermore, the video monitoring and acquisition device is used to obtain video data around the wind turbine equipment in real time, analyze the identity, behavior and trajectory of the inspection object, monitor the inspection quality and remind to update the inspection operation, thereby improving the inspection quality and inspection safety.

[0074] In an exemplary embodiment, the above-mentioned device also includes: a determination module, which is used to monitor the inspection quality of the current inspection object according to the analysis results, and remind the inspection object to update the inspection operation based on the inspection quality; when the first sub-result in the analysis results indicates that the inspection object is an abnormal object, the inspection object is prohibited from entering other sub-equipment areas in the fan equipment, and the alarm device in the fan equipment is instructed to send an alarm message to the inspection object, wherein the alarm message is used to indicate that the current inspection object has left the area where the fan equipment is located; when the second sub-result in the analysis results indicates that the inspection object has abnormal behavior, the target inspection content corresponding to the abnormal behavior is determined, and the inspection object is reminded to re-inspect the target inspection content; when the third sub-result in the analysis results indicates that the inspection object has an abnormal trajectory, it is determined that the inspection object has not performed the inspection of the fan equipment according to the preset plan, and a situation description needs to be reported to the management platform associated with the inspection object.

[0075] In an exemplary embodiment, the above-mentioned analysis module is also used to determine the transmission type of the video data; when it is determined that the transmission type is uploaded after the video monitoring acquisition device collects and processes the data, the video data is respectively input into multiple sub-models included in the preset analysis strategy for processing; when it is determined that the transmission type is uploaded directly after the video monitoring acquisition device collects the data, the video data is processed according to the preset video processing flow.

[0076] In an exemplary embodiment, the above-mentioned analysis module is also used to input the video data into an identity recognition sub-model to obtain a first sub-result of the identity information corresponding to the video data; wherein the identity recognition sub-model is trained using multiple pre-recorded historical images and historical identity information corresponding to the historical images, each historical image corresponds to a target inspection object, the input of the identity recognition sub-model is the historical image, and the output is the historical identity information; the video data is input into the inspection behavior discrimination sub-model to obtain a second sub-result of the inspection behavior corresponding to the video data; wherein the inspection behavior discrimination sub-model is trained using multiple pre-recorded historical behavior images and the discrimination results corresponding to the historical behavior images, each historical behavior image corresponds to an inspection operation, the input of the inspection behavior discrimination sub-model is the historical behavior image, and the output is the discrimination result.

[0077] In an exemplary embodiment, the above-mentioned analysis module also includes: a determination unit, which is used to input the video data into multiple sub-models included in the preset analysis strategy for processing, and then determine the inspection time for different inspection objects to perform inspection behaviors, wherein the inspection time at least includes the inspection start time and the inspection end time; sort the first sub-result and the second sub-result corresponding to the video data based on the inspection time, and generate an inspection trajectory in which the inspection object performs different inspection behaviors at different locations; determine the third sub-result based on the inspection trajectory, and determine the difference between the inspection trajectory and the planned trajectory reported in advance by the inspection object.

[0078] In an exemplary embodiment, the above-mentioned device also includes: an identification module, which is used to analyze the real-time video data according to a preset analysis strategy, and identify the behavioral characteristics of the inspection object in the real-time video data before obtaining the analysis results, wherein the behavioral characteristics include at least one of the following: the moving direction of the inspection object, the moving speed of the inspection object, the operation judgment of the inspection object, the gesture action of the inspection object, and the stay duration of the inspection object; when the behavioral characteristics meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be a valid inspection; when the behavioral characteristics do not meet the preset inspection characteristics corresponding to the fan equipment, the inspection of the inspection object is determined to be an invalid inspection.

[0079] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0080] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0081] S1. When it is determined that there is an inspection object around the wind turbine equipment, control the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation;

[0082] S2, receiving the real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object;

[0083] S3. Monitor the inspection quality of the current inspection object according to the analysis result, and remind the inspection object to update the inspection operation based on the inspection quality.

[0084] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0085] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0086] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps in any one of the above method embodiments are performed.

[0087] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0088] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0089] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0090] S1. When it is determined that there is an inspection object around the wind turbine equipment, control the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation;

[0091] S2, receiving the real-time video data transmitted by the video monitoring and acquisition device, analyzing the real-time video data according to a preset analysis strategy to obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object;

[0092] S3. Monitor the inspection quality of the current inspection object according to the analysis result, and remind the inspection object to update the inspection operation based on the inspection quality.

[0093] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0094] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0095] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0096] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for monitoring an inspection object, characterized in that: include: When it is determined that there are inspection objects around the wind turbine equipment, the video monitoring and acquisition device is controlled to acquire real-time video data of the area where the wind turbine equipment is located, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation; Receive the real-time video data transmitted by the video monitoring and acquisition device, analyze the real-time video data according to a preset analysis strategy, and obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; The inspection quality of the current inspection object is monitored according to the analysis result, and the inspection object is reminded to update the inspection operation based on the inspection quality.

2. The method according to claim 1, characterized in that Before monitoring the inspection quality of the current inspection object according to the analysis result and reminding the inspection object to update the inspection operation based on the inspection quality, the method further includes: In the case where the first sub-result in the analysis result indicates that the inspection object is an abnormal object, prohibiting the inspection object from entering other sub-equipment areas in the wind turbine device, and instructing an alarm device in the wind turbine device to send an alarm message to the inspection object, wherein the alarm message is used to instruct the current inspection object to leave the area where the wind turbine device is located; When the second sub-result in the analysis result indicates that the inspection object has abnormal behavior, determining the target inspection content corresponding to the abnormal behavior, and reminding the inspection object to re-inspect the target inspection content; When the third sub-result in the analysis result indicates that the inspection object has an abnormal trajectory, it is determined that the inspection object has not performed the inspection of the wind turbine equipment according to the preset plan, and a situation report needs to be submitted to the management platform associated with the inspection object.

3. The method according to claim 1, characterized in that Receiving real-time video data transmitted by the video monitoring and acquisition device, and analyzing the real-time video data according to a preset analysis strategy, including: Determining a transmission type of the video data; When it is determined that the transmission type is uploaded after being collected and processed by the video monitoring acquisition device, the video data is respectively input into a plurality of sub-models included in the preset analysis strategy for processing; When it is determined that the transmission type is direct uploading after acquisition by the video monitoring acquisition device, information processing is performed on the video data according to a preset video processing flow.

4. The method according to claim 3, characterized in that The video data are respectively input into a plurality of sub-models included in the preset analysis strategy for processing, including: The video data is input into the identity recognition sub-model to obtain a first sub-result of the identity information corresponding to the video data; wherein the identity recognition sub-model is trained using a plurality of pre-recorded historical images and the historical identity information corresponding to the historical images, each historical image corresponds to a target inspection object, the input of the identity recognition sub-model is the historical image, and the output is the historical identity information; The video data is input into the inspection behavior discrimination sub-model to obtain a second sub-result of the inspection behavior corresponding to the video data; wherein the inspection behavior discrimination sub-model is trained using a plurality of pre-recorded historical behavior images and the discrimination results corresponding to the historical behavior images, each historical behavior image corresponds to an inspection operation, the input of the inspection behavior discrimination sub-model is the historical behavior image, and the output is the discrimination result.

5. The method according to claim 4, characterized in that After the video data are respectively input into a plurality of sub-models included in the preset analysis strategy for processing, the method further includes: Determine the inspection time for performing inspection actions on different inspection objects, wherein the inspection time at least includes the inspection start time and the inspection end time; Sorting the first sub-result and the second sub-result corresponding to the video data based on the inspection time to generate inspection tracks of the inspection object performing different inspection actions at different locations; A third sub-result is determined according to the inspection trajectory, and a difference between the inspection trajectory and a planned trajectory reported in advance by the inspection object is determined.

6. The method according to claim 1, characterized in that Before analyzing the real-time video data according to a preset analysis strategy and obtaining an analysis result, the method further includes: Identify the behavior characteristics of the inspection object in the real-time video data, wherein the behavior characteristics include at least one of the following: the moving direction of the inspection object, the moving speed of the inspection object, the operation judgment of the inspection object, the gesture action of the inspection object, and the length of stay of the inspection object; In the case where the behavior characteristic satisfies the preset inspection characteristic corresponding to the wind turbine device, determining that the inspection of the inspection object is a valid inspection; When the behavior characteristic does not satisfy the preset inspection characteristic corresponding to the wind turbine device, it is determined that the inspection of the inspection object is an invalid inspection.

7. A monitoring device for an inspection object, characterized in that: include: A control module is used to control the video monitoring and acquisition device to acquire real-time video data of the area where the wind turbine equipment is located when it is determined that there is an inspection object around the wind turbine equipment, wherein the video monitoring and acquisition device is pre-configured inside the wind turbine equipment, and the video monitoring and acquisition device includes at least the following functions: frame extraction, face detection, information encoding, and time and image data aggregation; An analysis module is used to receive the real-time video data transmitted by the video monitoring and acquisition device, analyze the real-time video data according to a preset analysis strategy, and obtain an analysis result, wherein the analysis result includes: a first sub-result of the identity information corresponding to the inspection object, a second sub-result of the inspection behavior corresponding to the inspection object, and a third sub-result of the inspection trajectory corresponding to the inspection object; The monitoring module is used to monitor the inspection quality of the current inspection object according to the analysis result, and remind the inspection object to update the inspection operation based on the inspection quality.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 6 when executed.

9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 6 through the computer program.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images