Wind power plant safety management system and method based on face recognition and positioning

By adopting multimodal biometrics, real-time positioning and environmental perception technologies in wind farms, combining autonomous operation robots and 5G remote control, the problems of limited vision and inability to remote control of traditional wind farm safety management systems have been solved, and employees and Comprehensive monitoring and remote control of equipment improve the safety and operational efficiency of wind farms.

CN119067448APending Publication Date: 2024-12-03PENGLAI WIND POWER BRANCH OF HUANENG SHANDONG POWER GENERATION CO LTD +1

Patent Information

Application Number
CN202411202864.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The traditional wind farm safety management system has a limited vision, lacks health status assessment, and is unable to remotely control, resulting in delayed safety hazards and emergency responses, and it is impossible to effectively prevent and deal with risk events.

Method used

Using multimodal biometric system based on face recognition and positioning, real-time positioning and status monitoring system, environmental perception and risk warning system, autonomous operation robot system and 5G remote control and virtual reality interaction system, through multimodal biometric, Real-time positioning, environmental perception and remote control, fully grasp the identity, location and health of employees, timely discover potential risks, and realize remote control and coordinated operations.

Benefits of technology

Achieve comprehensive monitoring of the identity, location and health status of wind farm employees, timely discover potential risks, avoid safety accidents to the greatest extent, improve the safety and controllability of wind farms, reduce the demand for personnel to enter high-risk areas, and improve the Work safety and quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067448A_ABST
    Figure CN119067448A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of wind power plant safety management, and discloses a wind power plant safety management system and method based on face recognition and positioning, which comprehensively master the identity, the position and the health condition of an operator through multi-mode biological recognition, real-time positioning and state monitoring, and ensure that the position of the operator is clear and the physical and mental state of the operator is controllable. Meanwhile, through the environment sensing and early warning functions, potential risks such as weather and equipment can be found in time, and therefore safety accidents are avoided to the maximum extent. Through the autonomous operation robot deployed in the wind turbine generator, remote control can be realized through a 5G network, and the requirement of personnel for entering a high-risk area is reduced. And meanwhile, through a virtual or augmented reality technology, an operator and an expert can guide operation personally on the scene, so that the safety and the quality of the operation are greatly improved. And unmanned remote control and collaborative operation are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of wind farm safety management, and in particular relates to a wind farm safety management system and method based on face recognition and positioning. Background Art

[0002] The safe operation of wind farms is of great significance to energy supply and personnel safety. Traditional wind farm safety management mainly relies on manual inspections and on-site duty, which has many drawbacks and safety hazards. In recent years, with the development of technologies such as computer vision, positioning navigation, and environmental perception, some intelligent safety management systems have begun to be applied in wind farms. Through video monitoring, personnel positioning, environmental monitoring and other means, management efficiency can be improved and safety hazards can be discovered in a timely manner. However, these systems still have some shortcomings, such as limited field of view, lack of health status assessment, and inability to remotely control, which cannot fully meet the needs of wind farm safety management. At present, the inspection and maintenance of wind turbines are still highly dependent on manual labor and there are many risks. Workers not only need to work in a narrow cabin, but also need to inspect and repair high-altitude parts such as blades, facing serious safety hazards such as falling and electric shock. In addition, once an emergency occurs, it is difficult for on-site personnel to obtain rescue in time. Traditional emergency management measures are often lagging and passive, and cannot effectively prevent and deal with risk events.

[0003] The Chinese patent publication number is CN111416831A, and the name is a patent application for a wind turbine platform operation supervision system for an offshore wind farm, which includes: a face recognition module, an intelligent access control module, a monitoring and alarm module, an operation supervision module, and an on-site mobile APP module. By combining the operation supervision module and the on-site mobile APP module, the work tickets and operation tickets of the staff are digitized and mobilized, and the entire operation process of the wind turbine platform is managed and controlled. The staff can apply for and review the work tickets and deploy and disarm the locks and alarm devices on the wind turbine platform through the on-site mobile APP module. Although this patent application can monitor the site, it cannot realize the simultaneous monitoring of personnel and wind turbine equipment, and cannot avoid the occurrence of safety accidents to the greatest extent. Summary of the invention

[0004] The purpose of the present invention is to provide a wind farm safety management system and method based on face recognition and positioning. Through multimodal biometrics, real-time positioning and status monitoring, the identity, location and health status of employees are fully grasped, ensuring that the location of the operators is clear and their physical and mental states are controllable. At the same time, through environmental perception and early warning functions, potential risks such as weather and equipment can be discovered in a timely manner, thereby avoiding the occurrence of safety accidents to the greatest extent.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] A wind farm safety management system based on face recognition and positioning, comprising a multimodal biometric recognition system, a real-time positioning and employee status monitoring system, and an environmental perception and risk warning system that are interconnected, as well as an autonomous operation robot system and a 5G remote control and virtual reality interaction system that are interconnected; the multimodal biometric recognition system is integrated with a biometric feature recognition algorithm for realizing feature extraction and pattern matching and establishing a multimodal biometric feature database of employees; the real-time positioning and employee status monitoring system is integrated with a satellite navigation algorithm and an indoor positioning algorithm for realizing real-time positioning of employee positions; the real-time positioning and employee status monitoring system includes a wearable device for collecting employee physiological data; the environmental perception and risk warning system is used to collect multi-dimensional environmental data, and the environmental perception and risk warning system is used to establish a wind farm environmental model; the autonomous operation robot system includes an operation robot arranged in a wind turbine, and the operation robot is equipped with a variety of sensor devices; the 5G remote control and virtual reality interaction system includes a 5G network and augmented reality or virtual reality equipment, which is used to provide video transmission and support virtual reality or augmented reality equipment to achieve remote immersive control.

[0007] Optionally, it also includes an integrated security management platform, which is used to integrate data from multimodal biometric systems, real-time positioning and employee status monitoring systems, environmental perception and risk warning systems, autonomous operation robot systems, 5G remote control and virtual reality interaction systems.

[0008] Optionally, the integrated security management platform includes a remote access control system and an emergency command and collaboration system.

[0009] Optionally, the multimodal biometric recognition system adopts a weighted mask fusion strategy to perform weighted fusion of different biometric features, and the feature weights are adaptively determined by a multi-task neural network according to the discrimination ability of each feature.

[0010] Optionally, the indoor positioning submodule of the real-time positioning and employee status monitoring system uses wireless radio frequency identification, ultra-wideband and visual odometer to achieve indoor positioning.

[0011] Optionally, the environmental perception and risk warning system has a big data modeling and prediction submodule, which learns wind farm environmental patterns from historical data through a deep belief network and a long short-term memory network model.

[0012] Optionally, the sensor equipment carried by the autonomous working robot includes a high-definition binocular camera, an infrared thermal imager, a laser radar and a millimeter-wave radar.

[0013] Optionally, the 5G remote control and virtual reality interaction system adopts the H.266 / VVC video coding standard for efficient video compression, and combines edge computing to achieve real-time encoding and decoding.

[0014] Optionally, the environmental perception and risk warning system includes a weather station, a noise sensor and a vibration sensor.

[0015] The method for using the wind farm safety management system based on face recognition and positioning comprises the following steps:

[0016] Through the multimodal biometric recognition system, different biometric features are weighted and fused, the extracted multimodal biometric feature vector is matched with the biometric feature library, and the best matching identity result is output to achieve biometric feature recognition;

[0017] The real-time positioning and employee status monitoring system detects the employee's location in real time, and the wearable device collects physiological data in real time. The physical and mental health status of employees is evaluated by combining the employee's location information and the employee's identity information recognized by the multimodal biometric system;

[0018] The environmental perception and risk warning system collects multi-dimensional environmental data to predict potential safety risks. When danger occurs, it combines the employee location information and employee identity information to issue warnings to employees in the danger area.

[0019] Through 5G remote control and virtual reality interactive system and autonomous operation robot system, real-time holographic immersive remote control of the activities of the autonomous operation robot inside the wind turbine is achieved.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The wind farm safety management system based on face recognition and positioning of the present invention can fully grasp the identity, location and health status of employees through multimodal biometrics, real-time positioning and status monitoring, ensuring that the location of the operators is clear and the physical and mental state is controllable. At the same time, through the environmental perception and early warning function, potential risks such as weather and equipment can be discovered in time, thereby avoiding the occurrence of safety accidents to the greatest extent, and improving the safety and controllability of the wind farm. The autonomous operation robot deployed inside the wind turbine of the present invention can be remotely controlled through the 5G network, reducing the need for personnel to enter high-risk areas. At the same time, through virtual or augmented reality technology, operators and experts can guide the operation in person, greatly improving the safety and quality of the operation. Unmanned remote control and collaborative operation are realized. The present invention enhances the intelligent, automated and remote operation and maintenance management capabilities of the wind farm, improves the controllability and efficiency of safe production, and can improve the operational safety of the wind farm, improve the management level, and reduce the operational risks.

[0022] Furthermore, the integrated safety management platform of the present invention integrates the data of each subsystem to realize centralized monitoring and intelligent management of wind farms. With the help of artificial intelligence analysis and decision-making functions, the system can automatically identify abnormal events, match the optimal emergency plan, and support remote control and coordinated command, thereby strengthening the active defense and rapid response capabilities of wind farms. It provides intelligent safety management and decision-making support.

[0023] Furthermore, the present invention will record various security incident data in real time, and based on blockchain and virtual environment reconstruction technology, ensure the integrity and traceability of incident audit data, support post-analysis and simulation improvement. At the same time, through remote collaboration tools such as multi-party video conferencing, the efficiency of emergency command and the transparency of decision-making are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings:

[0025] Figure 1 It is a schematic diagram of the overall architecture of the wind farm safety management system of the present invention;

[0026] Figure 2 is a schematic diagram of a multimodal biometric recognition system module according to Embodiment 1 of the present invention;

[0027] Figure 3 is a schematic diagram of a module of a real-time positioning and employee status monitoring system according to Embodiment 2 of the present invention;

[0028] Figure 4 This is a schematic diagram of the environment perception and risk warning system module of Example 3 of the present invention;

[0029] Figure 5 is a schematic diagram of the module of the autonomous operation robot system according to Embodiment 4 of the present invention;

[0030] Figure 6 This is a schematic diagram of the 5G remote control and VR interaction system module of Example 5 of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that when an element is referred to as being "disposed on" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only and are not intended to be the only embodiment.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0034] The present invention is described in detail below with reference to the accompanying drawings.

[0035] like Figure 1 As shown, a wind farm safety management system based on face recognition and positioning of the present invention includes a multimodal biometric recognition system, a real-time positioning and employee status monitoring system, an environmental perception and risk warning system, an autonomous operation robot system, a 5G remote control and virtual reality interaction system, and an integrated safety management platform.

[0036] The multimodal biometric recognition system integrates multiple biometric recognition technologies such as face, iris, palm print, and gait, and uses a deep learning algorithm to extract features and match patterns on captured image and video data to achieve employee recognition.

[0037] The multimodal biometric recognition system pre-establishes a multimodal biometric database of wind farm employees, and associates the multiple biometric information of each employee with their identity information. The biometric data is encrypted and stored using a homomorphic encryption algorithm to ensure personal privacy. The biometric information of employees is captured in real time through a high-definition binocular camera, an iris scanner, and a ground pressure sensor.

[0038] Example 1

[0039] See also Figure 2The multimodal biometric recognition system integrates multiple biometric recognition technologies to achieve high-precision identity recognition of wind farm employees, laying the foundation for the entire safety management system. Specifically, it includes:

[0040] The multimodal biometric recognition system uses a variety of sensor devices to collect biometric information of employees. The data collected by the sensor devices include high-definition binocular cameras to collect face and gait videos, iris scanners to collect iris images, and pressure-sensitive plates to collect palm vein and palm print images.

[0041] Biometric processing for a multimodal biometric system consists of the following steps:

[0042] Face recognition uses deep learning models such as residual convolutional neural network (ResNet) to extract and encode features of captured face images to achieve face detection, tracking and recognition.

[0043] Iris recognition uses a hybrid model that combines convolutional and recurrent neural networks to normalize and extract features from iris images for iris encoding and matching.

[0044] Palmprint recognition, based on algorithms such as Gabor wavelet transform and local order matching features, extracts palmprint features from palm images and constructs palmprint templates for matching.

[0045] Gait recognition uses a three-dimensional convolutional neural network to extract spatiotemporal features from gait videos and encodes gait features in combination with a sequence data model. Specifically, the sequence data model is an LSTM sequence data model.

[0046] The biometric fusion of the multimodal biometric system adopts the weighted mask fusion method to weightedly fuse different features. The feature weights are adaptively determined by the multi-task neural network according to the discrimination ability of each feature. The fused biometric feature vector has stronger robustness and accuracy.

[0047] The construction of the biometric database is specifically as follows: a multimodal biometric template is established for each wind farm employee, including biometric information of face, iris, palm print, and gait. The biometric template is associated with the employee's identity information to build a wind farm employee biometric database.

[0048] By collecting biometric data of on-site personnel in real time, the feature vectors are extracted using each biometric recognition model, the extracted multimodal biometric vectors are matched with the biometric library, and the best matching identity result is output to achieve biometric recognition.

[0049] The multimodal biometric system uses homomorphic encryption technology to encrypt, store and calculate biometric data without decryption, ensuring the privacy and security of employees' biometric information. By integrating multiple biometric features such as face, iris, palm print and gait, and combining advanced technologies such as deep learning and multi-sensor fusion, the system achieves high-precision employee identity recognition, providing a solid foundation for the entire security management system. At the same time, the system encrypts and protects the privacy of employees' biometric information.

[0050] Example 2

[0051] See also Figure 3 The real-time positioning and employee status monitoring system uses multi-system satellite positioning such as GPS, Beidou, Galileo and Glo NASS to achieve high-precision positioning of outdoor employees. At the same time, accurate positioning of indoor employees is achieved through RFID, ultra-wideband (UWB) and visual odometer. Each employee is equipped with a wearable device to collect physiological data such as heart rate, body temperature, blood pressure and respiratory rate. The AI ​​model of the long short-term memory network (LSTM) is used to analyze physiological data and evaluate the physical and mental conditions of employees in real time. Once an abnormality is found, an alarm is immediately issued and the nearest rescue personnel are notified.

[0052] In this embodiment, the real-time positioning and employee status monitoring system is responsible for achieving real-time positioning and physical and mental status monitoring of wind farm employees, as follows:

[0053] The real-time positioning and employee status monitoring system includes a satellite positioning submodule and an indoor positioning submodule.

[0054] The satellite positioning submodule adopts multi-system satellite positioning and uses satellite navigation signals to locate the outdoor position of employees with high precision.

[0055] The indoor positioning submodule uses RFID, ultra-wideband (UWB) and visual odometry to accurately locate indoor employees. RFID positioning determines the location by exchanging information between the radio frequency reader and the electronic tag worn by the employee. Ultra-wideband positioning is based on ranging and positioning algorithms, and uses propagation delay to estimate the target distance to determine the location. Visual odometry uses a camera to collect on-site images and combines visual odometry calculation methods to estimate employee displacement and posture. The positioning data of RFID, ultra-wideband (UWB) and visual odometry are fused through algorithms such as Kalman filtering to obtain a high-precision indoor location solution.

[0056] The real-time positioning and employee status monitoring system also includes a physiological data collection sub-module and an AI health assessment sub-module.

[0057] Employees are equipped with wearable devices to collect physiological data such as body temperature, heart rate, blood pressure, and respiratory rate in real time. These data are transmitted to the status monitoring module via low-power Bluetooth and other wireless methods. The AI ​​health assessment submodule uses deep learning models such as long short-term memory networks (LSTM) and combines attention mechanisms to intelligently analyze the collected multi-dimensional physiological data to assess the physical and mental health of employees, including fatigue, tension, anxiety, and extreme emotions.

[0058] The positioning data and health status data are organically integrated, and the real-time location and physical and mental condition of each employee are presented in an intuitive way through a visual interface, which is convenient for managers to monitor comprehensively. Through high-precision real-time positioning technology and artificial intelligence-assisted health status assessment, the system realizes all-round monitoring of wind farm employees, timely discovers and handles safety hazards such as abnormal position and health, and effectively ensures the personal safety of employees. At the same time, the system overcomes the defects of existing technologies such as inaccurate positioning and lack of health status assessment.

[0059] Example 3

[0060] See also Figure 4 The environmental perception and risk warning system deploys various environmental monitoring equipment such as weather stations, noise sensors, and vibration sensors to collect multi-dimensional environmental data such as wind speed, temperature and humidity, noise level, and equipment vibration. Based on big data analysis and machine learning algorithms, the system establishes an environmental model for the wind farm and predicts environmental conditions in the future. If extreme weather, equipment failure, and other risk situations that may lead to major safety accidents are predicted, the system will issue a warning and provide response suggestions and avoidance routes.

[0061] In this embodiment, the environmental perception and risk warning system monitors the environmental conditions of the wind farm in real time, predicts potential safety risks, and issues warnings, as follows:

[0062] The environmental perception and risk warning system includes an environmental data acquisition module, an environmental modeling and risk analysis module, and a warning decision and response module.

[0063] The environmental data acquisition module includes a meteorological monitoring submodule, a noise monitoring submodule, a noise sensor array and a vibration monitoring submodule.

[0064] The meteorological monitoring submodule is equipped with a meteorological monitoring station to collect meteorological data on wind speed, wind direction, temperature, humidity, air pressure, rain and snow. The noise level of the wind turbine during operation is measured by arranging a noise sensor array. The vibration monitoring submodule is equipped with vibration sensors to monitor the vibration of key components such as the blades and transmission system of the unit.

[0065] The environmental modeling and risk analysis module includes a big data analysis sub-module, a machine learning modeling sub-module, and a risk analysis sub-module.

[0066] The big data analysis submodule uses parallel computing and online analysis and processing big data technologies to fuse, clean and process the collected multi-source heterogeneous environmental data. The machine learning modeling submodule uses AI models based on deep belief networks (DBN) and long short-term memory networks (LSTM) to learn the environmental patterns of wind farms from historical data and build wind farm environmental models. The risk analysis submodule inputs real-time environmental data into the environmental model, uses the Monte Carlo simulation method to predict the development trend of the environment in the future, and evaluates the probability of extreme weather and equipment failure.

[0067] The early warning decision and response module includes an early warning decision submodule, an early warning release submodule and an emergency response submodule.

[0068] The early warning decision submodule sets thresholds for different risk levels. When the probability of a risk event exceeds the threshold, an early warning of the corresponding level is triggered. The early warning release submodule releases early warning information through multiple channels, and also provides response suggestions and risk avoidance routes. The emergency response submodule initiates the corresponding emergency response plan according to the early warning level, such as temporary evacuation of personnel and reinforcement of facilities.

[0069] Through comprehensive environmental data collection, combined with big data analysis and machine learning technology to build a wind farm environmental model, intelligent predictions are made on future environmental development trends and risk situations, early warnings are issued and response measures are given. Compared with traditional passive responses, active warnings are achieved, reducing losses caused by emergencies and improving the safe operation capabilities of wind farms.

[0070] Example 4

[0071] See also Figure 5 The autonomous operation robot system deploys multiple operation robots inside the wind turbine for daily inspection and maintenance of the wind turbine. These operation robots are equipped with high-definition visual systems, infrared thermal imagers, laser radars, and millimeter-wave radar sensors to achieve autonomous navigation and defect detection functions. Operators remotely control the robots and perform immersive virtual operations through real-time video streams, reducing the need for personnel to enter high-risk areas such as narrow and high altitude areas, thereby greatly improving safety.

[0072] In this embodiment, the autonomous operation robot system replaces manual labor with intelligent robots, reduces the need for personnel to enter high-risk areas, and improves the safety of wind farms. The autonomous operation robot system includes a robot hardware system, an autonomous navigation system, a machine vision and defect detection system, a remote control and command system, and a task planning and scheduling system.

[0073] The robot hardware system adopts a modular design, which is convenient for maintenance and upgrading. The body is made of lightweight and high-strength alloy material and has sufficient load-bearing capacity. The robot hardware system is equipped with a servo motor and a reducer to achieve multi-degree-of-freedom movement. It also has omnidirectional mobility and climbing functions. The robot hardware system is equipped with a high-definition binocular vision camera, an infrared thermal imager, a lidar and a millimeter-wave radar sensor for environmental perception and defect detection. The autonomous navigation system combines visual SLAM and laser SLAM to achieve autonomous positioning and map construction of the robot in known and unknown environments. The autonomous navigation system is based on the constructed three-dimensional environmental map, combined with the robot kinematic model, and uses A* or RRT genetic algorithm for global and local path planning.

[0074] The machine vision and defect detection system uses deep learning models such as convolutional neural networks to detect defects such as cracks, corrosion, and looseness in images of key components such as blades, bolts, and welds. Through structured light and visual measurement technology, the detected defects are reconstructed in three dimensions and the size, location, and other parameters of the defects are calculated.

[0075] The remote control and command system transmits the video stream from the robot to the remote operation end in real time through the 5G network, realizing low-latency video monitoring. The remote operator can control the robot's motion through virtual reality or augmented reality devices. The remote control and command system supports multiple experts to remotely access the system at the same time for real-time discussion and collaborative work.

[0076] The task planning and scheduling system can automatically allocate and schedule robots to perform corresponding inspection and maintenance tasks according to the wind farm's maintenance plan. Once a robot fails, the system will automatically plan a response plan.

[0077] Autonomous robots can perform autonomous navigation, precise detection and remote control inside wind turbines, greatly reducing the need for personnel to enter high-risk environments such as high altitude and narrow areas, and effectively improving the safety of wind farms. At the same time, through task planning and scheduling, efficient management of robots is achieved. The system overcomes many defects of traditional manual inspection and maintenance and has strong innovation and practical value.

[0078] Example 5

[0079] See also Figure 6The 5G remote control and virtual reality interaction system deploys a 5G millimeter wave wireless network, providing Gbps-level high bandwidth and millisecond-level low-latency transmission. Remote operators can use augmented reality (AR) or virtual reality (VR) head-mounted display devices, combined with force-reflecting glove devices, to achieve immersive remote control. The system supports real-time collaboration among multiple people, and remote experts can guide on-site operators in real time. In addition, the system uses edge computing technology to delegate some computing tasks to nearby edge nodes, further shortening network transmission delays and improving system response speed.

[0080] In this embodiment, the 5G remote control and virtual reality interaction system utilizes the high bandwidth and low latency characteristics of the 5G network to achieve remote control of wind turbines and improve the operating efficiency and safety of wind farms.

[0081] Deploy 5G millimeter wave wireless networks in wind farms to provide Gbps-level high bandwidth and millisecond-level low-latency wireless connections. Use large-scale MIMO and beamforming to ensure high network reliability and coverage. Deploy edge computing nodes to achieve computing and storage at the edge of the network. High-resolution panoramic video acquisition equipment is installed in wind turbines and control centers. The H.266 / VVC video coding standard is used to achieve efficient video compression. Combined with edge computing, real-time encoding and decoding of video streams can be performed to reduce network latency.

[0082] The 5G remote control and virtual reality interaction system includes a virtual reality interaction system and an augmented reality system.

[0083] The virtual reality interaction system includes virtual reality devices such as VR head-mounted display devices and force feedback gloves. It builds a high-fidelity wind turbine virtual environment based on the Unity3D game engine, renders a three-dimensional perspective in real time, and synchronizes force, touch and other multi-modal interactions with the movement of the operating equipment to provide an immersive experience.

[0084] The augmented reality system uses augmented reality devices such as optical perspective headsets or tablets to integrate real-time video of wind turbines with virtual content such as three-dimensional models and operation panels. Based on technologies such as SLAM and plane detection, it achieves precise alignment and perspective tracking of virtual and real environments.

[0085] Operators can conduct intuitive remote control of robots and wind turbines through a virtual environment, supporting real-time collaboration among multiple people. Remote experts can communicate with on-site operators through voice or video and provide real-time guidance. Administrators can remotely monitor and control on-site access control and alarms.

[0086] Edge nodes carry out computing-intensive tasks including virtual reality rendering and video encoding and decoding. They optimize network transmission through technologies such as workload segmentation and collaborative rendering, and automatically optimize network topology and resource scheduling based on AI algorithms such as deep reinforcement learning.

[0087] Operators can control and maintain wind turbines through immersive interaction, avoiding entering dangerous areas, thus greatly improving operation safety. At the same time, it supports real-time guidance from remote experts, improving operation quality and efficiency. The system overcomes the shortcomings of traditional remote control in terms of on-site and real-time operation, and has significant innovation and practical value.

[0088] The integrated security management platform can visualize comprehensive information such as real-time personnel location, physical and mental status, environmental data, etc. Based on artificial intelligence algorithms, the platform can automatically identify abnormal events and issue alarms. All events will be recorded and archived for post-audit and analysis.

[0089] In this embodiment, the integrated security management platform is responsible for integrating the data of the above-mentioned subsystems and realizing intelligent security monitoring, early warning and emergency response based on artificial intelligence algorithms.

[0090] The data fusion and storage of the integrated security management platform adopts standardized data access, and through unified data interface specifications, it realizes seamless access to multimodal biometrics, real-time positioning, environmental monitoring, and robotic subsystem data.

[0091] The big data storage and management of the integrated security management platform adopts the Hadoop distributed storage system, which has the storage and management capabilities of PB-level massive data and supports the integrated storage of structured data and unstructured data.

[0092] The integrated safety management platform has a visual display system, which uses 2D or 3D visual rendering and builds a highly interactive 2D or 3D visual interface based on WebGL or Unity graphics rendering engine. It can intuitively display holographic information of wind farm regional layout, real-time personnel location, and environmental conditions.

[0093] The integrated security management platform has an intelligent early warning and expert system, which combines AI early warning algorithms with expert decision-making systems to assist administrators in quickly discovering anomalies and making correct decisions. Through graphic annotation and other methods, it intuitively presents the type, location, and response suggestion information of abnormal events.

[0094] The integrated security management platform has an intelligent analysis and decision-making system, which can intelligently analyze real-time data based on deep learning algorithms. It can automatically identify various abnormal security events such as location abnormalities, physiological abnormalities, and environmental risks. Through the preset emergency plan template, it can intelligently match the optimal emergency response strategy according to the event type and impact range. It can present the decision basis and response process in an explainable form to assist administrators in making judgments.

[0095] The integrated security management platform has a remote control and collaboration system and uses intelligent access control to automatically control the entrance access and monitoring equipment of the wind farm area according to personnel permissions and event levels.

[0096] The integrated security management platform has an integrated voice recognition system to support administrators to remotely control via voice commands.

[0097] The integrated security management platform has an emergency command and collaboration system that provides remote collaboration tools such as multi-party video conferencing and whiteboard sharing to support the coordinated operations of headquarters, experts, and local personnel. Remote experts can obtain data and on-site videos in real time to provide support for on-site command.

[0098] The event audit and tracing of the integrated security management platform records the on-site video and data changes in real time through full-process video recording and event tracing, retaining complete event audit clues. Blockchain technology is used to ensure the immutability and traceability of event data. The entire process of the event is restored through data playback and 3D visualization technology.

[0099] The integrated safety management platform can realize intelligent, visual and remote management of wind farms, and strengthen risk warning and emergency response capabilities. The platform integrates cutting-edge technologies such as artificial intelligence, big data, virtual reality, and blockchain, demonstrating outstanding technological innovation. At the same time, it provides efficient auxiliary tools for management decision makers, greatly improving the operational safety and management level of wind farms.

[0100] The present invention uses multimodal biometric technology to accurately identify the identity of employees; integrates satellite positioning and indoor positioning to achieve real-time positioning of employees and assess their physical and mental state; deploys various environmental sensors to build a wind farm environmental model for risk prediction and early warning; arranges autonomous operating robots inside the wind turbine to support remote control and defect detection; and uses 5G networks and virtual reality technology to achieve holographic immersive remote control and expert collaboration.

[0101] The integrated security management platform integrates the data of each system, and realizes intelligent monitoring, early warning and emergency response of wind farms through technologies such as big data storage, visual rendering, and AI analysis and decision-making; supports remote access control and emergency command collaboration; and ensures the integrity and traceability of event audits based on technologies such as blockchain. This invention constructs an integrated intelligent security management solution, which significantly improves the operational safety and management level of wind farms, and has excellent innovation and practical value.

[0102] Unless otherwise specified, the device elements involved in the above embodiments are all conventional device elements, and the structural settings, working modes or control modes involved are all conventional settings, working modes or control modes in the art unless otherwise specified.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Other modifications or equivalent substitutions made to the technical solution of the present invention by ordinary technicians in the field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solution of the present invention.

Claims

1. A wind farm safety management system based on face recognition and positioning, characterized in that: It includes a multimodal biometric system, a real-time positioning and employee status monitoring system, and an environmental perception and risk warning system that are interconnected, as well as an autonomous operating robot system and a 5G remote control and virtual reality interaction system that are interconnected; the multimodal biometric system integrates a biometric feature recognition algorithm to achieve feature extraction and pattern matching and establish a multimodal biometric feature database for employees; the real-time positioning and employee status monitoring system integrates a satellite navigation algorithm and an indoor positioning algorithm to achieve real-time positioning of employee positions; the real-time positioning and employee status monitoring system includes wearable devices for collecting employee physiological data; the environmental perception and risk warning system is used to collect multi-dimensional environmental data, and the environmental perception and risk warning system is used to establish a wind farm environmental model; the autonomous operating robot system includes an operating robot arranged in a wind turbine, and the operating robot is equipped with a variety of sensor devices; the 5G remote control and virtual reality interaction system includes a 5G network and augmented reality or virtual reality equipment, which are used to provide video transmission and support virtual reality or augmented reality equipment to achieve remote immersive control.

2. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: It also includes an integrated security management platform, which is used to integrate data from multimodal biometric systems, real-time positioning and employee status monitoring systems, environmental perception and risk warning systems, autonomous operation robot systems, 5G remote control and virtual reality interaction systems.

3. A wind farm safety management system based on face recognition and positioning according to claim 2, characterized in that: The integrated security management platform includes a remote access control system and an emergency command and coordination system.

4. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The multimodal biometric recognition system adopts a weighted mask fusion strategy to perform weighted fusion of different biometric features, and the feature weights are adaptively determined by a multi-task neural network according to the discrimination ability of each feature.

5. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The indoor positioning submodule of the real-time positioning and employee status monitoring system uses wireless radio frequency identification, ultra-wideband and visual odometer to achieve indoor positioning.

6. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The environmental perception and risk warning system has a big data modeling and prediction submodule, which learns wind farm environmental patterns from historical data through a deep belief network and a long short-term memory network model.

7. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The sensor equipment carried by the autonomous working robot includes a high-definition binocular camera, an infrared thermal imager, a laser radar and a millimeter-wave radar.

8. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The 5G remote control and virtual reality interaction system adopts the H.266 / VVC video coding standard for efficient video compression, and combines edge computing to achieve real-time encoding and decoding.

9. A wind farm safety management system based on face recognition and positioning according to claim 1, characterized in that: The environmental perception and risk warning system includes a weather station, a noise sensor and a vibration sensor.

10. A method for using a wind farm safety management system based on face recognition and positioning according to any one of claims 1 to 9, characterized in that: The following steps are involved: Through the multimodal biometric recognition system, different biometric features are weighted and fused, the extracted multimodal biometric feature vector is matched with the biometric feature library, and the best matching identity result is output to achieve biometric feature recognition; The real-time positioning and employee status monitoring system detects the employee's location in real time, and the wearable device collects physiological data in real time. The physical and mental health status of employees is evaluated by combining the employee's location information and the employee's identity information recognized by the multimodal biometric system; The environmental perception and risk warning system collects multi-dimensional environmental data to predict potential safety risks. When danger occurs, it combines the employee location information and employee identity information to issue warnings to employees in the danger area. Through 5G remote control and virtual reality interactive system and autonomous operation robot system, real-time holographic immersive remote control of the activities of the autonomous operation robot inside the wind turbine is achieved.

Citation Information

Patent Citations

  • Fan platform operation supervision system for offshore wind plant

    CN111416831A

Cited By

  • Full-scene high-place operation intelligent monitoring system

    CN119723801A

  • Full-scene high-altitude operation intelligent monitoring system

    CN119723801B