A smart wearable terminal device and a large hydropower station emergency management method

Abstract

The present application relates to the fields of safety wearable equipment technology and emergency management, and discloses a smart wearable terminal device and a method for emergency management of a large hydropower station. The device includes a head-mounted device, a bracelet device and a waist-span device; the head-mounted device includes a multi-module camera, noise-cancelling headphones, a microphone and augmented reality glasses arranged on the helmet body; the bracelet device is used to monitor the physical signs data and exercise data of the operator; the waist-span device includes a main control module, a 5G communication module and a UWB personnel positioning module; the main control module is connected to the multi-module camera, noise-cancelling headphones, a microphone, augmented reality glasses and a bracelet device, and the main control module interacts with the data processing platform through the 5G communication module. The present application can realize the comprehensive monitoring of the physical signs, environmental identification and operation intensity of the operators, provide comprehensive and effective data support for the emergency management of large hydropower stations, and meet the precise needs of the emergency management of large hydropower stations.

Description

Technical Field

[0001] The present application relates to the fields of safety wearable equipment technology and emergency management, and specifically to a smart wearable terminal device and an emergency management method for a large hydropower station. Background Art

[0002] The operating environment of large hydropower stations is extremely complex, with many safety risks and management challenges. There are a large number of underground corridors and passages in hydropower stations. The signals of traditional walkie-talkies, mobile phones and other equipment cannot be effectively covered and transmitted, and there are a large number of areas without signals. When operators enter the relevant areas for work, communication is not smooth. Traditional information transmission and communication methods make it difficult to efficiently perceive the factors affecting the emergency site. After emergency response personnel enter the site, it is difficult to efficiently transmit the situation of each operating surface back to the command center, and it is difficult to obtain emergency plans and command guidance. It is difficult for experts at the command center to perceive the situation of each site and personnel and comprehensively dispatch emergency resources.

[0003] However, in the daily operation and emergency management of hydropower stations, it is very important to monitor the operators in real time and implement effective management. Traditional management methods are simply unable to accurately and real-time grasp the status of operators. For example, the current acquisition of information such as the physical signs and work intensity of operators is often untimely and inaccurate, so there is no way to provide comprehensive and effective data support for emergency management.

[0004] At present, smart wearable devices have been applied to a certain extent in some other fields. For example, in the field of fire protection, this smart wearable device integration system is used to collect environmental information, vital signs information, and operation process data of on-site personnel, thereby realizing real-time control of operation and maintenance safety risks. Power companies are also using smart wearable technology to achieve functions such as remote technical support, operation guidance, and safety supervision. These applications fully demonstrate the potential of smart wearable devices in the operation management of the energy industry. However, in view of the special operating environment and needs of large hydropower stations, the existing smart wearable devices cannot meet the precise needs of emergency management of large hydropower stations in terms of comprehensive monitoring of physical signs, environmental identification, and operation intensity of operators. Summary of the invention

[0005] The present application provides a smart wearable terminal device to solve the problem in the prior art that the existing smart wearable devices cannot meet the precise needs of emergency management of large hydropower stations in terms of comprehensive monitoring of workers' physical signs, environmental identification and work intensity.

[0006] Correspondingly, the present application also provides a large hydropower station emergency management method to ensure the application of the above-mentioned equipment.

[0007] In order to solve the above technical problems, the present application discloses a smart wearable terminal device, which includes a head-mounted device, a wristband device and a waist-mounted device;

[0008] The head-mounted device includes a helmet body, and a multi-module camera, noise reduction headphones, a microphone and augmented reality glasses arranged on the helmet body;

[0009] The wristband device is used to monitor the physical signs and exercise data of the operators;

[0010] The waist-span device includes a main control module, a 5G communication module, and a UWB personnel positioning module;

[0011] The main control module is connected to the multi-module camera, noise reduction headphones, microphone and augmented reality glasses by wires, and is wirelessly connected to the bracelet device;

[0012] The 5G communication module is wirelessly connected to the main control module to realize the interaction between the main control module and the data processing platform.

[0013] Preferably, the helmet body includes a head ring, and the multi-module camera, noise reduction headphones, microphone and augmented reality glasses are all connected to the head ring.

[0014] Preferably, the headband is also provided with an interface conversion module, and the multi-module camera, noise reduction headphones, microphone, augmented reality glasses and knob buttons are all connected to the main control module through the interface conversion module;

[0015] The interface conversion module is used to convert the image data collected by the multi-module camera, the voice data of the microphone and the key function data of the knob button, and transmit them to the main control module through TYPE-C;

[0016] The interface conversion module is also used to convert the target display information in the main control module into MIPI interface data and transmit it to the augmented reality glasses;

[0017] The interface conversion module is also used to convert the voice data received by the main control module into IIS interface data and transmit it to the noise reduction headset.

[0018] Preferably, the augmented reality glasses are used to display target display information, which includes video call information, other people's perspective information, display device description information, emergency plan information, personnel identification information, license information, and tool inspection information;

[0019] The headband is provided with a knob button, which is used to control the flipping of the content displayed by the augmented reality glasses;

[0020] The functions of the knob button include jumping to the previous page, jumping to the next page, and determining the content of the current page.

[0021] Preferably, the noise reduction headset uses an adaptive dynamic filtering algorithm to filter out noise in the voice data, and uses a convolutional neural network to recognize the voiceprint features in the voice data.

[0022] Preferably, the waist-hung device also includes a container, and the main control module, 5G communication module and UWB personnel positioning module are all arranged in the container.

[0023] Preferably, the waist-hung device also includes a backup power supply, which is used to supply power to the main control module and to supply power to the UWB personnel positioning module through the main control module.

[0024] The present application also discloses a large hydropower station emergency management method, which is implemented based on any of the above-mentioned smart wearable terminal devices, and the method includes:

[0025] Acquire images of the current working environment through multi-module cameras;

[0026] In the main control module, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, and a three-dimensional map of the image is constructed based on the real-time positioning and mapping algorithm. The category information and location information of the object are mapped into the three-dimensional map to identify the dangerous area.

[0027] The object's category information, location information, and three-dimensional map are sent to the augmented reality glasses for display through the main control module;

[0028] Obtain the physical sign data and exercise data of the monitoring workers through the wristband device;

[0029] In the main control module, the physical sign data and exercise data are processed based on the preset behavior process intelligent recognition and evaluation algorithm to obtain the behavior category and work intensity level;

[0030] Obtain the positioning status information of the operator through the UWB personnel positioning module, multi-module camera and wristband device;

[0031] In the main control module, the movement status information of the operator is determined through the positioning status information;

[0032] The operator's mobility status information, dangerous area behavior category and work intensity are sent to the data processing platform through the 5G communication module.

[0033] Preferably, the multi-module camera includes a telephoto camera, a thermal imaging camera and a wide-angle camera, which are used to respectively obtain a telephoto image, a thermal imaging image and a wide-angle image of the current working environment;

[0034] In the main control module, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, and a three-dimensional map of the image is built based on the real-time positioning and mapping algorithm. The category information and location information of the object are mapped to the three-dimensional map to identify dangerous areas, including:

[0035] Preprocessing the telephoto image, thermal image and wide-angle image; the preprocessing includes image denoising, color normalization and image alignment;

[0036] Use the YOLO target detection algorithm to classify the objects in the preprocessed telephoto image, thermal image, and wide-angle image, and select the location of the object to obtain the category information and location information;

[0037] The real-time positioning and mapping algorithm is used to extract the key feature points in the pre-processed telephoto image, thermal image and wide-angle image, and the relative displacement between the telephoto camera, thermal camera and wide-angle camera is calculated through feature matching;

[0038] Based on relative displacement, a 3D map is constructed through graph optimization, and the position and posture of the telephoto camera, thermal camera, and wide-angle camera in space are estimated;

[0039] Map the category information and location information of the object into a three-dimensional map to obtain an environment layout map;

[0040] Identify hazardous areas based on the environment layout map.

[0041] Preferably, in the main control module, the physical sign data includes heart rate and blood oxygen saturation; the exercise data includes cadence and number of steps;

[0042] Based on the preset behavior process intelligent recognition and evaluation algorithm, the physical sign data and exercise data are processed to obtain the behavior category and work intensity level, including:

[0043] Normalize the heart rate, blood oxygen saturation, cadence and number of steps;

[0044] The standardized heart rate, blood oxygen saturation, cadence and number of steps are divided into certain time windows, and statistical features are calculated for each time window; the statistical features include mean, maximum, minimum and standard deviation; the physiological features of each time window include heart rate features, blood oxygen saturation features, cadence features and number of steps features;

[0045] Assign a label to each time window; the label includes the work behavior category and the work intensity level; the work behavior category includes walking, standing and resting; the work intensity level includes mild, moderate and severe;

[0046] The physiological characteristics of each time window are used as the input of the preset intelligent recognition model. The dependencies between different time steps are learned through the long short-term memory network to capture the dynamic changes of work behavior. The physiological characteristics are judged based on the labels to obtain the behavior category and work intensity level.

[0047] In this application, the images of the working site environment and the working process are collected by a multi-module camera, voice communication is realized by a microphone and a noise reduction headset, the required images and text information are displayed by an augmented reality glasses, the physical signs data and exercise data of the operators are monitored by a wristband device, and the operators are located by a UWB personnel positioning module. The main control module is connected with the above-mentioned multi-module camera, noise reduction headset, microphone, augmented reality glasses, wristband device and UWB personnel positioning module, and the data sent by the above-mentioned devices can be analyzed and processed, and the operation intensity and workload can be evaluated by combining the physical signs data, exercise data and positioning, and more accurate positioning status information can be determined. The main control module is connected with the 5G communication module, and the information interaction with the data processing platform can be realized through the 5G communication module, so that experts outside the working environment can provide guidance; it can also realize interaction with other operators, realize multi-person collaborative operation and mutual rescue in the first time. Therefore, in the embodiment of the present application, comprehensive monitoring of the physical signs, environmental identification and operation intensity of the operators can be realized, providing comprehensive and effective data support for the emergency management of large hydropower stations, and meeting the precise needs of emergency management of large hydropower stations.

[0048] Additional aspects and advantages of the present application will be given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0050] Figure 1 A schematic diagram of a smart wearable terminal device provided in an embodiment of the present application;

[0051] Figure 2 This is a communication connection diagram between modules of the smart wearable terminal device provided in an embodiment of the present application.

[0052] Among them, 1-multi-module camera; 2-augmented reality glasses; 3-noise-cancelling headphones; 4-microphone; 5-knob button; 6-interface conversion module; 7-main control module; 8-UWB personnel positioning module; 9-5G communication module; 10-bracelet device; 11-lighting module. DETAILED DESCRIPTION

[0053] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0054] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0055] Those skilled in the art will appreciate that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0056] With regard to the technical problems existing in the prior art, the present application provides a smart wearable terminal device and a large hydropower station emergency management method, aiming to solve at least one of the technical problems of the prior art.

[0057] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0058] The present application embodiment provides a smart wearable terminal device, such as Figure 1 and Figure 2As shown in the figure, the device includes a head-mounted device, a bracelet device 10 and a waist-band device; the head-mounted device includes a helmet body, and a multi-module camera 1, noise-cancelling headphones 3, a microphone 4 and augmented reality (AR) glasses 2 arranged on the helmet body; the bracelet device 10 is used to monitor the physical signs data and exercise data of the operating personnel; the waist-band device includes a main control module 7, a 5G communication module 9 and a UWB personnel positioning module 8; the main control module 7 is wiredly connected to the multi-module camera 1, the noise-cancelling headphones 3, the microphone 4 and the augmented reality glasses 2, and is wirelessly connected to the bracelet device 10; the 5G communication module 9 is wirelessly connected to the main control module 7, and is used to realize the interaction between the main control module 7 and the data processing platform.

[0059] In the embodiment of the present application, the outer shell of the helmet body is made of high-strength lightweight material, which combines firmness and lightness. At the same time, the internal padding is designed to be comfortable and breathable to ensure the comfort of the wearer.

[0060] The multi-module camera 1 can collect images of the work site environment and the work process, identify and analyze the images through the main control module 7 to obtain the dangerous areas of the current working environment, and send the identified dangerous area information to the data processing platform through the 5G communication module 9 for alarm.

[0061] The noise reduction earphones 3 and the microphone 4 can realize the communication between the operator and the remote instructor, thus realizing the remote instruction.

[0062] The augmented reality glasses 2 are used to display the target display information sent by the main control module 7, such as video call information, other people's perspective information, etc., to facilitate operators to obtain relevant information so as to make quick and accurate decisions in emergency situations.

[0063] The wristband device 10 adopts a smart wristband. The shell of the smart wristband is designed with a polycarbonate polymer plastic with a waterproof coating, and the back cover is designed with a silicone flexible sealing ring to achieve IP68 waterproof, which can work normally in a humid environment. The strap design of the smart wristband is comfortable and breathable to ensure the comfort of the wearer. The wristband device 10 collects the physical sign data and exercise data of the operator and sends it to the main control module 7. The main control module 7 determines the operator's behavior, work intensity and workload by analyzing the physical sign data and exercise data. The operator's behavior, work intensity and workload are sent to the data processing platform through the 5G communication module 9 to facilitate better coordination of the operator's work tasks.

[0064] In addition, the positioning status information of the operator can be obtained through the UWB personnel positioning module 8, the multi-module camera 1 and the bracelet device 10. Among them, the motion detection function of the smart bracelet (such as collecting the number of steps, cadence and other physical signs) can assist in judging the movement status and position changes of the operator, the multi-module camera 1 can transmit the image data of the environment where the operator is located in real time, and the UWB personnel positioning module 8 performs positioning. The trinity provides more comprehensive information for the rescue operation.

[0065] The smart wearable terminal device in the embodiment of the present application can automatically monitor and transmit data in daily operations, respond quickly in emergencies, and be regularly maintained and upgraded as needed to adapt to the complex environment of the hydropower station and improve the level of emergency management and the safety of operators.

[0066] Based on the above, the embodiments of the present application can realize multi-dimensional perception and intelligent identification of production factors, and information integration and sharing between people on site and between the site and the command center. Digital means and 5G communication technology are used to improve information perception and management capabilities, break down information transmission barriers, and realize full-domain sharing of what is heard and seen, integrated command and dispatch, real-time learning guidance, and operation information anywhere.

[0067] In the embodiment of the present application, the images of the working site environment and the working process are collected through the multi-module camera 1, the voice communication is realized through the microphone 4 and the noise reduction headset 3, the required images and text information are displayed through the augmented reality glasses 2, the physical sign data and exercise data of the operating personnel are monitored through the wristband device 10, and the operating personnel are located through the UWB personnel positioning module 8. The main control module 7 is connected with the above-mentioned multi-module camera 1, the noise reduction headset 3, the microphone 4, the augmented reality glasses 2, the wristband device 10 and the UWB personnel positioning module 8, and the data sent by the above-mentioned devices can be analyzed and processed, and the working intensity and workload can be evaluated by combining the physical sign data, exercise data and positioning, and more accurate positioning status information can be determined. The main control module 7 is connected with the 5G communication module 9, and the information interaction with the data processing platform can be realized through the 5G communication module 9, so as to facilitate the guidance of experts outside the working environment; it can also realize interaction with other operating personnel, realize multi-person collaborative operation and first-time mutual rescue. Therefore, in the embodiments of the present application, comprehensive monitoring of the physical signs, environmental identification and work intensity of the operators can be achieved, providing comprehensive and effective data support for the emergency management of large hydropower stations, and meeting the precise needs of emergency management of large hydropower stations.

[0068] In an optional embodiment, the helmet body includes a head ring, and the multi-module camera 1, noise reduction headphones 3, microphone 4 and augmented reality glasses 2 are all connected to the head ring.

[0069] In an optional embodiment, the multi-module camera 1 adopts a lightweight modular design mode by integrating multiple functional modules (such as telephoto, thermal imaging, and wide-angle) into the same sensor unit or sensor array. The multi-module camera 1 includes a telephoto camera for viewing personnel and equipment conditions at a long distance to obtain a telephoto image; a thermal imaging camera for viewing the temperature of joints, bearings and other components to obtain a thermal imaging image; and a wide-angle camera for viewing the on-site conditions over a large range to obtain a wide-angle image. In the embodiment of the present application, the multi-module camera 1 that integrates multiple functional modules can meet multiple viewing needs at the same time, comprehensively monitor the working environment, and avoid missing key information.

[0070] In the embodiment of the present application, in the main control module 7, advanced video intelligent analysis technologies, such as the YOLO target detection algorithm and the simultaneous localization and mapping (SLAM) algorithm, are used to accurately identify key environmental elements such as dangerous areas from the images obtained by the multi-module camera 1.

[0071] The implementation process of video intelligent analysis technology is as follows:

[0072] Step 1. Multi-camera input: The smart helmet is equipped with multiple cameras (including telephoto, thermal imaging, and wide-angle) to collect video streams in the environment in real time.

[0073] Step 2. Data preprocessing: Preprocess the data input from each camera (telephoto image, thermal image, and wide-angle image), including image denoising, color normalization, and image alignment, so as to be input into the YOLO model and SLAM algorithm built based on YOLO target detection.

[0074] Step 3. Target detection: The pre-processed telephoto image, thermal image, and wide-angle image are input into the YOLO model for target detection. The YOLO model uses a convolutional neural network (CNN) to classify objects in the image and frame the location of the object (bounding boxes), thereby obtaining the category information and location information of the object.

[0075] Step 4. Visual SLAM for environmental mapping and positioning: Use the SLAM algorithm to extract key feature points from the pre-processed telephoto image, thermal image, and wide-angle image, and calculate the relative displacement between the telephoto camera, thermal camera, and wide-angle camera through feature matching (such as ORB features). Based on the relative displacement calculation, the SLAM algorithm constructs a dense or sparse three-dimensional map through graph optimization, and estimates the position and posture of the telephoto camera, thermal camera, and wide-angle camera in space;

[0076] Step 5. Environmental understanding and risk assessment: Map the location information and category information of the target object detected by the YOLO model to the 3D map constructed by the SLAM algorithm. Combined with the environmental layout map calculated by the SLAM algorithm, potential dangerous areas can be identified and an alarm can be issued.

[0077] Step 6. Real-time rendering and augmented reality: Through augmented reality technology, the target object category and location information recognized by the YOLO model are superimposed on the augmented reality glasses 2. The three-dimensional map constructed by the SLAM algorithm can be presented in the augmented reality glasses 2 through AR technology, providing real-time spatial navigation and path planning for operators, and providing intuitive and convenient operation guidance for operators in the complex hydropower station environment.

[0078] In an optional embodiment, the augmented reality glasses 2 are used to display target display information, which includes video call information, other people's perspective information, display device description information, emergency plan information, personnel identification information, license information, and tool inspection information;

[0079] The headband is provided with a knob button 5, which is used to control the flipping of the content displayed by the augmented reality glasses 2;

[0080] The functions of knob button 5 include jumping to the previous page, jumping to the next page and determining the content of the current page.

[0081] In the embodiment of the present application, the augmented reality glasses 2 display the above-mentioned target display information. Combined with the 5G communication module 9, the operating personnel can obtain relevant information, thereby improving the efficiency of the operating personnel in obtaining information and facilitating quick and accurate decision-making in emergency situations.

[0082] The target display information displayed by the augmented reality glasses 2 is switched through the knob button 5, wherein the functions of going to the previous page, jumping to the next page, and determining the content of the current page are concentrated on one knob button 5 for easy operation.

[0083] In an optional embodiment, the augmented reality glasses 2 include a connecting arm, a first rotating plug is provided on the head ring, and the connecting arm is connected to the first rotating plug. Among them, the first rotating plug can support the augmented reality glasses 2 and the head ring to rotate up and down at the junction, and can rotate up to 45 degrees in the clockwise direction, parallel to the line of sight. When AR display is not needed, the entire augmented reality glasses 2 can be lifted above the helmet to avoid blocking the line of sight. The AR lens supports forward and backward pushing, which can ensure that different operators can adjust to the most appropriate viewing distance. In addition, the augmented reality glasses 2 use LCOS chips, which minimize the impact of AR lenses on the line of sight through a monocular method.

[0084] In an optional embodiment, the headband is further provided with a second rotating plug, the second rotating plug is provided with a jack, and the noise reduction earphone 3 is led out of the jack of the second rotating plug by a cable to avoid the earphone charging. Optionally, the jack of the second rotating plug is a 3.5mm round head jack.

[0085] The second rotating plug is also provided with a hook for hanging the noise reduction earphone 3. When the earphone is not in use, the noise reduction earphone 3 can be clamped on the hook of the second rotating plug to avoid affecting the operation.

[0086] In an optional embodiment, the noise reduction headset 3 uses an adaptive dynamic filtering algorithm to filter noise in the voice data and uses a convolutional neural network to recognize voiceprint features in the voice data.

[0087] By filtering out noise through an adaptive dynamic filtering algorithm, the on-site operator's conversation and video playback can be guaranteed. For actively identified equipment noise, a convolutional neural network-based equipment fault identification model is used to perform voiceprint feature recognition of moving equipment or energized equipment.

[0088] In the embodiment of the present application, the noise reduction earphone 3 ensures the clarity of voice communication in a noisy hydropower station environment, facilitating communication between operators and with the command center.

[0089] In an optional embodiment, the lead wire of the microphone 4 is made of an armored metal hose, which facilitates the adjustment of the distance between the microphone 4 and the mouth, ensuring that the human voice can be well collected in a high-noise environment.

[0090] In an optional embodiment, a third rotating plug is provided on the head ring, and the microphone 4 is connected to the third rotating plug. The third rotating plug can support the up and down rotation of the microphone 4. When the microphone 4 is not needed, the microphone 4 can be rotated to the top of the helmet body to avoid affecting the operation.

[0091] In an optional embodiment, the headband is further provided with an interface conversion module 6, and the multi-module camera 1, the noise reduction headset 3, the microphone 4, the augmented reality glasses 2 and the knob button 5 are all connected to the main control module 7 through the interface conversion module 6;

[0092] The interface conversion module 6 is used to convert the image data collected by the multi-module camera 1, the voice data of the microphone 4 and the key function data of the knob button 5, and transmit them to the main control module 7 via TYPE-C;

[0093] The interface conversion module 6 is also used to convert the target display information in the main control module 7 into MIPI interface data and transmit it to the augmented reality glasses 2;

[0094] The interface conversion module 6 is also used to convert the voice data received by the main control module 7 into IIS interface data, and transmit it to the noise reduction headset 3.

[0095] In the embodiment of the present application, the knob button transmits a signal to the augmented reality glasses 2 through the interface conversion module 6 to control the flipping of the target display information in the augmented reality glasses 2. The interface conversion module 6 is implemented using the self-controllable Longxun LT7911UXC chip.

[0096] In an optional embodiment, the head-mounted device further includes a lighting module 11; a light and sturdy slide rail is provided on the head ring, and the lighting module 11 is detachably mounted on the slide rail through a fourth rotating plug. The slide rail is designed as a groove structure, and the lighting module 11 can be fixed on the slide rail through the fourth rotating plug module when needed, and can be removed when not needed. The lighting module 11 can be rotated up and down through the fourth rotating plug, which is convenient for adjusting the center position of the light.

[0097] In an optional embodiment, the waist-wrap device further includes a container, and the main control module 7, the 5G communication module 9 and the UWB personnel positioning module 8 are all disposed in the container.

[0098] In the embodiment of the present application, the augmented reality glasses 2, the multi-module camera 1, the noise reduction headphones 3 and the microphone 4 that must be placed on the head are placed on the head. The remaining necessary equipment is distributed with emphasis using a waist-carrying design, in which a container is set around the waist, and the main control module 7, the 5G communication module 9 and the UWB personnel positioning module 8 are arranged in the waist-carrying container. The lighting module 11 that is not necessary is designed to be detachable, which greatly reduces the head load of the wearer.

[0099] In an optional embodiment, the waist-hung device further includes a backup power supply, which is used to supply power to the main control module 7 and to supply power to the UWB personnel positioning module 8 through the main control module 7 .

[0100] Most of the existing smart helmet devices use built-in batteries in the helmet. On the one hand, it will cause the head to bear a large weight, affecting the wearing comfort and putting pressure on the neck; it also limits the internal space of the helmet, which is not conducive to functional expansion and may cause poor heat dissipation. On the other hand, maintenance is inconvenient, and battery replacement requires professional operation, which increases costs and has poor versatility. In the embodiment of the present application, the backup power supply is set in the waist-span container and connected to the main control module 7, and the main control module 7 is connected to the head-mounted device through the follow-up wiring harness, thereby realizing power supply to the head-mounted device. And because the backup power supply is set in the waist-span container, the weight of the head-mounted device is reduced, and the wearing comfort and space utilization of the head-mounted device are improved.

[0101] Among them, the backup power supply adopts a high-energy-density lithium battery, and the main control module 7 implements intelligent power management to dynamically adjust the power supply according to the status of the components.

[0102] In an optional embodiment, the physical sign data monitored by the smart bracelet include heart rate and blood oxygen saturation, and the exercise data include cadence and number of steps.

[0103] Among them, the smart bracelet is connected to the 5G communication module 9 via Bluetooth to realize 5G communication, and then transmits the physical sign data and exercise data of the detected workers to the comprehensive emergency management and control platform in real time.

[0104] In the embodiment of the present application, the body sign data and exercise volume data are collected by a smart bracelet, and the data are processed by the main control module 7 to make a scientific and accurate assessment of the work intensity and workload of the operators.

[0105] In terms of vital sign monitoring, the smart bracelet has built-in heart rate sensors and blood oxygen sensors to collect the heart rate and blood oxygen saturation of the workers. Advanced sensing technology can accurately collect the data of the workers and ensure the reliability of the data. The main control module 7 efficiently processes the collected heart rate and blood oxygen saturation, and then transmits them to the data processing platform or interacts with the smart helmet through wireless methods such as Bluetooth.

[0106] In terms of exercise volume monitoring, the wristband device 10 has a built-in acceleration sensor to collect the operator's cadence and number of steps.

[0107] In the main control module 7, combined with the preset behavior process intelligent recognition and evaluation algorithm, the work intensity and workload of the operators can be scientifically and accurately evaluated based on the cadence, number of steps, heart rate, blood oxygen and other data. The wristband device 10 and the head-mounted device cooperate with each other to provide all-round protection for the vital sign monitoring and operation management of the operators of large hydropower stations.

[0108] Optionally, the steps for implementing the behavior process intelligent identification and evaluation algorithm are as follows:

[0109] Step 1. Input data parameters: heart rate (HR), blood oxygen saturation (SpO2), cadence (number of steps / time), number of steps (total number of steps)

[0110] Step 2. Data standardization: In order to process the data at the same scale, the embodiment of the present application standardizes the heart rate, blood oxygen saturation, cadence and number of steps (mean is 0, standard deviation is 1)

[0111] Step 3. Data feature extraction: divide the standardized heart rate, blood oxygen saturation, cadence and number of steps into certain time windows (every 1 minute or every 5 minutes), and calculate statistical features for each window. The physiological features contained in each window include heart rate features, blood oxygen saturation features, cadence features and number of steps features. Among them, statistical features include mean, maximum, minimum and standard deviation;

[0112] Step 4. Data label design: Assign a label to each time window, which includes the work behavior category and work intensity level. For example, the work behavior category includes walking, standing and resting; the work intensity level includes mild, moderate and severe, which can be adaptively set according to needs.

[0113] Step 5. Process intelligent identification model:

[0114] Input layer: The physiological signal features of each time window are used as the input of the preset intelligent recognition model:

[0115] timesteps = X_train_scaled.shape[1] represents the time window size

[0116] n_features = X_train_scaled.shape[2] represents the number of features

[0117] LSTM layer: Processes time series data through a long short-term memory network (LSTM), learns the dependencies between different time steps, and captures the dynamic changes of job behaviors:

[0118] #First layer of LSTM

[0119] model.add(LSTM(units=128,activation='tanh',input_shape=(timesteps,n_features),return_sequences=True))

[0120] model.add(Dropout(0.2))

[0121] #The second layer of LSTM

[0122] model.add(LSTM(units=64,activation='tanh'))

[0123] model.add(Dropout(0.2))

[0124] Output layer: The physiological characteristics are judged based on the labels, and the output is the behavior category (e.g., walking, resting, standing) and / or the work intensity level (mild, moderate, severe) for each time window:

[0125] In the action recognition classification task:

[0126] model.add(Dense(3,activation='softmax'))

[0127] Output 3 behavior categories: walking, resting, standing.

[0128] In the homework intensity assessment task:

[0129] By inputting the activity type (such as walking, resting) and physiological indicators (such as heart rate), the intensity of the work can be judged according to different rules. Through this simple rule engine, the high, medium and low levels of work intensity can be evaluated:

[0130] If the behavior recognition is walking and the heart rate is high (>120), the work intensity is high ('high');

[0131] If the behavior identification is walking, the heart rate is low (≤120), and the work intensity is moderate ('moderate');

[0132] If the behavior identification is rest, the heart rate is low (<80), and the work intensity is low ('low');

[0133] In other cases, the default job intensity is medium.

[0134] In the embodiment of the present application, the physical sign monitoring sensor technology is combined with the smart bracelet, which can grasp the physical condition of the operators in real time, evaluate the work intensity and workload, detect abnormalities in time, prevent accidents, and arrange work tasks reasonably.

[0135] In an optional embodiment, the 5G communication module 9 transmits the voice and image data acquired by the main control module 7 to the data processing platform through the surrounding 5G network, so as to timely discover hidden dangers and faults. This enables the operator to communicate with experts outside the operating environment and obtain expert guidance. The 5G communication module 9 is better integrated into the data processing platform to realize functions such as positioning and improve management and operation efficiency.

[0136] In an optional embodiment, the UWB personnel positioning modules 8 and 5G communication modules 9 on the waists of multiple people are combined with the 5G network to form a collaborative map between multiple smart wearable terminal devices. Each smart wearable terminal device on site calculates the distance to other terminals in real time, and can push each other's voice, video and text information, thereby realizing multi-person collaborative work and mutual rescue in the first time, improving work efficiency and the safety of operators.

[0137] In an optional embodiment, the main control module 7 is provided with an edge computing chip for intelligent analysis (such as Huawei Ascend (Ascend310)), which can perform edge intelligent identification of personnel, behaviors, equipment, instruments, control units, environment and other elements monitored by the head-mounted device and the bracelet device 10, and upload abnormal and dangerous information.

[0138] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application also provides a large hydropower station emergency management method, the method comprising:

[0139] Acquire an image of the current working environment through the multi-module camera 1;

[0140] In the main control module 7, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, a three-dimensional map of the image is constructed based on the real-time positioning and mapping algorithm, and the category information and location information of the object are mapped into the three-dimensional map to identify the dangerous area;

[0141] The main control module 7 sends the category information, location information and three-dimensional map of the object to the augmented reality glasses 2 for display;

[0142] Acquire the physical sign data and exercise volume data of the monitored operator through the wristband device 10;

[0143] In the main control module 7, the physical sign data and exercise data are processed based on the preset behavior process intelligent recognition and evaluation algorithm to obtain the behavior category and work intensity level;

[0144] The positioning status information of the operator is obtained through the UWB personnel positioning module 8, the multi-module camera 1 and the wristband device 10;

[0145] In the main control module 7, the movement status information of the operator is determined by the positioning status information;

[0146] The operator's movement status information, dangerous area behavior category and work intensity are sent to the data processing platform through the 5G communication module 9.

[0147] In the embodiment of the present application, the images of the working site environment and the working process are collected by a multi-module camera 1, voice communication is realized by a microphone 4 and a noise reduction headset 3, the required images and text information are displayed by an augmented reality glasses 2, the physical sign data and exercise data of the operating personnel are monitored by a bracelet device 10, the operating personnel are positioned by a UWB personnel positioning module 8, and the data sent by the above-mentioned devices are analyzed and processed by a main control module 7, and the operating intensity and workload are evaluated by combining the physical sign data, exercise data and positioning, and more accurate positioning status information is determined. Through the 5G communication module 9, information interaction with the data processing platform can be realized to facilitate guidance by experts outside the working environment; interaction with other operating personnel can also be realized to realize multi-person collaborative operation and mutual rescue in the first time. Therefore, in the embodiment of the present application, comprehensive monitoring of the physical signs, environmental identification and working intensity of the operating personnel can be realized, providing comprehensive and effective data support for the emergency management of large hydropower stations, and meeting the precise needs of emergency management of large hydropower stations.

[0148] In an optional embodiment, the multi-module camera 1 includes a telephoto camera, a thermal imaging camera and a wide-angle camera, which are used to respectively obtain a telephoto image, a thermal imaging image and a wide-angle image of the current working environment;

[0149] In the main control module 7, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, a three-dimensional map of the image is constructed based on the real-time positioning and map construction algorithm, and the category information and location information of the object are mapped to the three-dimensional map to identify the dangerous area, including:

[0150] Preprocessing the telephoto image, thermal image and wide-angle image, including image denoising, color normalization and image alignment;

[0151] Use the YOLO target detection algorithm to classify the objects in the preprocessed telephoto image, thermal image, and wide-angle image, and select the location of the object to obtain the category information and location information;

[0152] The real-time positioning and mapping algorithm is used to extract the key feature points in the pre-processed telephoto image, thermal image and wide-angle image, and the relative displacement between the telephoto camera, thermal camera and wide-angle camera is calculated through feature matching;

[0153] Based on relative displacement, a 3D map is constructed through graph optimization, and the position and posture of the telephoto camera, thermal camera, and wide-angle camera in space are estimated;

[0154] Map the category information and location information of the object into a three-dimensional map to obtain an environment layout map;

[0155] Identify hazardous areas based on the environment layout map.

[0156] In an embodiment of the present application, through augmented reality technology, the target object category and position information recognized by the YOLO model are superimposed on the augmented reality glasses 2, and the three-dimensional map constructed by the SLAM algorithm can be presented in the augmented reality glasses 2 through AR technology, providing real-time spatial navigation and path planning for operators, and providing intuitive and convenient operation guidance for operators in complex hydropower station environments.

[0157] In an optional embodiment, in the main control module 7, the physical sign data includes heart rate and blood oxygen saturation; the exercise data includes cadence and number of steps;

[0158] Based on the preset behavior process intelligent recognition and evaluation algorithm, the physical sign data and exercise data are processed to obtain the behavior category and work intensity level, including:

[0159] The heart rate, blood oxygen saturation, cadence and number of steps were normalized (mean = 0, standard deviation = 1) to facilitate processing on the same scale;

[0160] The standardized heart rate, blood oxygen saturation, cadence and number of steps are divided into certain time windows (every 1 minute or every 5 minutes), and statistical features are calculated for each time window; the statistical features include mean, maximum, minimum and standard deviation; the physiological features of each time window include heart rate features, blood oxygen saturation features, cadence features and number of steps features;

[0161] Assign a label to each time window; the label includes the work behavior category and work intensity level; the work behavior category includes walking, standing and resting; the work intensity level includes mild, moderate and severe, which can be adaptively set according to needs;

[0162] The physiological characteristics of each time window are used as the input of the preset intelligent recognition model. The dependencies between different time steps are learned through the long short-term memory network to capture the dynamic changes of work behavior. The physiological characteristics are judged based on the labels to obtain the behavior category and work intensity level.

[0163] Specifically, the input layer takes the physiological signal features of each time window as the input of the preset intelligent recognition model:

[0164] timesteps = X_train_scaled.shape[1] represents the time window size

[0165] n_features = X_train_scaled.shape[2] represents the number of features

[0166] LSTM layer: Processes time series data through a long short-term memory network (LSTM), learns the dependencies between different time steps, and captures the dynamic changes of job behaviors:

[0167] #First layer of LSTM

[0168] model.add(LSTM(units=128,activation='tanh',input_shape=(timesteps,n_features),return_sequences=True))

[0169] model.add(Dropout(0.2))

[0170] #The second layer of LSTM

[0171] model.add(LSTM(units=64,activation='tanh'))

[0172] model.add(Dropout(0.2))

[0173] Output layer: The physiological characteristics are judged based on the labels, and the output is the behavior category (e.g., walking, resting, standing) and / or the work intensity level (mild, moderate, severe) for each time window:

[0174] In the action recognition classification task:

[0175] model.add(Dense(3,activation='softmax'))

[0176] Output 3 behavior categories: walking, resting, standing.

[0177] In the homework intensity assessment task:

[0178] By inputting the activity type (such as walking, resting) and physiological indicators (such as heart rate), the intensity of the work can be judged according to different rules. Through this simple rule engine, the high, medium and low levels of work intensity can be evaluated:

[0179] If the behavior recognition is walking and the heart rate is high (>120), the work intensity is high ('high');

[0180] If the behavior identification is walking, the heart rate is low (≤120), and the work intensity is moderate ('moderate');

[0181] If the behavior identification is rest, the heart rate is low (<80), and the work intensity is low ('low');

[0182] In other cases, the default job intensity is medium.

[0183] The large hydropower station emergency management method provided in the embodiment of the present application can achieve Figure 1 and Figure 2 To avoid repetition, the various functions implemented in the smart wearable terminal device embodiment will not be described here.

[0184] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A smart wearable terminal device for emergency management of large hydropower stations, characterized in that: The equipment includes a head-mounted device, a wristband device and a waist-mounted device; The head-mounted device includes a helmet body, and a multi-module camera, a noise reduction headset, a microphone and augmented reality glasses arranged on the helmet body; The wristband device is used to monitor the physical sign data and exercise data of the operator; The waist-span device includes a main control module, a 5G communication module and a UWB personnel positioning module; The main control module is connected to the multi-module camera, the noise reduction headset, the microphone and the augmented reality glasses by wires, and is connected to the bracelet device wirelessly; The 5G communication module is wirelessly connected to the main control module to realize the interaction between the main control module and the data processing platform.

2. The smart wearable terminal device according to claim 1, characterized in that: The helmet body comprises a head ring, and the multi-module camera, the noise reduction headphones, the microphone and the augmented reality glasses are all connected to the head ring.

3. The smart wearable terminal device according to claim 3, characterized in that: The headband is also provided with an interface conversion module, through which the multi-module camera, the noise reduction headset, the microphone, the augmented reality glasses and the knob button are all connected to the main control module; The interface conversion module is used to convert the image data collected by the multi-module camera, the voice data of the microphone and the key function data of the knob button, and transmit them to the main control module via TYPE-C; The interface conversion module is also used to convert the target display information in the main control module into MIPI interface data and transmit it to the augmented reality glasses; The interface conversion module is also used to convert the voice data received by the main control module into IIS interface data, and transmit it to the noise reduction headset.

4. The smart wearable terminal device according to claim 2, characterized in that: The augmented reality glasses are used to display target display information, which includes video call information, other people's perspective information, display device description information, emergency plan information, personnel identification information, license information and tool inspection information; The headband is provided with a knob button, and the knob button is used to control the flipping of the content displayed by the augmented reality glasses; The functions of the knob button include jumping to the previous page, jumping to the next page, and determining the content of the current page.

5. The smart wearable terminal device according to claim 1, characterized in that: The noise reduction headset uses an adaptive dynamic filtering algorithm to filter noise in voice data and uses a convolutional neural network to recognize voiceprint features in voice data.

6. The smart wearable terminal device according to claim 1, characterized in that: The waist-mounted device also includes a container, and the main control module, the 5G communication module and the UWB personnel positioning module are all arranged in the container.

7. The smart wearable terminal device according to claim 1, characterized in that: The waist-mounted device also includes a backup power supply, which is used to supply power to the main control module and to supply power to the UWB personnel positioning module through the main control module.

8. A large hydropower station emergency management method, characterized in that: The method is implemented based on the smart wearable terminal device described in any one of claims 1 to 7, and includes: Acquire images of the current working environment through multi-module cameras; In the main control module, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, a three-dimensional map of the image is constructed based on the real-time positioning and mapping algorithm, and the category information and location information of the object are mapped into the three-dimensional map to identify the dangerous area; The main control module sends the category information, location information and the three-dimensional map of the object to the augmented reality glasses for display; Obtain the physical sign data and exercise data of the monitoring workers through the wristband device; In the main control module, the physical sign data and the exercise data are processed based on a preset behavior process intelligent recognition and evaluation algorithm to obtain the behavior category and the work intensity level; Obtain the positioning status information of the operator through the UWB personnel positioning module, multi-module camera and wristband device; In the main control module, the movement status information of the operator is determined through the positioning status information; The operator's movement status information, the behavior category in the dangerous area, and the work intensity are sent to the data processing platform through the 5G communication module.

9. The large hydropower station emergency management method according to claim 8, characterized in that: The images of the current working environment acquired by the multi-module camera include a telephoto image, a thermal image and a wide-angle image; In the main control module, the category information and location information of the object are obtained from the image based on the YOLO target detection algorithm, a three-dimensional map of the image is constructed based on the real-time positioning and map construction algorithm, and the category information and location information of the object are mapped to the three-dimensional map to identify the dangerous area, including: Preprocessing the telephoto image, the thermal image and the wide-angle image; the preprocessing includes image denoising, color normalization and image alignment; Using the YOLO target detection algorithm to classify the objects in the preprocessed telephoto image, thermal image, and wide-angle image, respectively, and select the positions of the objects to obtain the category information and the position information; Using a real-time positioning and mapping algorithm to extract key feature points in the preprocessed telephoto image, thermal image and wide-angle image, respectively, and calculating the relative displacement between the telephoto camera, the thermal image camera and the wide-angle camera through feature matching; Based on the relative displacement, construct the three-dimensional map through graph optimization, and estimate the positions and postures of the telephoto camera, the thermal imaging camera, and the wide-angle camera in space; Mapping the category information and the location information of the object into the three-dimensional map to obtain an environment layout map; The dangerous area is identified based on the environment layout map.

10. The large hydropower station emergency management method according to claim 9, characterized in that: In the main control module, the physical sign data includes heart rate and blood oxygen saturation; the exercise data includes cadence and number of steps; The preset behavior process intelligent recognition and evaluation algorithm processes the physical sign data and the exercise volume data to obtain the behavior category and the work intensity level, including: Performing data normalization processing on the heart rate, the blood oxygen saturation, the step frequency and the number of steps; The standardized heart rate, blood oxygen saturation, cadence and number of steps are divided into certain time windows, and statistical features are calculated for each time window; wherein the statistical features include mean, maximum, minimum and standard deviation; the physiological features of each time window include heart rate features, blood oxygen saturation features, cadence features and number of steps features; Assign a label to each time window; the label includes a work behavior category and a work intensity level; the work behavior category includes walking, standing and resting; the work intensity level includes mild, moderate and severe; The physiological characteristics of each time window are used as the input of the preset intelligent recognition model. The dependencies between different time steps are learned through the long short-term memory network to capture the dynamic changes of the work behavior. The physiological characteristics are judged based on the labels to obtain the behavior category and the work intensity level.

Images