Hydropower station personnel safety early warning method and system based on digital twinning

Abstract

The invention provides a hydropower station personnel safety early warning method and system based on digital twinning, and relates to the technical field of data processing. The method comprises the following steps: constructing a digital twinborn model according to building information data and geographic space data of a hydropower station, and marking a space coordinate range of each monitoring area in the digital twinborn model; collecting monitoring video data of each monitoring area, and identifying an intertidal zone image and an image of a person entering the intertidal zone from the monitoring video data; determining a position coordinate corresponding to the personnel image, and determining a target monitoring area corresponding to the personnel image according to a space coordinate range to which the position coordinate belongs; mounting a preset video clip corresponding to the personnel image to a position corresponding to the target monitoring area in the digital twin model; and determining the relative position of the personnel in the mud flat area, generating corresponding early warning information, and pushing the early warning information. According to the technical scheme, spatial positioning and visual early warning of the personnel entering the hydropower station monitoring area can be realized.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a personnel safety early warning method and system for a hydropower station based on digital twins. Background Art

[0002] As critical energy infrastructure, hydropower stations operate in complex and diverse environments. This is particularly true of tidal flats, which, due to their low-lying terrain, frequently fluctuating water flows, and open environment, are often prone to safety incidents. Safety risks in these areas are particularly acute during flood season or during flood discharges. Accidents such as drowning and impact injuries are highly likely to occur when people mistakenly enter or remain on these areas. Therefore, effective personnel monitoring and risk warnings in these areas are crucial for ensuring safe operation of hydropower stations.

[0003] In the related art, when conducting safety monitoring of hydropower stations, industrial television systems are usually used in conjunction with manual on-site inspections to conduct image observation and personnel control in the tidal flat area. However, this monitoring method has certain shortcomings. On the one hand, the industrial television system relies on fixed-placed camera equipment, and its monitoring content is often presented in the form of two-dimensional images, which makes it difficult to achieve spatial positioning of personnel in the tidal flat area. On the other hand, the manual on-site inspection method relies on regular inspections and subjective judgments by personnel, which has problems such as delayed response, high labor intensity, and inability to quickly warn in emergencies. Therefore, there is still room for improvement in the scene expression ability and warning effect of the safety warning technology for hydropower station personnel in the related art.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute prior art known to ordinary technicians in the field. Summary of the Invention

[0005] The purpose of the embodiments disclosed herein is to provide a hydropower station personnel safety warning method based on digital twins, a hydropower station personnel safety warning system based on digital twins, an electronic device and a computer-readable storage medium. By spatially associating personnel identification results with the hydropower station digital twin model and performing a warning response, the scene expression capability and warning effect of the hydropower station personnel safety warning technology can be improved at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0007] According to a first aspect of an embodiment of the present disclosure, a personnel safety warning method for a hydropower station based on digital twins is provided, the method comprising: constructing a digital twin model based on building information data and geographic spatial data of the hydropower station, and marking the spatial coordinate range of each monitoring area in the digital twin model; collecting monitoring video data of each of the monitoring areas, and using a machine learning algorithm to identify a tidal flat area image and an image of a person entering the tidal flat area from the monitoring video data; determining the position coordinates corresponding to the person image, and determining the target monitoring area corresponding to the person image based on the spatial coordinate range to which the position coordinates belong; mounting a preset video clip corresponding to the person image to a position corresponding to the target monitoring area in the digital twin model, and visually displaying it; determining the relative position of the person in the tidal flat area based on the tidal flat area image and the corresponding person image, generating corresponding warning information based on the relative position, and pushing the warning information in a variety of ways.

[0008] According to a second aspect of an embodiment of the present disclosure, a digital twin-based hydropower station personnel safety warning system is provided, which is used to implement the above-mentioned digital twin-based hydropower station personnel safety warning method. The system includes: a twin model construction module, which is used to construct a digital twin model based on the building information data and geographic spatial data of the hydropower station, and mark the spatial coordinate range of each monitoring area in the digital twin model; an image recognition module, which is used to collect monitoring video data of each monitoring area, and use a machine learning algorithm to identify a tidal flat area image and a person image entering the tidal flat area from the monitoring video data; a target area determination module, which is used to determine the position coordinates corresponding to the person image, and determine the target monitoring area corresponding to the person image based on the spatial coordinate range to which the position coordinates belong; a video mapping module, which is used to mount a preset video clip corresponding to the person image to the position corresponding to the target monitoring area in the digital twin model and visually display it; and a warning information generation module, which is used to determine the relative position of the person in the tidal flat area based on the tidal flat area image and the corresponding person image, generate corresponding warning information based on the relative position, and push the warning information in various ways.

[0009] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, a digital twin-based hydropower station personnel safety warning method as in the first aspect is implemented.

[0010] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the digital twin-based hydropower station personnel safety warning method as in the first aspect is implemented.

[0011] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects: The digital twin-based hydropower station personnel safety warning method in the disclosed embodiments, on the one hand, utilizes the hydropower station's building information data and geospatial data to construct a digital twin model, and annotates the spatial coordinate range of the monitoring area within it. This provides an accurate spatial reference basis for subsequent recognition results, effectively resolving the problem in related technologies where monitoring results cannot be correlated with the actual spatial environment. On the other hand, by identifying images of the mudflat area and images of entering personnel based on monitoring video data, and further determining the corresponding position coordinates of the personnel images, spatial positioning can be achieved based on image recognition, thereby improving the accuracy of identifying personnel entry behavior. Furthermore, pre-set video clips corresponding to the personnel images are mounted to the corresponding locations of the target monitoring areas in the digital twin model and visualized, allowing management personnel to intuitively observe personnel behavior in a unified virtual environment. Furthermore, generating corresponding warning information based on the personnel's relative position in the mudflat area can improve the targeted nature of the warning response. This can, to a certain extent, enhance the scenario-based representation capabilities and warning effectiveness of hydropower station personnel safety warning technology.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort.

[0014] Figure 1 A flow chart of a hydropower station personnel safety early warning method based on digital twins according to some embodiments of the present disclosure is schematically shown.

[0015] Figure 2 A schematic diagram of a process for constructing a digital twin model according to some embodiments of the present disclosure is schematically shown.

[0016] Figure 3 A schematic diagram of the architecture of a hydropower station personnel safety warning system based on digital twins according to some embodiments of the present disclosure is schematically shown.

[0017] Figure 4 The present invention schematically illustrates a composition diagram of a hydropower station personnel safety warning system based on digital twin according to some embodiments of the present disclosure.

[0018] Figure 5 A schematic structural diagram of a computer system of an electronic device according to some embodiments of the present disclosure is schematically shown.

[0019] Figure 6 A schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure is schematically shown.

[0020] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts. DETAILED DESCRIPTION

[0021] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with this specification. Rather, they are merely examples of apparatus and methods consistent with certain aspects of this specification, as detailed in the appended claims.

[0022] The terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this specification. As used in this specification and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc. may be used in this specification to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information without departing from the scope of this specification. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0024] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0025] In addition, the described features, structures or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present disclosure.

[0026] Furthermore, the drawings are schematic illustrations only and are not necessarily drawn to scale. The block diagrams shown in the drawings are merely functional entities and do not necessarily correspond to physically separate entities. In other words, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0027] In this example embodiment, a hydropower station personnel safety early warning method based on digital twins is first provided. Figure 1 The following schematically illustrates a flow chart of a method for early warning of personnel safety in a hydropower station based on digital twins according to some embodiments of the present disclosure. Figure 1 As shown, the digital twin-based hydropower station personnel safety early warning method may include the following steps: Step S110: constructing a digital twin model based on the building information data and geographic spatial data of the hydropower station, and marking the spatial coordinate range of each monitoring area in the digital twin model; Step S120: collecting monitoring video data of each monitoring area, and using a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data; Step S130, determining the position coordinates corresponding to the person image, and determining the target monitoring area corresponding to the person image based on the spatial coordinate range to which the position coordinates belong; Step S140: Mount the preset video clip corresponding to the person image to the location corresponding to the target monitoring area in the digital twin model and perform a visual display; Step S150: Determine the relative position of the person in the tidal flat area based on the tidal flat area image and the corresponding person image, generate corresponding warning information based on the relative position, and push the warning information through various means.

[0028] According to the digital twin-based hydropower station personnel safety warning method of this example embodiment, on the one hand, by constructing a digital twin model using the hydropower station's building information data and geospatial data and annotating the spatial coordinate range of the monitoring area, an accurate spatial reference basis can be provided for subsequent recognition results. On the other hand, by identifying images of the mudflat area and images of entering personnel based on monitoring video data, and further determining the position coordinates corresponding to the personnel images, spatial positioning can be achieved based on image recognition, thereby improving the accuracy of identifying personnel entry behavior. On the other hand, the preset video clips corresponding to the personnel images are mounted to the corresponding positions of the target monitoring areas in the digital twin model and visualized, allowing managers to intuitively observe personnel behavior in a unified virtual environment. In addition, generating corresponding warning information based on the relative positions of personnel in the mudflat area can improve the targeted warning response. As a result, the scene expression capabilities and warning effects of hydropower station personnel safety warning technology can be improved to a certain extent.

[0029] Below, the digital twin-based hydropower station personnel safety warning method in this example embodiment will be further explained.

[0030] In step S110, a digital twin model is constructed based on the building information data and geographic space data of the hydropower station, and the spatial coordinate range of each monitoring area is marked in the digital twin model.

[0031] Building information data can represent the three-dimensional geometric parameters, spatial positional relationships, and structural attribute information related to the hydropower station's structural components. Specifically, this data can include building information for target objects such as powerhouse structures, hydraulic structures, tidal flats, reservoirs, power station areas, and dams. Geospatial data can represent the terrain undulations, elevation distribution, image coverage, and spatial positioning information related to the hydropower station's geographic area. A digital twin model can represent a virtual three-dimensional spatial scene generated by the fusion of building information data and geospatial data. This model can be constructed based on BIM modeling and GIS scene construction techniques, using a unified spatial coordinate system to describe the spatial correspondence between components and geographic elements. The monitoring area can represent target areas within the hydropower station that require key monitoring. This can include tidal flats, flood discharge channels, and other areas with potential risks. The spatial coordinate range can represent the spatial boundary corresponding to the monitoring area in the digital twin model.

[0032] In step S120, monitoring video data of each monitoring area is collected, and a machine learning algorithm is used to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data.

[0033] The monitoring video data may represent video content data collected by monitoring equipment installed within the monitoring area, and the monitoring video data may include visible light video data and infrared video data. The machine learning algorithm may represent an algorithm used to extract features and recognize image content from the monitoring video data. The machine learning algorithm may include other suitable algorithms such as convolutional neural networks, semantic segmentation algorithms, target detection algorithms, and thermal feature recognition algorithms. The tidal flat area image may represent an image that can represent the tidal flat area of a hydropower station. The human image may represent an image region with human appearance characteristics identified in the tidal flat area image via an image recognition algorithm.

[0034] In step S130 , the position coordinates corresponding to the person image are determined, and the target monitoring area corresponding to the person image is determined according to the spatial coordinate range to which the position coordinates belong.

[0035] The location coordinates can represent the coordinate information of the spatial location corresponding to the person image at the hydropower station. The target monitoring area can represent the monitoring area corresponding to the location coordinates. In this step, by determining the location coordinates corresponding to the person image and combining them with the spatial coordinate range to determine the corresponding target monitoring area, the recognition results can be accurately located in the spatial dimension and assigned to the region, thereby improving the targeted nature of subsequent warning responses.

[0036] In step S140, the preset video clip corresponding to the personnel image is mounted to the position corresponding to the target monitoring area in the digital twin model and visualized.

[0037] The preset video clips can represent sequences of video frames extracted from the monitoring video data at predetermined time intervals after a person's image is identified and associated with that person's image, for visualization and playback within the digital twin model. This step, by adding the preset video clips corresponding to the person's image to the corresponding location in the target monitoring area within the digital twin model, dynamically associates the recognition results with the three-dimensional spatial scene, enhancing the visualization of risk events and improving managers' intuitive perception and judgment of personnel behavior and on-site locations.

[0038] In step S150, the relative position of the person in the tidal flat area is determined based on the tidal flat area image and the corresponding person image, corresponding warning information is generated based on the relative position, and the warning information is pushed through various methods.

[0039] Relative position can represent a person's spatial position relative to the tidal flat boundary, center, or pre-set reference point within the tidal flat area. This relative position reflects the person's risk zone within the tidal flat area, ensuring that warning results correspond to the person's actual risk zone, thereby enhancing the relevance of warning information. After receiving warning information, it can be pushed to a remote management terminal via pop-up prompts, voice broadcasts, or image overlays. It can also be synchronized to on-site management personnel and target personnel via text messages, mobile app notifications, or dedicated network channels.

[0040] The technical contents of the above embodiments are described in detail below.

[0041] In some embodiments, reference Figure 2 As shown in the figure, a digital twin model is constructed based on the building information data and geospatial data of the hydropower station, which specifically includes the following technical steps: Step S210 , collecting building information data and geographic space data of the hydropower station, where the building information data includes three-dimensional component information of the powerhouse structure, hydraulic structures and tidal flat area, and the geographic space data includes terrain elevation data and remote sensing images.

[0042] Among them, the powerhouse structure can represent the powerhouse building entity within a hydropower station. Hydraulic structures can represent engineering structures within the hydropower station used for purposes such as water diversion, flood discharge, water retention, or navigation, such as dams, intake towers, and diversion channels. Tidal flats can represent shallows or wetlands located within the hydropower station reservoir or downstream river channel, which are periodically exposed due to water level fluctuations throughout the year. These areas are characterized by low-lying terrain, wadable water, and high risk of human exposure. Three-dimensional component information can represent the geometric dimensions, three-dimensional shape, structural relationships, and material properties of the spatial components that form the powerhouse structure, hydraulic structures, or tidal flat area boundaries. This data can be used to generate virtual component objects within the BIM environment. Elevation data can represent spatial numerical information that reflects the topographical changes within the hydropower station area. Remote sensing images can represent ground reflection images of the hydropower station area acquired using aerial or satellite remote sensing equipment. These images can include characteristic information such as surface texture, vegetation, water areas, and artificial facilities.

[0043] Step S220: constructing a BIM structural model based on the building information data, and constructing a GIS basic scene based on the geographic space data.

[0044] The BIM structural model represents a three-dimensional digital engineering model built based on building information data. This model can include the geometric shape, spatial position, structural relationships, and attribute information of various structural components of the hydropower station. It can also represent the spatial combination of physical objects such as powerhouse structures, hydraulic structures, and tidal flats in a unified coordinate system. The GIS basic scene represents a spatial scene environment with real geographic reference coordinates built based on geospatial data. Its content can include terrain elevation, remote sensing imagery, feature boundaries, and spatial projection information, and is used to express the geographical background and surface spatial distribution characteristics of the area where the hydropower station is located.

[0045] In step S230, a unified coordinate projection system is used to spatially align the BIM structural model and the GIS basic scene, and the aligned BIM structural model and the GIS basic scene are data-fused to generate a three-dimensional model dataset containing the correspondence between structural information and geographic information.

[0046] Among them, the coordinate projection system can represent the projection rules used to unify three-dimensional geographic spatial data and building structure data into the same spatial reference framework, and is used to ensure that the BIM structural model and the GIS basic scene are superimposed and positionally corresponding in the same spatial scale and coordinate system. Spatial registration can represent the process of spatially aligning the positions of structural components in the BIM structural model with the geographic elements in the GIS basic scene based on the coordinate projection system. Data fusion can represent the operation process of combining the component information in the BIM structural model with the terrain and landform information in the GIS basic scene after completing the spatial registration. The three-dimensional model dataset can represent the spatial data collection formed after the fusion of structural information and geographic information. In this step, the BIM structural model and the GIS basic scene are spatially registered using a unified coordinate projection system, and the two are data-fused to generate a three-dimensional model dataset containing the correspondence between structural information and geographic information, which can achieve accurate matching of building components and terrain.

[0047] Step S240: Import the 3D model dataset into a simulation engine environment to construct a digital twin model. The simulation engine environment can be an interactive visualization platform for loading and running 3D model datasets. This environment supports rendering and status updates of BIM structural components and GIS spatial data, enabling 3D view display and monitoring content mounting.

[0048] In some embodiments, after building the digital twin model, the monitoring video data can also be rendered to the corresponding position in the digital twin model, which specifically includes the following technical steps: First, the spatial position information and observation direction information of the monitoring equipment installed in the monitoring area are obtained. The spatial position information includes the positioning coordinates of the monitoring equipment, and the observation direction information includes the pitch angle and horizontal viewing angle. Among them, the monitoring equipment can refer to an image acquisition device deployed in the monitoring area of the hydropower station for acquiring monitoring video data. It can include fixed cameras, spherical pan-tilt cameras, infrared cameras, and other devices with image acquisition functions. The spatial position information can be used to describe the spatial layout of the monitoring equipment in a unified coordinate system. The observation direction information can be used to indicate the orientation and coverage angle of the field of view of the monitoring equipment. The positioning coordinates can represent the geographic location coordinate values corresponding to the monitoring equipment in the spatial position information.

[0049] Then, based on the spatial position information and observation direction information, the video mounting points corresponding to the monitoring devices are annotated in the digital twin model. The video mounting points can represent spatially annotated locations used to associate monitoring video data in the digital twin model. They can be set based on the spatial position information and observation direction information of the monitoring devices and are used to indicate the projection position and viewing direction of the video data in the three-dimensional scene. In a specific implementation, annotating the video mounting points corresponding to the monitoring devices in the digital twin model can be performed through the following steps: First, based on the acquired spatial position information of the monitoring devices, the positioning coordinates of each monitoring device in the digital twin model are determined, and spatial anchor point data is generated based on the positioning coordinates. Next, in combination with the observation direction information corresponding to each monitoring device, the pitch angle and horizontal viewing angle parameters are combined with the spatial anchor point data to construct a monitoring viewing area containing a spatial pointing vector. Then, based on the spatial interaction relationship between the monitoring viewing area and the digital twin model, the visible coverage area of each monitoring device is determined. Furthermore, a video mounting point is established at the center of each visible coverage area.

[0050] Secondly, a mapping data table is constructed based on the correspondence between the video mount points and the monitoring devices, and the mapping data table is loaded into the simulation engine environment. The mapping data table can represent an associative data structure used to record the spatial correspondence between the video mount points and the monitoring devices, including the positioning coordinates and observation direction parameters of the monitoring devices and their corresponding video mount point information in the digital twin model. This mapping data table can serve as the mapping basis for the visualization rendering process.

[0051] Finally, within the simulation engine environment, the monitored video data is rendered to the corresponding video mount point locations in the digital twin model according to the mapping data table. In this embodiment, by obtaining the spatial location and observation direction information of the monitoring device and annotating the video mount points in the digital twin model based on this information, a one-to-one correspondence between the monitoring device and the digital scene is achieved. Furthermore, the mapping data table provides unified device mount point mapping rules, enhancing the efficiency and manageability of video data calls within the simulation engine.

[0052] In some embodiments, a machine learning algorithm is used to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data, specifically including the following technical steps: using a 5G network to transmit the monitoring video data to an edge node, each edge node having corresponding monitoring equipment; the edge node uses a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data.

[0053] Among them, the edge node can represent a data processing unit deployed in a local area of a hydropower station and close to the monitoring equipment. It has certain computing, storage and intelligent recognition capabilities, and is used to quickly analyze and process the locally collected monitoring video data without relying on a remote data center, and feed back the processing results to the monitoring system. In this embodiment, by utilizing the 5G network to transmit the monitoring video data to the nearest edge node at high speed, each edge node can correspond to one or more monitoring devices and have localized data processing capabilities, thereby greatly reducing the delay of data during network transmission. On the one hand, the edge node uses machine learning algorithms to perform local recognition of monitoring video data, which can quickly extract images of mudflat areas and personnel images, improve recognition efficiency and alleviate the central processing load. On the other hand, the edge node computing mode can pre-process and filter the data, reduce the image data transmitted to the remote monitoring system, and save network bandwidth and cloud platform storage resources.

[0054] In some embodiments, the monitoring video data includes visible light video data and infrared video data, and the edge node uses a machine learning algorithm to identify the mudflat area image and the image of the person entering the mudflat area from the monitoring video data, which specifically includes the following technical steps: in response to the monitoring video data being visible light video data, the edge node performs a reflection suppression operation on the visible light video data to obtain visible light video optimization data; uses a semantic segmentation network to perform a region recognition operation on the visible light video optimization data to obtain the mudflat area image and the image of the person entering the mudflat area; in response to the monitoring video data being infrared video data, the edge node inputs the infrared video data into a thermal feature recognition network to obtain the mudflat area image and the image of the person entering the mudflat area.

[0055] The reflection suppression operation can refer to the process of reducing or eliminating reflection interference in visible light video data caused by reflective materials such as water or glass during image data preprocessing, using image processing techniques such as image enhancement, brightness compression, or local contrast adjustment. A semantic segmentation network can refer to an image recognition structure based on a deep neural network that can divide the input image into several pixel-level regions with clear semantic labels to extract specific target category information in the image. A thermal feature recognition network can refer to a deep learning model used to process infrared images, which can identify the outline of heat source areas in specific areas from infrared video data.

[0056] In some embodiments, a reflection suppression operation is performed on visible light video data, which may specifically include the following technical steps: based on a preset image segmentation rule, the visible light video data is divided into multiple image sub-blocks. The brightness histogram, saturation distribution and edge gradient features are extracted for each image sub-block to generate an image feature group that reflects the current ambient lighting conditions. According to the variation range of brightness, saturation and edge distribution in the image feature group, the judgment parameters of reflection recognition are adaptively adjusted to determine multiple reflection area recognition thresholds. Based on the judgment parameters and the recognition threshold, the visible light video data is regionally scanned to extract image areas with significant reflection intensity, and a corresponding image mask is generated. Based on the image mask and the original image data, a local pixel replacement operation is performed on the reflective area, and the pixel value is corrected using multi-scale filtering and texture reconstruction to generate a reflection suppression image, which is then used for mudflat area recognition and personnel detection.

[0057] Since there are often multiple monitoring devices set up in a monitoring area, in some embodiments, the preset video clip corresponding to the personnel image is mounted to the position corresponding to the target monitoring area in the digital twin model, which specifically includes the following technical steps: responding to the personnel image corresponding to multiple target monitoring video data, wherein the multiple target monitoring video data cover the same target monitoring area; selecting the optimal monitoring video data from the multiple target monitoring video data based on the shooting angle and image clarity; using the optimal monitoring video data as the main perspective content and the remaining target monitoring video data as the auxiliary perspective content, and uniformly mounting them to the position corresponding to the target monitoring area in the digital twin model. In this embodiment, the optimal monitoring video data is selected based on the two dimensions of shooting angle and image clarity, which can ensure the quality of the associated monitoring video in the digital twin model. In addition, by setting the optimal monitoring video data as the main perspective content and the remaining videos as auxiliary perspective content to the digital twin model, it is helpful to build a multi-angle visual display view and enhance the spatial restoration capability of the event scene in the target monitoring area.

[0058] Furthermore, the optimal monitoring video data from multiple target monitoring video data sets can be selected based on shooting angle and image clarity. This can be accomplished through the following technical steps: obtaining the spatial location and observation direction information of the monitoring device corresponding to each target monitoring video data set, and using this spatial location and observation direction information to generate the monitoring view area for each monitoring device. A spatial overlap analysis is performed between the spatial extent of the mudflat area in the digital twin model and the monitoring view area, and the degree of field of view coverage of the mudflat area by each monitoring device is calculated. The imaging position of the person image in each target monitoring video data set is extracted, and based on the relative relationship between the person's orientation and the camera position, an orientation angle score is calculated. The occlusion ratio within the target area is also determined. Image clarity indicators are determined for each target monitoring video data set based on image quality parameters such as edge clarity, texture integrity, and noise distribution. A weighted fusion is performed based on the field of view coverage, orientation angle score, occlusion ratio, and image clarity indicator to generate a unified scoring index. Based on this unified scoring index, the video data with the highest score from the multiple target monitoring video data sets is selected as the optimal monitoring video data set.

[0059] In some embodiments, the relative position of the person in the tidal flat area is determined based on the tidal flat area image and the corresponding person image, which specifically includes the following technical steps: obtaining the facial image coordinates in the person image, and converting the facial image coordinates into three-dimensional space coordinates; determining the tidal flat area range of the tidal flat area image in the digital twin model, and constructing multiple risk segments with the center point of the tidal flat area range as a reference; mapping the three-dimensional space coordinates to the tidal flat area range to determine the risk segment corresponding to the person image, and determining the risk level corresponding to the person based on the risk segment.

[0060] The facial image coordinates may represent two-dimensional image coordinate information used to characterize the facial location in a person's image. For example, the multiple risk segments may be multi-layered annular risk segments, which may represent multiple concentric ring-shaped spatial regions divided sequentially according to preset radius increments, with the center point of the tidal flat area as the reference point. Each layer of risk segments has an independent risk level identifier, which is used to grade the location of people entering the tidal flat area. The multi-layered annular risk segments may include a first risk segment, a second risk segment, and a third risk segment. The first risk segment may represent the high-risk area closest to the center point within the tidal flat area and is used to identify people in dangerous areas. The second risk segment may represent a medium-risk area, covering the annular area extending from the first risk segment to the mid-range range, and is used to cover the edge of the tidal flat area. The third risk segment may represent a relatively safe edge area and is used to mark target people who have not yet entered the tidal flat area but are showing signs of entering.

[0061] Of course, in other embodiments of the present disclosure, the above-mentioned multiple risk sections can also be rectangular or polygonal areas arranged in a strip-like manner along the mudflat boundary direction. The area can be divided based on the shortest distance relationship between the water boundary line and the outer contour of the mudflat area, forming multiple parallel levels expanding from the inside to the outside, each level corresponding to a risk level, the inner area close to the water edge is set as a high-risk section, the middle area is set as a medium-risk section, and the outer area far from the water edge is set as a low-risk section.

[0062] After determining the risk segment corresponding to the personnel image, the risk level can be determined according to the corresponding risk segment, and the corresponding warning information can be generated according to the risk level. For example, in response to the personnel image being in the first risk segment and the corresponding risk level being level one, a red emergency warning information is generated, and warning measures such as audio and video alarms, pop-up prompts, and SMS notifications are simultaneously triggered to quickly respond to behaviors in high-risk areas. In response to the personnel image being in the second risk segment and the corresponding risk level being level two, an orange warning information is generated, and a manual review prompt is pushed to the monitoring terminal to strengthen the patrol response to the medium-risk area. In response to the personnel image being in the third risk segment and the corresponding risk level being level three, a yellow prompt information is generated, and the target personnel position and movement trajectory are highlighted in the digital twin model interface to indicate that there is a potential trend of approaching the mudflat area.

[0063] In some embodiments, electronic fences can also be used to obtain images of the mudflat area, which specifically includes the following technical steps: presetting an electronic fence area in the monitoring area; in response to identifying a target person image in the monitoring video data, obtaining the person position coordinates corresponding to the target person image; in response to the person position coordinates belonging to the electronic fence area, obtaining an image of the mudflat area after the person enters the electronic fence area from the monitoring video data.

[0064] The electronic fence area can represent a two-dimensional or three-dimensional virtual space range preset within the monitoring area based on geographic boundaries or spatial constraints, and is used to determine whether the target person has entered the designated risk area. The electronic fence area can be configured by setting spatial boundary lines or surfaces in the digital twin model, and the boundary parameters can be adjusted according to actual conditions. The person position coordinates can represent the spatial position information associated with the target person image, used to represent the current position of the target person in the digital twin model. The person position coordinates can be converted from the coordinates of the person's footsteps image in the video image through geometric mapping or perspective backprojection.

[0065] Furthermore, an electronic fence area can be set up using the following technical steps: Using a digital twin model, obtain boundary data for the tidal flat area within the monitored area. This boundary data may include a set of multiple three-dimensional coordinate points representing the outline of the tidal flat area. Connect these multiple three-dimensional coordinate points to form a closed boundary line, which is used to define the outer boundary of the electronic fence. The closed boundary line is converted into spatial area data consistent with the digital twin model using a unified coordinate projection system, generating an electronic fence area corresponding to the tidal flat area. This electronic fence area is a closed three-dimensional area.

[0066] In some embodiments, the movement trajectory of people can also be predicted, so as to provide early warning information before people enter the tidal flat area. This can be done specifically through the following technical steps: in response to the presence of people in the monitoring video data, a monitoring image sequence containing people is extracted from the monitoring video data; the historical movement trajectory of the people is constructed based on the monitoring image sequence; the predicted movement trajectory of the people is generated based on the historical movement trajectory; and in response to the existence of an intersection between the predicted movement trajectory and the tidal flat area, early warning information is generated.

[0067] The monitoring image sequence can represent a collection of image frames containing images of the target person, extracted continuously in chronological order from the monitoring video data. This image sequence is used to characterize the target person's movement status within a certain time range. The historical motion trajectory can represent the path information formed by connecting the target person's position coordinates extracted from each image frame in the monitoring image sequence in chronological order. This path information is used to describe the person's actual movement trajectory near the mudflat area. The predicted motion trajectory can represent a continuous path generated by a trajectory prediction algorithm based on the motion features extracted from the historical motion trajectory. It is used to characterize the target person's possible movement trajectory at a future moment.

[0068] Specifically, constructing a person's historical motion trajectory based on a surveillance image sequence can be accomplished through the following technical steps: extracting time information corresponding to each frame in the surveillance image sequence to identify the temporal order of the frames; performing target detection on each frame to extract the target person's image position information and convert the image position information into corresponding image coordinates; performing spatial backprojection on the image coordinates based on the spatial position information and observation direction information of the monitoring device to obtain the corresponding three-dimensional position coordinates of the target person in three-dimensional space; arranging the three-dimensional position coordinates in a temporal order according to the time information, and connecting consecutive coordinate points to construct a historical motion trajectory corresponding to the target person's movement process; generating a predicted motion trajectory based on the historical motion trajectory can be accomplished through the following technical steps: extracting the person's position coordinates at multiple consecutive moments in the historical motion trajectory, calculating the displacement vectors and corresponding time intervals between adjacent position coordinates; calculating the person's instantaneous motion velocity at each consecutive moment based on the displacement vectors and time intervals, and further calculating the velocity change between adjacent instantaneous motion velocities; determining the acceleration direction and acceleration amplitude of the person's motion based on the velocity change, and using the acceleration direction and acceleration amplitude to update the person's current motion state parameters. Based on the updated motion state parameters, the predicted position coordinates for the next moment are generated through extrapolation. This extrapolation process is then iterated to generate continuous predicted position coordinates for multiple subsequent moments. These continuous predicted position coordinates are connected in chronological order to construct a predicted motion trajectory corresponding to the person's future movement path.

[0069] It should be noted that although the steps of the method disclosed herein are depicted in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in that particular order, or that all steps must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one, and / or one step may be decomposed into multiple steps.

[0070] In the embodiment of the present disclosure, Figure 3The illustrated digital twin-based hydropower station personnel safety early warning system architecture implements the early warning method described in the aforementioned embodiments. As shown, a first and second tidal flat pedestrian monitoring network camera are located in monitoring areas to collect video data from each monitoring area of the hydropower station. The first and second tidal flat pedestrian monitoring network cameras are connected to an industrial router via a switch and transmit the collected video data to a front-end intelligent analyzer. The front-end intelligent analyzer, acting as an edge node, receives the monitoring video data from the tidal flat pedestrian monitoring network cameras and uses a machine learning algorithm to identify images of the tidal flat area and those entering the area. The recognition results are synchronously transmitted to a video image storage module, which stores preset video clips corresponding to the person images. The video image storage module transmits the image data to the tidal flat pedestrian intelligent recognition management platform. The tidal flat pedestrian intelligent recognition management platform determines the location coordinates corresponding to the person image and, based on the spatial coordinate range to which the location coordinates fall, further identifies the target monitoring area corresponding to the person image. The preset video clip is then added to the location corresponding to the target monitoring area in the digital twin model.

[0071] The front-end intelligent analyzer can also access the tidal flat pedestrian intelligent identification and management platform in real time via the intranet's general routing encapsulation protocol. Furthermore, the platform can determine the relative position of individuals within the tidal flat area based on images of the tidal flat area and corresponding images of individuals, and generate corresponding warning information based on these relative positions. User terminals can access the platform to obtain relevant warning data, and all processed data is archived and managed in a database, supporting full-process monitoring of human behavior and visual warning displays.

[0072] In addition, in the embodiment of the present disclosure, a hydropower station personnel safety warning system based on digital twin is also provided. Figure 4 As shown, the digital twin-based hydropower station personnel safety warning system 400 may include: a twin model construction module 410, an image recognition module 420, a target area determination module 430, a video mapping module 440, and a warning information generation module 450. Among them: The twin model construction module 410 can be used to construct a digital twin model based on the building information data and geographic space data of the hydropower station, and mark the spatial coordinate range of each monitoring area in the digital twin model.

[0073] The image recognition module 420 can be used to collect monitoring video data of each monitoring area, and use a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data.

[0074] The target area determination module 430 may be configured to determine the position coordinates corresponding to the person image, and determine the target monitoring area corresponding to the person image according to the spatial coordinate range to which the position coordinates belong.

[0075] The video mapping module 440 can be used to mount the preset video clip corresponding to the personnel image to the position corresponding to the target monitoring area in the digital twin model and perform visual display.

[0076] The warning information generation module 450 can be used to determine the relative position of personnel in the tidal flat area based on the tidal flat area image and the corresponding personnel image, generate corresponding warning information based on the relative position, and push the warning information in various ways.

[0077] The specific details of each module in the above digital twin-based hydropower station personnel safety warning system have been described in detail in the corresponding digital twin-based hydropower station personnel safety warning method, so they will not be repeated here.

[0078] It should be noted that although the detailed description above mentions several modules or units of the digital twin-based hydropower station personnel safety warning system, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in a single module or unit. Conversely, the features and functions of a single module or unit described above can be further divided and embodied by multiple modules or units.

[0079] In addition, in an exemplary embodiment of the present disclosure, an electronic device is also provided that can implement the above-mentioned digital twin-based hydropower station personnel safety warning method.

[0080] Those skilled in the art will appreciate that various aspects of the present disclosure may be implemented as systems, methods, or program products. Therefore, various aspects of the present disclosure may be implemented in the following forms: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which may be collectively referred to herein as a "circuit," "module," or "system."

[0081] Refer to the following Figure 5 hereinafter, an electronic device 500 according to an embodiment of the present disclosure is described. Figure 5 The electronic device 500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0082] like Figure 5As shown, electronic device 500 is implemented as a general-purpose computing device. Components of electronic device 500 may include, but are not limited to, the aforementioned at least one processing unit 510, the aforementioned at least one storage unit 520, a bus 530 connecting various system components (including storage unit 520 and processing unit 510), and a display unit 540.

[0083] The storage unit stores program code, which can be executed by the processing unit 510, causing the processing unit 510 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of the present disclosure. The storage unit 520 may include a readable medium in the form of a volatile memory unit, such as a random access memory unit (RAM) 521 and / or a cache memory unit 522, and may further include a read-only memory unit (ROM) 523.

[0084] The storage unit 520 may also include a program / utility 524 having a set (at least one) of program modules 525, such program modules 525 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0085] Bus 530 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0086] The electronic device 500 can also communicate with one or more external devices 570 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with the electronic device 500, and / or any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., a router, modem, etc.). This communication can occur via an input / output (I / O) interface 550. Furthermore, the electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 560. As shown, the network adapter 560 communicates with other modules of the electronic device 500 via a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the electronic device 500, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0087] Through the description of the above embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by combining software with necessary hardware.

[0088] In exemplary embodiments of the present disclosure, a computer-readable storage medium is also provided, on which is stored a program product capable of implementing the aforementioned methods of this specification. In some possible embodiments, various aspects of the present disclosure may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Methods" section of this specification.

[0089] refer to Figure 6 As shown, a program product 600 for implementing the aforementioned digital twin-based hydropower station personnel safety warning method according to an embodiment of the present disclosure is described. This program product 600 can be implemented in a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0090] The program product may utilize any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0091] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0092] Furthermore, the figures above are merely illustrative of the processes included in the methods according to exemplary embodiments of the present disclosure and are not intended to be limiting. It is readily understood that the processes illustrated in the figures above do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0093] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A hydropower station personnel safety early warning method based on digital twins, characterized in that: include: Constructing a digital twin model based on the hydropower station's architectural information data and geographic spatial data, and marking the spatial coordinate range of each monitoring area in the digital twin model; Collecting monitoring video data of each of the monitoring areas, and using a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data; Determining the position coordinates corresponding to the person image, and determining the target monitoring area corresponding to the person image based on the spatial coordinate range to which the position coordinates belong; Mounting the preset video clip corresponding to the personnel image to the position corresponding to the target monitoring area in the digital twin model and visually displaying it; According to the tidal flat area image and the corresponding personnel image, the relative position of the personnel in the tidal flat area is determined, corresponding warning information is generated according to the relative position, and the warning information is pushed in a variety of ways.

2. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: The digital twin model is constructed based on the building information data and geographic space data of the hydropower station, including: Collecting architectural information data and geographic spatial data of the hydropower station, wherein the architectural information data includes three-dimensional component information of the powerhouse structure, hydraulic structures, and tidal flat areas, and the geographic spatial data includes terrain elevation data and remote sensing images; Constructing a BIM structural model based on the building information data, and constructing a GIS basic scene based on the geographic space data; A unified coordinate projection system is used to spatially register the BIM structural model with the GIS basic scene, and the registered BIM structural model and the GIS basic scene are data-fused to generate a three-dimensional model dataset containing the correspondence between structural information and geographic information; Import the three-dimensional model data set into the simulation engine environment to construct the digital twin model.

3. The hydropower station personnel safety early warning method based on digital twin according to claim 2 is characterized in that: After constructing the digital twin model, the method further includes: Acquiring spatial position information and observation direction information of a monitoring device set in the monitoring area, wherein the spatial position information includes the positioning coordinates of the monitoring device, and the observation direction information includes the pitch angle and the horizontal viewing angle; Marking a video mounting point corresponding to the monitoring device in the digital twin model according to the spatial position information and the observation direction information; Constructing a mapping data table based on the correspondence between the video mounting points and the monitoring devices, and loading the mapping data table into the simulation engine environment; In the simulation engine environment, the monitoring video data is rendered to the corresponding video mounting point position in the digital twin model according to the mapping data table.

4. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: The method of using a machine learning algorithm to identify images of a tidal flat area and images of people entering the tidal flat area from the monitoring video data includes: The monitoring video data is transmitted to edge nodes using a 5G network, each edge node having a corresponding monitoring device; The edge node uses a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data.

5. The hydropower station personnel safety early warning method based on digital twin according to claim 4 is characterized in that: The monitoring video data includes visible light video data and infrared video data. The edge node uses a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data, including: In response to the monitored video data being visible light video data, the edge node performs a reflection suppression operation on the visible light video data to obtain visible light video optimization data; Using a semantic segmentation network to perform a region recognition operation on the visible light video optimization data to obtain an image of the tidal flat area and an image of a person entering the tidal flat area; In response to the monitoring video data being infrared video data, the edge node inputs the infrared video data into a thermal feature recognition network to obtain an image of the tidal flat area and an image of a person entering the tidal flat area.

6. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: The step of mounting the preset video clip corresponding to the personnel image to a position corresponding to the target monitoring area in the digital twin model includes: In response to the personnel image corresponding to a plurality of target monitoring video data, wherein the plurality of target monitoring video data cover the same target monitoring area; Selecting optimal monitoring video data from the plurality of target monitoring video data based on shooting angle and image clarity; The optimal monitoring video data is used as the main perspective content, and the remaining target monitoring video data is used as the auxiliary perspective content, and they are uniformly mounted to the position corresponding to the target monitoring area in the digital twin model.

7. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: The determining the relative position of the person in the tidal flat area according to the tidal flat area image and the corresponding person image includes: Obtaining facial image coordinates in the person image, and converting the facial image coordinates into three-dimensional space coordinates; Determine the tidal flat area range of the tidal flat area image in the digital twin model, and construct a plurality of risk segments with the center point of the tidal flat area range as a reference; The three-dimensional space coordinates are mapped to the tidal flat area to determine the risk segment corresponding to the person image, and the risk level corresponding to the person is determined according to the risk segment.

8. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: The process of acquiring the tidal flat area image further includes: Presetting an electronic fence area in the monitoring area; In response to identifying a target person image in the monitoring video data, obtaining person position coordinates corresponding to the target person image; In response to the person's position coordinates belonging to the electronic fence area, an image of the mudflat area after the person enters the electronic fence area is obtained from the monitoring video data.

9. The hydropower station personnel safety early warning method based on digital twin according to claim 1 is characterized in that: Also includes: In response to a person existing in the monitoring video data, extracting a monitoring image sequence containing the person from the monitoring video data; constructing a historical movement trajectory of the person based on the monitoring image sequence; generating a predicted movement trajectory of the person based on the historical movement trajectory; In response to the predicted motion trajectory intersecting with the tidal flat area, early warning information is generated.

10. A hydropower station personnel safety early warning system based on digital twins, used to implement the hydropower station personnel safety early warning method based on digital twins as claimed in any one of claims 1 to 9, characterized in that: The system comprises: A twin model construction module is used to construct a digital twin model based on the building information data and geographic spatial data of the hydropower station, and to mark the spatial coordinate range of each monitoring area in the digital twin model; An image recognition module is used to collect monitoring video data of each monitoring area and use a machine learning algorithm to identify images of the tidal flat area and images of people entering the tidal flat area from the monitoring video data; a target area determination module, configured to determine the position coordinates corresponding to the person image, and determine the target monitoring area corresponding to the person image based on the spatial coordinate range to which the position coordinates belong; A video mapping module is used to mount the preset video clip corresponding to the person image to the position corresponding to the target monitoring area in the digital twin model and perform visual display; The warning information generation module is used to determine the relative position of the personnel in the tidal flat area based on the tidal flat area image and the corresponding personnel image, generate corresponding warning information based on the relative position, and push the warning information in various ways.

Images