Intelligent early warning method for hydropower station based on fusion of PLC sensor data and image recognition

Abstract

This invention relates to the field of intelligent monitoring and early warning technology for industrial safety, and discloses an intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition. The method includes: collecting real-time water level data output by PLC sensors within the hydropower station powerhouse and video image data collected by monitoring cameras; establishing a unified time series based on the collection timestamps; performing flooding scene recognition on the video image data to obtain structural area recognition results; calculating water level change data and establishing a correspondence between it and water body change data in the structural areas; performing fusion judgment processing on the two types of data to generate a comprehensive risk judgment result; and executing alarm linkage control and recording event data based on the judgment result. This invention improves data consistency in the flooding risk judgment process by constructing synchronous PLC sensor data and video image data containing structural area identifiers, and establishing a correspondence between water level change data and water body change data in the structural areas.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring and early warning technology for industrial safety, and in particular to an intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition. Background Technology

[0002] Hydropower station powerhouses typically house generator sets, water supply and drainage systems, and electrical equipment. The powerhouse also contains underground structural spaces and various pipe connections. When external water inflow changes, the drainage system malfunctions, or leaks occur in structural components, water accumulation or a rise in water level can easily occur inside the powerhouse.

[0003] In existing technologies, flood monitoring of hydropower plant buildings typically employs PLC sensors or video surveillance systems for status detection. One common technical solution involves using PLC water level sensors to detect water levels in sump pits or low-lying areas of the plant, triggering alarm signals based on changes in water level measurements. Another technical solution involves using monitoring cameras to capture video images of the plant site and utilizing image analysis to identify surface water accumulation or leakage.

[0004] However, the inventors of this application discovered in the process of implementing the relevant technical solutions that the above-mentioned prior art has at least the following technical problems: In the existing hydropower plant flood monitoring technology, there is a lack of unified time and structural area correspondence between PLC water level detection data and video image recognition results, which makes it impossible to correlate the overall water level change information with the water body change status of local structural areas, thereby affecting the data consistency of the flood risk determination process. Summary of the Invention

[0005] To overcome the above shortcomings, this invention provides an intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition. It aims to improve the problem that in the existing hydropower plant flood monitoring technology, there is a lack of unified time and structural region correspondence between PLC water level detection data and video image recognition results, which affects the data consistency of the flood risk assessment process.

[0006] This invention provides the following technical solution: a smart early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition, comprising the following steps: S1. Collect real-time water level data output by PLC sensors inside the hydropower station powerhouse, and collect video image data output by the monitoring camera equipment in the powerhouse. S2. Establish a unified time series based on the acquisition timestamp of PLC sensor data and the frame timestamp of video image data, and generate synchronous PLC sensor data corresponding to the video image frames. S3. Obtain the structural area information of the powerhouse based on the building information model of the hydropower station, and map the structural area information of the powerhouse to the video image data to generate video image data containing structural area identifiers; S4. Input the video image data containing the structural region identifier into the flood scene multi-task recognition model to obtain the water accumulation area recognition result, leakage location recognition result, water flow pattern recognition result and water body state recognition result corresponding to the structural region. S5. Based on the structural region identification results at continuous time intervals, perform regional change tracking processing to generate structural region water body change data; S6. Calculate water level change data based on the synchronous PLC sensor data, and establish the correspondence between the water level change data and the water body change data in the structural area; S7. Based on the correspondence, perform fusion judgment processing on the water level change data and the water body change data of the structural area to generate a comprehensive risk judgment result; S8. Generate early warning control instructions based on the comprehensive risk assessment results, and execute alarm linkage control and event data recording.

[0007] Preferably, in step S2, the step of establishing a unified time series based on the acquisition timestamp of the PLC sensor data and the frame timestamp of the video image data includes: The acquisition timestamp sequence of PLC sensor data and the frame timestamp sequence of video image data are obtained respectively, and the PLC sensor data is sorted according to the acquisition time order. Construct a unified time reference sequence based on the frame timestamps of video image data; The time interval of the PLC sensor data is made consistent according to the unified time reference sequence, and the PLC sensor data with time missing intervals is compensated. Based on a unified time reference sequence, PLC sensor data within the time range corresponding to each video image frame is selected to generate synchronous PLC sensor data corresponding to the video image frame.

[0008] Preferably, in step S3, the step of generating video image data containing structural region identifiers includes: Read the three-dimensional structural data of the powerhouse in the building information model of the hydropower station, and divide the internal area of ​​the powerhouse into structural areas and number the areas according to the structural characteristics of the powerhouse. Obtain the installation location parameters and field-of-view parameters of the surveillance camera equipment; Establish a spatial mapping relationship between the field of view of the surveillance camera equipment and the structural area of ​​the factory building; Project the factory building structure area onto the video image coordinate space; The corresponding pixel regions in the video image are processed by writing structural region identifiers to form video image data containing structural region identifiers.

[0009] Preferably, in step S4, the step of inputting the video image data containing structural region identifiers into the flood scene multi-task recognition model includes: Perform image preprocessing operations on video image data containing structural region identifiers; Region segmentation of video images based on structural region identifiers; Extract image feature data for each structural region separately; Within each structural area, processes such as water accumulation area segmentation and identification, leakage location detection, water flow pattern classification, and water body state determination are performed. The identification results of each structural region are then associated and stored according to the structural region number to generate structural region identification results.

[0010] Preferably, in step S5, the step of performing region change tracking processing based on the structural region identification results at consecutive time points includes: Obtain the structural region identification results at consecutive time points in chronological order; Under the same structural region numbering, determine the recognition region corresponding to consecutive time periods; Perform position matching and boundary tracking processing on the identified region; Record the area and location changes of the identified region over consecutive time periods; Based on the area change information and location change information, water body change data for the structural region is generated.

[0011] Preferably, in step S6, the step of calculating the water level change data based on the synchronous PLC sensor data includes: Read synchronous PLC sensor data in chronological order; Establish a continuous time-series buffer sequence of water level data; Perform continuous sampling and verification processing on the water level data; Calculate the water level change between adjacent sampling times; A water level change data sequence is formed based on the water level changes over a continuous period of time.

[0012] Preferably, in step S6, the step of establishing the correspondence between the water level change data and the water body change data of the structural area includes: Determine the range of the factory building structure area corresponding to the water level change data based on the structural area identifier; The water level change data is matched to different regions according to the structural region number; The water body change data of the structural region is correlated with the water level change data of the corresponding region. And establish a regional correspondence between water level change data and water body change data in structural areas.

[0013] Preferably, in step S7, the step of performing fusion determination processing on the water level change data and the water body change data of the structural area based on the correspondence includes: Read the region correspondence; Obtain the PLC-side risk assessment data corresponding to water level change data and the visual-side risk assessment data corresponding to water body change data in the structural area, respectively. The risk assessment data from the PLC side and the risk assessment data from the vision side are weighted and calculated according to the preset fusion weights; and a comprehensive risk assessment result is generated based on the calculation results.

[0014] Preferably, in step S8, the step of executing alarm linkage control includes: Early warning level information is generated based on the comprehensive risk assessment results; Generate corresponding control commands based on the warning level information; The control commands are sent to the audible and visual alarm devices, the plant drainage control devices, and the remote monitoring terminal; and the execution status information of the control commands is recorded.

[0015] Preferably, in step S8, the event data recording step includes: Record the synchronous PLC sensor data corresponding to the comprehensive risk assessment results; Record video image data containing structural region identifiers, record structural region identification results and structural region water body change data, and establish a unified event identifier for the above data for associated storage.

[0016] The present invention has the following beneficial effects: 1. This invention constructs synchronous PLC sensor data and video image data containing structural area identifiers, and establishes a correspondence between water level change data and water body change data in structural areas. This enables unified correlation analysis of overall water level information and local structural area change information in the plant, thereby improving the data consistency of the flood risk assessment process by constructing synchronous PLC sensor data and video image data containing structural area identifiers, and establishing a correspondence between water level change data and structural area water body change data.

[0017] 2. This invention divides the internal structure of the powerhouse into structural regions based on the building information model of the hydropower station and maps these structural regions to the coordinate space of video images. Water accumulation area identification and regional change tracking are performed in each structural region, so that the image recognition results form a definite correspondence with the actual structural position of the powerhouse. This enables the water change analysis process to be carried out within a specific structural region, avoiding data mixing between different monitoring areas and improving the accuracy of regional status determination.

[0018] 3. This invention generates early warning control instructions based on the comprehensive risk assessment results, and establishes a unified event identifier to link and store the control execution process with synchronous PLC sensor data, structural area identification results, and water body change data. This enables early warning triggering, equipment linkage control, and operation data recording to form a continuous processing flow, thereby realizing data association between early warning results and on-site control behavior and improving the traceability of the hydropower station's operating status. Attached Figure Description

[0019] Figure 1 This is a flowchart of the intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition proposed in this invention. Detailed Implementation

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 Reference Figure 1 This invention provides an intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition, comprising the following steps: S1. Collect real-time water level data output by PLC sensors inside the hydropower station powerhouse, and collect video image data output by the monitoring camera equipment in the powerhouse. S2. Establish a unified time series based on the acquisition timestamp of PLC sensor data and the frame timestamp of video image data, and generate synchronous PLC sensor data corresponding to the video image frames. S3. Obtain the structural area information of the powerhouse based on the building information model of the hydropower station, and map the structural area information of the powerhouse to the video image data to generate video image data containing structural area identifiers; S4. Input the video image data containing the structural region identifier into the flood scene multi-task recognition model to obtain the water accumulation area recognition result, leakage location recognition result, water flow pattern recognition result and water body state recognition result corresponding to the structural region. S5. Based on the structural region identification results at continuous time intervals, perform regional change tracking processing to generate structural region water body change data; S6. Calculate water level change data based on synchronous PLC sensor data, and establish the correspondence between water level change data and water body change data in the structural area; S7. Based on the correspondence, perform fusion judgment processing on the water level change data and the water body change data of the structural area to generate a comprehensive risk judgment result; S8. Generate early warning control instructions based on the comprehensive risk assessment results, and execute alarm linkage control and event data recording.

[0022] Specifically, in step S1, the PLC sensor detects the water level in the plant according to a set sampling period and continuously outputs water level measurement data; the monitoring camera acquires video images of the plant site according to a predetermined acquisition frame rate and forms continuous video image frame data. The water level data and video image data each carry corresponding acquisition time information for subsequent time matching processing. In step S2, the acquisition timestamp sequence of the PLC sensor data and the frame timestamp sequence of the video image data are processed uniformly. First, the PLC sensor data is arranged in chronological order. Then, using the video image frame time as a unified time reference, time matching processing is performed on the PLC sensor data, ensuring that each video image frame corresponds to the PLC water level data at the same time, thereby forming a synchronized PLC sensor data sequence.

[0023] In step S3, the structural information of the powerhouse in the hydropower station building information model is read, the internal space of the powerhouse is divided into structural regions, and a unique region number is assigned to each structural region. Based on the installation location and field of view of the monitoring camera equipment, a spatial correspondence is established between the structural regions and the pixel regions in the video image, and structural region identification information is written into the video image data, so that each pixel region in the video image corresponds to a specific powerhouse structural region. In step S4, the video image data containing the structural region identification is input into the flood scene multi-task recognition model, and the image data within each structural region is processed separately. By extracting image feature information, the identification results of water accumulation area, leakage location, water flow pattern, and water state are obtained within the corresponding structural region, and the identification results are recorded according to the structural region number.

[0024] In step S5, the structural region identification results at consecutive time points are acquired in chronological order. Matching processing is performed on the identified regions at consecutive time points under the same structural region number. The boundary positions of the identified regions are tracked, and the range changes of the identified regions at consecutive time points are recorded, thereby forming structural region water change data reflecting the water body change state within the structural region. In step S6, synchronous PLC sensor data is read in chronological order, and the water level data at consecutive time points are compared and processed to obtain water level change data. Subsequently, based on the structural region number, the water level change data is associated with the water body change data of the corresponding structural region, so that the water level change information and the structural region change information form a corresponding relationship data.

[0025] In step S7, based on the established correspondence data, the water level change data and the water body change data in the structural area are jointly processed to obtain a comprehensive risk assessment result representing the plant's operating status. In step S8, an early warning control command is generated based on the comprehensive risk assessment result and sent to the alarm equipment and plant control equipment to execute linkage control. Simultaneously, the synchronous PLC sensor data, video image data, structural area identification results, water body change data, and comprehensive risk assessment result at the time of triggering the early warning are correlated and recorded to form a complete event data record.

[0026] Through the above processing flow, the correlation analysis of PLC sensor water level data and video image recognition results under the conditions of unified time series and unified structural area was realized, and a continuous processing flow from data acquisition, status recognition, change analysis, fusion judgment to early warning control was formed.

[0027] Furthermore, in step S2, the step of establishing a unified time series based on the acquisition timestamp of the PLC sensor data and the frame timestamp of the video image data includes: The acquisition timestamp sequence of PLC sensor data and the frame timestamp sequence of video image data are obtained respectively, and the PLC sensor data is sorted according to the acquisition time order. Construct a unified time reference sequence based on the frame timestamps of video image data; The time interval of PLC sensor data is made consistent according to a unified time reference sequence, and compensation is performed on PLC sensor data with missing time intervals. Based on a unified time reference sequence, PLC sensor data within the time range corresponding to each video image frame is selected to generate synchronous PLC sensor data corresponding to the video image frame.

[0028] Specifically, during the PLC sensor data acquisition process, each PLC sensor outputs water level measurement values ​​according to a fixed sampling period and records the corresponding acquisition time when the data is output, forming a PLC sensor timestamp sequence. During the acquisition process by the monitoring camera equipment, video image data is continuously generated into image frames according to the video frame rate, and the frame acquisition time is recorded synchronously, thus forming a video image frame timestamp sequence.

[0029] Let the timestamp sequence acquired by the PLC sensor be represented as: ;in, This represents the i-th PLC sampling time; m represents the number of PLC sampling data. The video image frame timestamp sequence is represented as: ;in, The j-th video frame was captured at a specific time; n represents the number of video frames.

[0030] Because the sampling period of the PLC sensor differs from the video image acquisition period, the video image frame timestamps need to be used as a unified time reference sequence for time alignment of the PLC sensor data. First, the PLC sensor data is sorted in ascending order according to the acquisition time, forming a continuous time series. During the construction of the unified time reference sequence, the video image frame timestamp sequence is used as the standard time axis, and the PLC sensor data is mapped onto this time axis. When the video image frame time falls between two adjacent PLC sampling times, time interval consistency processing is performed on the PLC sensor data, enabling the PLC data to establish a correspondence with the video image frames.

[0031] Synchronous PLC sensor data is obtained through time interpolation, and the calculation process is as follows: ; in, This represents the synchronized water level data corresponding to the j-th frame of the video image; and These represent the water level measurements at two adjacent PLC sampling times; Indicates the time of video image frame acquisition; , This indicates the PLC sampling time before or after the video frame.

[0032] When PLC sensor data contains sampling gaps within a unified time series, continuity verification is performed on adjacent valid sampled data, and compensation calculations are executed based on time intervals to maintain the continuity of the time series, thereby preventing time discontinuities from affecting subsequent calculations. After completing the time consistency processing, a synchronized PLC sensor data sequence is established: The sequence length is consistent with the number of video image frames, ensuring that each video image frame corresponds to a unique synchronized water level data.

[0033] Through the above processing, the original PLC sensor data and video image data are unified under the same time reference, so that data with different sampling frequencies form a frame-by-frame correspondence. This ensures that the subsequent structural area identification results and water level change data are correlated and calculated under the same time conditions, thus providing a consistent data input basis for subsequent area change analysis and fusion judgment.

[0034] Furthermore, in step S3, the step of generating video image data containing structural region identifiers includes: Read the three-dimensional structural data of the powerhouse in the building information model of the hydropower station, and divide the internal area of ​​the powerhouse into structural areas and number the areas according to the structural characteristics of the powerhouse. Obtain the installation location parameters and field-of-view parameters of the surveillance camera equipment; Establish a spatial mapping relationship between the field of view of the surveillance camera equipment and the structural area of ​​the factory building; Project the factory building structure area onto the video image coordinate space; The corresponding pixel regions in the video image are processed by writing structural region identifiers to form video image data containing structural region identifiers.

[0035] Specifically, this step first reads the 3D structural data of the powerhouse stored in the building information model of the hydropower station. The 3D structural data includes the spatial location parameters of the powerhouse wall structure, equipment foundation structure, drainage area structure, pipeline connection area, and ground area. Based on the spatial continuity and functional distribution of the powerhouse structure, the internal space of the powerhouse is divided into regions, and a unique structural region number is assigned to each structural region, thus forming a set of structural regions. Let the set of powerhouse structural regions be represented as: ;in, This represents the k-th structural region; k represents the total number of structural regions.

[0036] After completing the structural area division, the installation location and attitude parameters of the monitoring camera equipment are obtained. The installation location parameters include the spatial coordinates of the camera equipment in the factory coordinate system. The attitude parameters include the horizontal rotation angle, pitch angle, and imaging direction parameters of the camera equipment. The field of view of the camera equipment is determined by combining the focal length and imaging size parameters. To establish a correspondence between the three-dimensional structural area and the two-dimensional video image, a mapping relationship between the factory spatial coordinate system and the video image coordinate system needs to be established. First, the spatial points in the factory structural area are represented as three-dimensional coordinates: ;in, , , This represents the three-dimensional coordinates of any point in the structural region within the factory's spatial coordinate system.

[0037] The imaging model of the camera device converts three-dimensional spatial coordinates into video image coordinates, and its projection relationship is expressed as follows: ; Where (u,v) represents the pixel coordinates of the projected image; s represents the scaling factor; K represents the camera device's internal parameter matrix, used to describe the focal length and image center position; R represents the spatial rotation matrix, used to describe the orientation of the camera device; T represents the spatial translation vector, used to describe the installation location of the camera equipment.

[0038] Through the above spatial mapping calculation, the boundary range of the factory structure area in three-dimensional space is transformed into the pixel space of the video image, thereby determining the pixel coverage range of each structure area in the video image.

[0039] After projection processing, the pixel regions in the video image are assigned to specific regions. When a pixel's coordinates fall within the projection range of a given structural region, the pixel is assigned the corresponding structural region number. The structural region identification function is expressed as follows: ;in, This indicates the structural region number corresponding to the pixel position (u,v); This indicates the corresponding factory building structure area.

[0040] Through the above processing, area identification information consistent with the factory building structure is formed in the video image data, giving each structural area in the video image a definite spatial origin. After the structural area identification is written, the video image data not only contains image grayscale or color information, but also structural area number information, thus generating video image data containing structural area identification.

[0041] Through the above spatial mapping and region identification processing, the recognition results in the video image can establish a definite correspondence with the actual factory structure location, realizing a unified expression between the image recognition results and the physical structure of the factory, and providing a unified spatial benchmark for subsequent structural region change tracking and multi-source data fusion judgment.

[0042] Furthermore, in step S4, the step of inputting video image data containing structural region identifiers into the multi-task recognition model for flooded scenes includes: Perform image preprocessing operations on video image data containing structural region identifiers; Region segmentation of video images based on structural region identifiers; Extract image feature data for each structural region separately; Within each structural area, processes such as water accumulation area segmentation and identification, leakage location detection, water flow pattern classification, and water body state determination are performed. The identification results of each structural region are then associated and stored according to the structural region number to generate structural region identification results.

[0043] Specifically, this step first performs image preprocessing on the video image data containing structural region identifiers. Image preprocessing includes image size unification, brightness normalization, and noise suppression to ensure the input image data meets uniform data format requirements. Let the input video image frame be represented as: ;in, The pixel coordinates at time t are: The image's grayscale or color values.

[0044] The preprocessed image is obtained after normalization: ; in, This represents the normalized pixel value; This represents the minimum pixel value in the image. This represents the maximum pixel value of the image.

[0045] After image preprocessing, the video image is divided into regions based on the structural region identifier function generated in step S3. For any structural region number... Extract the corresponding pixel set to form the image data of the structural region: ; in, This represents the image data corresponding to the k-th structural region; This indicates the structural region number to which the pixel belongs.

[0046] Subsequently, image feature extraction was performed separately within each structural region. Feature extraction included texture feature extraction, edge information extraction, and statistical processing of regional brightness distribution, used to characterize the surface state and boundary changes of the water body within the region. The feature extraction results are expressed as follows: ;in, Φ represents the image feature data of the k-th structural region; Φ() represents the feature extraction operation process.

[0047] After obtaining the structural region feature data, water accumulation region segmentation and identification processing is performed within each structural region. By classifying and determining the pixel features within the region, the set of pixels belonging to the water accumulation region is extracted as follows: ;in, This represents the water accumulation area within the k-th structural region; This indicates the value used to determine if a pixel belongs to a water body region. This indicates the threshold for determining water accumulation.

[0048] After identifying the waterlogged areas, leakage location detection is performed on local anomalies within the structural area. This is achieved by detecting continuous watermarks and concentrated edge areas to determine the set of leakage locations. Subsequently, based on the direction of continuous pixel distribution and regional morphological parameters, water flow pattern classification is performed on the identified area to distinguish different water flow distribution states. Simultaneously, by analyzing pixel grayscale distribution and texture changes within the area, the water state is determined, resulting in a water state identification result.

[0049] The above identification results are uniformly associated according to the structural region number to form a set of structural region identification results: ; in, This represents the identification result of the k-th structural region at time t; This indicates the results of the waterlogged area identification; This indicates the result of the leak location identification; This indicates the results of water flow pattern recognition; This indicates the result of water body status identification.

[0050] After completing the above processing, all structural region identification results are associated and stored according to region number to form complete structural region identification result data. By performing image recognition processing under structural region constraints, the image recognition process is limited to the specific factory building structural region, realizing the correspondence between the identification results and the actual structural location, and providing a unified regional data foundation for subsequent structural region change tracking and water level data fusion judgment.

[0051] Furthermore, in step S5, the step of performing region change tracking processing based on the structural region identification results at consecutive time points includes: Obtain the structural region identification results at consecutive time points in chronological order; Under the same structural region numbering, determine the recognition region corresponding to consecutive time periods; Perform location matching and boundary tracking processing on the identified region; Record the area and location changes of the identified region over consecutive time periods; Data on water body changes in structural regions are generated based on information on area and location changes.

[0052] Specifically, in this step, the structural region identification result data generated in step S4 is first read in a unified time series order. For any time t, the structural region identification result contains information on the water accumulation areas corresponding to multiple structural region numbers, thus forming a region identification sequence over continuous time. Let the water accumulation area of ​​the k-th structural region at time t be represented as: ;in, The structural region number is The set of pixels in the recognition region; t represents the time series number.

[0053] In continuous time processing, adjacent time t is selected. The identification regions for 1 and t are determined by performing region correspondence analysis under the condition of the same structural region number. Through structural region number constraints, variation analysis is performed only within the same factory building structure at consecutive time points. After completing region number matching, position matching processing is performed on the identification regions at consecutive time points. Position matching is achieved by calculating the center position of the region. Let the coordinates of the center of the identification region be: ; in, , These represent the x-coordinate and y-coordinate of the center of the identification region of the structural region at time t, respectively.

[0054] The coordinates of the region center are calculated by averaging the coordinates of all pixels within the region, and are used to represent the location of the water body. Subsequently, boundary tracking processing is performed. By extracting the outer contour of the identified region, corresponding matching is performed on the region boundaries at consecutive time points to obtain the changes in the region's morphology. During boundary tracking, the sets of boundary points at consecutive time points are correlated to ensure that the same water body region maintains continuous identification in the time series.

[0055] After completing position matching and boundary tracking, the area of ​​the identified region is calculated. The identified area of ​​the structural region at time t is expressed as: ; in, This represents the area of ​​water accumulation in the structural region at time t; Indicates the number of pixels in the recognition area; This represents the actual area corresponding to a single pixel.

[0056] Based on the area calculation results at continuous time intervals, the change in the area of ​​the structural region is obtained: ;in, This indicates the area change of the structural region over consecutive time points. Simultaneously, the change in position is calculated by measuring the change in the center position of the region over consecutive time points. ; in, It indicates the distance of change in the center position of the identified area, and is used to characterize the change in the direction of water body expansion.

[0057] After obtaining information on area and location changes, the two types of change data are recorded uniformly to form water body change data for the structural region: ;in, This represents the water change data of the k-th structural region at time t.

[0058] Through the above-mentioned regional change tracking processing, the water state within the structural area is transformed from a single-moment identification result into continuous time-varying data, realizing a continuous description of the changes in the water accumulation range and spatial movement, thereby providing a time-continuous data foundation for subsequent correlation analysis between water level change data and structural area change data.

[0059] Furthermore, in step S6, the step of calculating water level change data based on synchronous PLC sensor data includes: Read synchronous PLC sensor data in chronological order; Establish a continuous time-series buffer sequence of water level data; Perform continuous sampling and verification processing on the water level data; Calculate the water level change between adjacent sampling times; A water level change data sequence is formed based on the water level changes over a continuous period of time.

[0060] In step S6, the steps for establishing the correspondence between water level change data and water body change data in the structural region include: Determine the range of the factory building structure area corresponding to the water level change data based on the structural area identifier; The water level change data is matched to different regions according to the structural region number; The water body change data of the structural region is correlated with the water level change data of the corresponding region. And establish a regional correspondence between water level change data and water body change data in structural areas.

[0061] Specifically, in this step, the synchronous PLC sensor data sequence generated in step S2 is read sequentially. The synchronous PLC sensor data has already established a time correspondence with the video image frames; therefore, the water level data at each moment corresponds to a unique time index. Let the synchronous PLC sensor water level data sequence be represented as: ;in, This represents the synchronized water level value at time t. n represents the total number of simultaneous samples.

[0062] To ensure the continuity of subsequent calculations, water level data at consecutive time points are written into a water level buffer sequence and updated chronologically. When new synchronous PLC sensor data enters the buffer sequence, continuous sampling verification is performed. This verification process includes time interval consistency checks and data validity checks. When an abnormal sampling interval is detected, the time series continuity is maintained using adjacent valid sampled data. After completing the sampling verification, the change in water level data at consecutive time points is calculated. The change in water level between adjacent sampling times is expressed as: ;in, This represents the change in water level at time t relative to the previous time. and These represent the synchronized water level values ​​at two consecutive moments. By sequentially recording the water level changes over a continuous time period, a water level change data sequence is formed: This sequence is used to describe the overall water level change of the plant over time.

[0063] After obtaining the water level change data, the process of establishing the correspondence between the water level change data and the water body change data of the structural area is executed. First, based on the structural area identification information generated in step S3, the factory structure area number to which each video image frame belongs is determined, so that the water level change data in the time series can establish an association range with the corresponding structural area.

[0064] Since the synchronous PLC sensor data and video image data correspond under the same time series, the water change data of the k-th structural region at time t can be expressed as: ;in, This represents the water level change data for the k-th structural region generated in step S5. Subsequently, region matching processing is performed based on the structural region number, associating the water level change data under the same time index with the corresponding structural region's water level change data to form region-related data: ; in, The structural region number is Regional correspondence data; This indicates overall water level changes; This indicates information about local water changes within a structural region.

[0065] Through the above-mentioned correlation processing, the overall water level change information of the plant is made to correspond with the water body change status in each structural area under the conditions of unified time and unified area number, thereby establishing a complete set of regional correspondence data.

[0066] By uniformly linking the water level change data from the synchronous PLC sensor with the water body change data of the structural area, the overall water level change trend and the local structural area change state are expressed in a consistent way. This provides a unified input basis for joint calculation of different data sources in subsequent fusion judgment processing, and enables collaborative analysis of multi-source monitoring data in the same structural area dimension.

[0067] Furthermore, in step S7, the step of performing fusion determination processing on the water level change data and the water body change data of the structural region based on the correspondence includes: Read the region mapping relationship; Obtain the PLC-side risk assessment data corresponding to water level change data and the visual-side risk assessment data corresponding to water body change data in the structural area, respectively. The risk assessment data from the PLC side and the risk assessment data from the vision side are weighted and calculated according to the preset fusion weights; and a comprehensive risk assessment result is generated based on the calculation results.

[0068] Specifically, in this step, the regional correspondence data established in step S6 is read first. The regional correspondence data, indexed by the structural region number, unifies the water level change data and structural region water body change data within the same time series, ensuring that the fusion and judgment processing is performed under the same time and structural region conditions. Let the regional correspondence data of the k-th structural region at time t be represented as: After reading the regional correspondence, PLC-side risk assessment data is first generated based on water level change data. This PLC-side risk assessment data is obtained through cumulative analysis of continuous water level changes and is used to characterize the overall water level change status of the plant. The PLC-side risk assessment value is expressed as: ; in, This represents the risk assessment data on the PLC side at time t. This represents the change in water level at time i. N represents the length of the continuous time window involved in the calculation.

[0069] By using continuous time windows, the PLC-side risk assessment data reflects the water level change trend within a certain time range. Subsequently, visual risk assessment data is generated based on the water body change data within the structural area. Visual risk assessment is obtained through comprehensive calculation of changes in the water area and location within the structural area. The visual risk assessment data is represented as follows: ; in, This represents the visual risk assessment data for the structural region; This indicates the change in the area of ​​the structural region; Indicates the amount of change in the position of the structural region; , This represents the weighting coefficient for the changing data.

[0070] After obtaining risk assessment data from both the PLC and vision sides, a fusion calculation is performed on the two types of data. The comprehensive risk assessment result is expressed as follows: ; in, The structural region number is The comprehensive risk assessment results; This represents the risk assessment data from the PLC side; This represents visual risk assessment data; and This represents the fusion weight parameters, and satisfies α+β=1.

[0071] By calculating the comprehensive risk assessment results for each structural region separately, a risk assessment sequence for the plant's structural regions is formed, and the results are used as the input basis for subsequent early warning and control steps. Through the above-mentioned fusion assessment process, the overall water level change information and the local water body change information of the structural region are jointly calculated under the condition of a unified regional correspondence, converting data from different sources into a unified risk assessment result, realizing a comprehensive assessment of the plant's operating status, and providing a consistent data basis for the generation of subsequent early warning and control instructions.

[0072] Furthermore, in step S8, the steps for executing alarm linkage control include: Early warning level information is generated based on the comprehensive risk assessment results; Generate corresponding control commands based on the warning level information; The control commands are sent to the audible and visual alarm devices, the plant drainage control devices, and the remote monitoring terminal; and the execution status information of the control commands is recorded.

[0073] In step S8, the event data recording step includes: Record the synchronous PLC sensor data corresponding to the comprehensive risk assessment results; Record video image data containing structural region identifiers, record structural region identification results and structural region water body change data, and establish a unified event identifier for the above data for associated storage.

[0074] Specifically, this step first reads the comprehensive risk assessment result generated in step S7. The comprehensive risk assessment result is organized based on structural region numbers and time series indices, reflecting the operational status of different structural regions of the plant at the current moment. Let the structural region number be... The comprehensive risk assessment result at time t is expressed as: ;in, This represents the risk assessment value of the k-th structural region at time t.

[0075] The warning level classification is performed based on the comprehensive risk assessment results. This involves comparing the comprehensive risk assessment value with a preset level range to generate corresponding warning level information. The warning level determination process is as follows: ; in, Indicates the warning level corresponding to the structural region; G() represents the level mapping function, which is used to convert risk assessment values ​​into discrete level numbers.

[0076] Upon receiving the warning level information, control command data is generated based on the warning level. Control commands include alarm trigger commands, drainage equipment start commands, and remote information transmission commands. Control commands are organized according to structural area numbers and sent to the corresponding execution devices via the control communication interface. During the control command transmission process, the command transmission time, execution device identifier, and execution feedback status are recorded, forming control execution status data. Execution status information is used to characterize the transmission result of the control commands and the device response.

[0077] After completing the alarm linkage control, event data recording and processing are performed. First, a unified event identifier is generated based on the current alarm trigger time. Let the event identifier be represented as: ; in, Indicates the event identifier; Indicates the time when the event occurred; This indicates the corresponding structural region number.

[0078] Subsequently, using event identifiers as indexes, relevant operational data is uniformly associated and stored, including synchronous PLC sensor data sequences, video image data containing structural region identifiers, structural region identification results, and structural region water change data. All types of data are bound according to unified event identifiers, forming complete record units for data corresponding to the same early warning process. During data storage, event data is written to the event record database in chronological order, maintaining consistency between structural region numbers and time indexes, thereby ensuring that historical operational data can be retrieved and accessed according to event dimensions.

[0079] Through the aforementioned alarm linkage control and event data recording and processing, the comprehensive risk assessment results can directly drive the field equipment to perform control operations and simultaneously form a complete data recording link, realizing a continuous processing process from risk assessment to control execution and event recording, thereby ensuring that the early warning process has a clear data correspondence and traceable operation records.

[0080] Example 2 Based on Example 1, in order to further improve the accuracy of flood scene identification and the reliability of risk assessment in hydropower plant, this example introduces a deep learning-based visual recognition model and a multimodal data fusion network in steps S4 and S7, so that PLC sensor data and video image data are jointly modeled at the feature level, thereby achieving higher accuracy flood risk warning.

[0081] The overall process of this embodiment is basically the same as that of Embodiment 1. The difference is that a multi-task recognition model based on deep convolutional neural networks is used in the process of flooding scene recognition, and joint reasoning of PLC sensor data and visual feature data is realized through deep fusion network in the risk judgment stage.

[0082] In step S4, video image data containing structural region identifiers is input into a deep learning recognition model for flooded scenes. This model uses a convolutional neural network (CNN) as the feature extraction backbone and combines a feature pyramid structure for multi-scale feature fusion to improve the ability to identify flooded areas and seepage areas at different scales.

[0083] First, feature extraction is performed on the input image. Let the input image be: Where H and W represent the image height and width, respectively, and C represents the number of channels.

[0084] Extracting image features using convolutional networks: ;in: For depth feature maps, This represents the feature extraction function of a convolutional neural network. This represents the network parameters. After obtaining the feature maps, a multi-task learning structure is used to simultaneously complete the tasks of water accumulation segmentation, leakage detection, and water flow pattern classification. The network shares the same feature extraction layer and establishes task branches on higher-level features, thereby achieving multi-task joint learning. The water accumulation area segmentation task adopts a pixel-level classification method, using a softmax classification function to obtain the water body probability distribution. ; in: This represents the probability that a pixel belongs to a water body region. This represents the classification score output by the network.

[0085] When the pixel probability exceeds a preset threshold, it is identified as a water accumulation area. For the leakage location detection task, a bounding box regression method based on an object detection network is used to locate abnormal water stains within the structural region. The detection network identifies leakage areas by predicting the bounding box position and confidence level. The water flow pattern recognition task uses a classification network for judgment. By analyzing regional texture features and dynamic morphological features, the water flow state is divided into categories such as static water accumulation, slow-flowing water, and fast-flowing water. During the model training phase, a joint loss function is used for optimization, and its overall loss function is expressed as: ; in: To divide the waterlogged area into sections for loss, Losses due to leakage detection Losses are classified according to water flow patterns. , , This represents the task weighting coefficient.

[0086] Through multi-task joint training, the model can learn multiple water features simultaneously, thereby improving the overall recognition accuracy.

[0087] In step S5, to improve the ability to analyze water body change trends, this embodiment further introduces a time-series deep learning model to dynamically model the visual recognition results at continuous time points. The structural region features of the continuous time series are input into a Long Short-Term Memory (LSTM) network for time-series analysis. Let the feature representation of the k-th structural region in the time series be: ; in: This represents the visual feature vector of the structural region at time t.

[0088] LSTM networks use a gating structure to memorize and update historical information, thereby obtaining the trend characteristics of regional water body changes. The internal state update process of the network follows the standard LSTM computation procedure.

[0089] Through the above time series analysis, the characteristics of water change trends in the structural region can be obtained, which can be used to describe information such as water diffusion speed, change direction and change duration, thereby improving the ability to identify abnormal water accumulation trends.

[0090] In step S7, this embodiment employs a deep multimodal fusion network to jointly model PLC sensor data and visual recognition features. First, the PLC sensor water level change data undergoes feature encoding processing, converting the time-series water level data into sensor feature vectors: ; in: This represents a data sequence of water level changes. () represents the sensor feature coding network.

[0091] Simultaneously, the visual recognition results and water body change characteristics are input into the visual feature encoding network to obtain visual feature vectors: ;in: Indicates visual recognition features, () represents the visual feature encoding function. Subsequently, the two types of features are fused, and the fused features are represented as: ;in: Indicates feature splicing, () represents the fusion network. The final comprehensive risk probability is calculated using a fully connected network: Where: R represents the overall risk probability, For the Sigmoid function, , These are model parameters. When the risk probability exceeds a set threshold, the corresponding level of early warning control is triggered.

[0092] To reduce the need for manually labeled data, this embodiment can also employ self-supervised learning to pre-train the visual model. By performing temporal prediction or feature comparison learning on adjacent frames in the video sequence, the model can automatically learn the basic visual features of the power plant environment. During self-supervised training, by constructing positive and negative sample feature pairs, the model learns the feature consistency between different time frames, thereby improving the model's sensitivity to water body change features. After self-supervised pre-training, fine-tuning training with a small amount of labeled data can significantly improve the model's recognition performance in actual hydropower station environments.

[0093] By introducing a deep learning visual recognition model, a time series dynamic analysis model, and a multimodal fusion network on the basis of Example 1, PLC sensor data and video image data are jointly modeled at the feature level, thereby realizing intelligent judgment of the risk of flooding in the factory.

[0094] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A smart early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition, characterized in that, Includes the following steps: S1. Collect real-time water level data output by PLC sensors inside the hydropower station powerhouse, and collect video image data output by the monitoring camera equipment in the powerhouse. S2. Establish a unified time series based on the acquisition timestamp of PLC sensor data and the frame timestamp of video image data, and generate synchronous PLC sensor data corresponding to the video image frames. S3. Obtain the structural area information of the powerhouse based on the building information model of the hydropower station, and map the structural area information of the powerhouse to the video image data to generate video image data containing structural area identifiers; S4. Input the video image data containing the structural region identifier into the flood scene multi-task recognition model to obtain the water accumulation area recognition result, leakage location recognition result, water flow pattern recognition result and water body state recognition result corresponding to the structural region. S5. Based on the structural region identification results at continuous time intervals, perform regional change tracking processing to generate structural region water body change data; S6. Calculate water level change data based on the synchronous PLC sensor data, and establish the correspondence between the water level change data and the water body change data in the structural area; S7. Based on the correspondence, perform fusion judgment processing on the water level change data and the water body change data of the structural area to generate a comprehensive risk judgment result; S8. Generate early warning control instructions based on the comprehensive risk assessment results, and execute alarm linkage control and event data recording.

2. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S2, the step of establishing a unified time series based on the acquisition timestamp of the PLC sensor data and the frame timestamp of the video image data includes: The acquisition timestamp sequence of PLC sensor data and the frame timestamp sequence of video image data are obtained respectively, and the PLC sensor data is sorted according to the acquisition time order. Construct a unified time reference sequence based on the frame timestamps of video image data; The time interval of the PLC sensor data is made consistent according to the unified time reference sequence, and the PLC sensor data with time missing intervals is compensated. Based on a unified time reference sequence, PLC sensor data within the time range corresponding to each video image frame is selected to generate synchronous PLC sensor data corresponding to the video image frame.

3. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S3, the step of generating video image data containing structural region identifiers includes: Read the three-dimensional structural data of the powerhouse in the building information model of the hydropower station, and divide the internal area of ​​the powerhouse into structural areas and number the areas according to the structural characteristics of the powerhouse. Obtain the installation location parameters and field-of-view parameters of the surveillance camera equipment; Establish a spatial mapping relationship between the field of view of the surveillance camera equipment and the structural area of ​​the factory building; Project the factory building structure area onto the video image coordinate space; The corresponding pixel regions in the video image are processed by writing structural region identifiers to form video image data containing structural region identifiers.

4. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S4, the step of inputting the video image data containing structural region identifiers into the flood scene multi-task recognition model includes: Perform image preprocessing operations on video image data containing structural region identifiers; Region segmentation of video images based on structural region identifiers; Extract image feature data for each structural region separately; Within each structural area, processes such as water accumulation area segmentation and identification, leakage location detection, water flow pattern classification, and water body state determination are performed. The identification results of each structural region are then associated and stored according to the structural region number to generate structural region identification results.

5. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S5, the steps of performing region change tracking processing based on the structural region identification results at consecutive time points include: Obtain the structural region identification results at consecutive time points in chronological order; Under the same structural region numbering, determine the recognition region corresponding to consecutive time periods; Perform position matching and boundary tracking processing on the identified region; Record the area and location changes of the identified region over consecutive time periods; Based on the area change information and location change information, water body change data for the structural region is generated.

6. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S6, the step of calculating water level change data based on the synchronous PLC sensor data includes: Read synchronous PLC sensor data in chronological order; Establish a continuous time-series buffer sequence of water level data; Perform continuous sampling and verification processing on the water level data; Calculate the water level change between adjacent sampling times; A water level change data sequence is formed based on the water level changes over a continuous period of time.

7. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S6, the step of establishing the correspondence between the water level change data and the water body change data of the structural area includes: Determine the range of the factory building structure area corresponding to the water level change data based on the structural area identifier; The water level change data is matched to different regions according to the structural region number; The water body change data of the structural region is correlated with the water level change data of the corresponding region. And establish a regional correspondence between water level change data and water body change data in structural areas.

8. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S7, the step of performing fusion determination processing on the water level change data and the water body change data of the structural area based on the correspondence includes: Read the region correspondence; Obtain the PLC-side risk assessment data corresponding to water level change data and the visual-side risk assessment data corresponding to water body change data in the structural area, respectively. The risk assessment data from the PLC side and the risk assessment data from the vision side are weighted and calculated according to the preset fusion weights; and a comprehensive risk assessment result is generated based on the calculation results.

9. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S8, the step of executing alarm linkage control includes: Early warning level information is generated based on the comprehensive risk assessment results; Generate corresponding control commands based on the warning level information; The control commands are sent to the audible and visual alarm devices, the plant drainage control devices, and the remote monitoring terminal; and the execution status information of the control commands is recorded.

10. The intelligent early warning method for hydropower stations based on the fusion of PLC sensor data and image recognition according to claim 1, characterized in that, In step S8, the event data recording step includes: Record the synchronous PLC sensor data corresponding to the comprehensive risk assessment results; Record video image data containing structural region identifiers, record structural region identification results and structural region water body change data, and establish a unified event identifier for the above data for associated storage.

Images