A flood process data compression method and system, device and medium based on feature point recognition and nonlinear pumping

Abstract

The application relates to the technical field of flood simulation, in particular to a flood process data compression method and system based on feature point identification and nonlinear pumping, equipment and a medium. The method provided by the application mainly comprises feature point identification and segmentation based on a flow sequence, the flood process is divided into a pre-rise base flow section, a rising water section, a falling water section and a post-falling water base flow section; the rising water section and the falling water section are subjected to nonlinear weight pumping based on a tangent function; and the pre-rise base flow section and the post-falling water base flow section are subjected to equal-step coarse pumping by using an equal-step method. Through the above method, the high-fidelity compression of flood data is realized through technical means such as Savitzky-Golay filtering denoising, automatic identification of feature points by a slope threshold, nonlinear weight pumping by a tangent function and segmented differential processing, the flood peak area is adaptively encrypted and sampled, the key shape is ensured to be completely reserved, and the base flow redundant data is greatly simplified.

Description

Technical Field

[0001] This invention relates to the field of flood simulation technology, and more specifically, to a method, system, device, and medium for compressing flood process data based on feature point identification and nonlinear extraction. Background Technology

[0002] As core data in hydrological analysis, flood hydrographs are often presented in massive quantities due to their high resolution. The raw hydrograph data generated by commonly used flood process simulation software often reaches more than 100,000 rows. Direct use of these data can easily lead to low computational efficiency and high hardware resource consumption. In order to improve the computational efficiency of subsequent stages, it is necessary to extract a small portion of the data to represent the original process data.

[0003] Existing pumping techniques mostly employ equidistant sampling, which easily leads to the loss of key regional features such as flood peaks and results in a large amount of redundant data in the baseflow section. While manual segmentation and pumping methods exist, these are subjective, inefficient, and lack a unified standard for nonlinear sampling weights. Current technologies have not yet formed an integrated solution for automated feature point identification and phased, nonlinear, precise pumping, making it difficult to accurately preserve the core morphology and key hydrological features of the flood process while compressing data volume, thus failing to meet the high-efficiency and precise processing requirements of the hydrological field. Summary of the Invention

[0004] The purpose of this invention is to provide a flood process data compression method, system, device, and medium based on feature point identification and nonlinear pumping to solve the above-mentioned problems in the prior art.

[0005] This invention is achieved through the following technical solution: In a first aspect, the present invention provides a flood process data compression method based on feature point identification and nonlinear extraction, comprising: The acquired raw flood data is preprocessed and smoothed to reduce noise, resulting in a normalized flow sequence. Based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into the pre-rising baseflow stage, the rising water stage, the receding water stage, and the post-receding baseflow stage. Based on the tangent function, nonlinear weighted pumping is performed on the rising and receding water sections, and equal step size method is used to perform equal step size coarse pumping on the baseflow section before rising and the baseflow section after receding. The pumping data of the pre-flood baseflow stage, the flood rise stage, the receding stage, and the post-receding baseflow stage are spliced ​​together from the original time series to form a flood process pumping dataset.

[0006] Preferably, the preprocessing and smoothing noise reduction include: Savitzky-Golay filtering was used to smooth the data sequence. An odd window length and polynomial order were selected, and data points were fitted within the sliding window.

[0007] Preferably, the feature point identification and segmentation based on the traffic sequence includes: The first-order difference approximate derivative is calculated based on the flow sequence, and a change threshold is set to define the point with the largest peak flow as the peak point. Find the point closest to the flood peak that satisfies the condition that the first-order difference approximate derivative is greater than the change threshold and that the first-order difference approximate derivative of the subsequent L consecutive points is positive, and use this as the starting point for the rise. Find the point farthest from the flood peak that satisfies the first-order difference approximate derivative less than a negative change threshold as the point of impact. The flow sequence is divided into three parts: the starting point to the rising point is the pre-rising baseflow section, the rising point to the peak point is the rising water section, the rising water section to the falling point is the receding water section, and the falling point to the end point of the flow sequence is the post-receding baseflow section.

[0008] Preferably, the calculation of the first-order difference approximation derivative includes:

[0009] In the formula, It is the first-order difference approximation derivative. This represents the traffic flow at time i+1. The traffic corresponding to i, At time i+1, Let i be the time.

[0010] Preferably, the nonlinear weighting includes: Let the starting point of the rising or receding water level be 0 and the ending point be 1. Assign a relative ranking value to each data point in the rising or receding water level according to the time series to obtain the interval from 0 to 1. Divide the interval from 0 to 1 into equal parts. Each sub-interval is weighted based on the tangent function, and the number of points to be extracted in each interval is allocated according to the total number of points to be extracted based on the sub-interval weights. The sampling step size is calculated based on the number of original data points and the number of points to be sampled in each sub-interval, and data points are extracted according to the sampling step size.

[0011] Preferably, the calculation of the weights of each sub-interval based on the tangent function includes:

[0012]

[0013] In the formula, The function value, i.e., the constructor value. The corresponding function value after the value is obtained. The independent variable value is given in normalized coordinates, and its range is... , For sub-interval weights, Let be the upper bound of the j-th subinterval y. Let y be the lower bound of the j-th subinterval y.

[0014] Preferably, the calculation sampling step size includes:

[0015] In the formula, The sampling step size, The number of original data points in the sub-interval. This is the number of points to be drawn.

[0016] Secondly, the present invention also provides a flood process data compression system based on feature point recognition and nonlinear pumping, including the aforementioned flood process data compression method based on feature point recognition and nonlinear pumping, and further including: The data processing module is configured to preprocess and smooth the acquired raw flood data to obtain a normalized flow sequence; based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into a pre-flood baseflow segment, a flood rise segment, a flood recession segment, and a post-flood baseflow segment; nonlinear weighted pumping is performed on the flood rise segment and the flood recession segment based on the tangent function, and equal-step coarse pumping is performed on the pre-flood baseflow segment and the post-flood baseflow segment using the equal-step method; The output module is configured to splice the pumping data of the baseflow section before flood rise, the flood rise section, the flood recede section, and the baseflow section after flood recede according to the original time series to form a flood process pumping dataset.

[0017] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for compressing flood process data based on feature point recognition and nonlinear pumping.

[0018] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for compressing flood process data based on feature point recognition and nonlinear pumping.

[0019] The technical solution of the present invention has at least the following advantages and beneficial effects: The method provided by this invention mainly includes feature point identification and segmentation based on flow sequences, dividing the flood process into a pre-flood baseflow segment, a rising water segment, a receding water segment, and a post-receding baseflow segment. Nonlinear weighted extraction is applied to the rising and receding water segments based on the tangent function, and coarse extraction with equal step sizes is performed on the pre-flood and post-receding baseflow segments using a constant step size method. Through the above methods, using techniques such as Savitzky-Golay filtering for noise reduction, automatic feature point identification using slope thresholds, nonlinear weighted extraction with the tangent function, and segmented differential processing, high-fidelity compression of flood data is achieved. While ensuring the complete preservation of key morphologies through adaptive and intensified sampling in the flood peak area, redundant baseflow data is significantly reduced. This method possesses significant advantages such as high automation, flexible and adjustable parameters, and a controllable balance between compression ratio and accuracy, making it suitable for scenarios such as hydrological simulation data post-processing, flood process visualization, and batch flood analysis. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the process of the present invention; Figure 2 This is a comparison table of the original number of points and the number of points after pumping in each stage of the flood process according to the present invention; Figure 3 This is a table showing the number of pumping points for each flood level (increasing by 20%) in sections AB and AC of the present invention; Figure 4 This is a comparison diagram of the original and post-suction flood process curves of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0023] The independently described modules or sub-modules may or may not be physically separated; they may be implemented in software or hardware, and some modules or sub-modules may be implemented in software, with the processor calling the software to implement the function of these modules or sub-modules, while other modules or sub-modules may be implemented in hardware, such as through hardware circuits. Furthermore, some or all of the modules can be selected to achieve the purpose of this application's solution according to actual needs.

[0024] Please refer to Figures 1-4 The present invention provides a flood process data compression method based on feature point identification and nonlinear pumping, comprising: S1: The acquired raw flood data is preprocessed and smoothed to reduce noise, resulting in a normalized flow sequence; The raw flood data is input and its integrity is verified, and core hydrological index sequences such as time, flow rate, and water level are extracted. Then, Savitzky-Golay filtering is used to smooth the data sequence, selecting an odd window length (recommended initial value of 11) and a polynomial order (recommended second order), and data points are fitted within the sliding window. Subsequently, the filtering effect is verified to ensure that the flood peak attenuation is no greater than 1%. If the requirement is not met, the parameters are adjusted and refitting is performed to obtain a stable and reliable smoothed flow rate sequence. Finally, the smoothed sequence is normalized to unify the time step and index units, providing standardized data for subsequent analysis and calculation.

[0025] S2: Based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into the pre-rising baseflow stage, the rising water stage, the receding water stage, and the post-receding baseflow stage; S3: Based on the tangent function, nonlinear weighted pumping is performed on the rising and receding water sections, and equal step size method is used to perform equal step size coarse pumping on the baseflow section before rising and the baseflow section after receding. S4: The pumping data of the baseflow section before the rise, the rising water section, the receding water section, and the baseflow section after the receding water are spliced ​​together according to the original time series to form a flood process pumping dataset.

[0026] The method provided by this invention mainly includes feature point identification and segmentation based on flow sequences, dividing the flood process into a pre-flood baseflow segment, a rising water segment, a receding water segment, and a post-receding baseflow segment. Nonlinear weighted extraction is applied to the rising and receding water segments based on the tangent function, and coarse extraction with equal step sizes is performed on the pre-flood and post-receding baseflow segments using a constant step size method. Through the above methods, using techniques such as Savitzky-Golay filtering for noise reduction, automatic feature point identification using slope thresholds, nonlinear weighted extraction with the tangent function, and segmented differential processing, high-fidelity compression of flood data is achieved. While ensuring the complete preservation of key morphologies through adaptive and intensified sampling in the flood peak area, redundant baseflow data is significantly reduced. This method possesses significant advantages such as high automation, flexible and adjustable parameters, and a controllable balance between compression ratio and accuracy, making it suitable for scenarios such as hydrological simulation data post-processing, flood process visualization, and batch flood analysis.

[0027] An exemplary embodiment of the present invention includes feature point identification and segmentation based on the traffic sequence, comprising: The first-order difference approximate derivative is calculated based on the flow sequence, and a change threshold is set to define the point with the largest peak flow as the peak point. Find the point closest to the flood peak that satisfies the condition that the first-order difference approximate derivative is greater than the change threshold and that the first-order difference approximate derivative of the subsequent L consecutive points is positive, and use this as the starting point for the rise. Find the point farthest from the flood peak that satisfies the first-order difference approximate derivative less than a negative change threshold as the point of impact. The flow sequence is divided into three parts: the starting point to the rising point is the pre-rising baseflow section, the rising point to the peak point is the rising water section, the rising water section to the falling point is the receding water section, and the falling point to the end point of the flow sequence is the post-receding baseflow section.

[0028] Specifically, calculating the first-order difference approximate derivative includes:

[0029] In the formula, It is the first-order difference approximation derivative. This represents the traffic flow at time i+1. The traffic corresponding to i, At time i+1, Let i be the time.

[0030] In one embodiment, an automatic identification mechanism based on a slope threshold is established. For discrete time series, the first-order difference approximates the derivative, and a change threshold is set. For peak flow 0.5%-1% or 2-3 times the standard deviation of the basestream.

[0031] Positioned as Flood peak point A, find the first point that meets the requirements during the rising water level. Furthermore, the point B is the starting point of the rise, where L consecutive points (L is 3-5) are positive. The last point satisfying this condition is then searched in the receding water section. The point where the water enters a stable phase is taken as the point of impact, C.

[0032] The sequence is marked with D and E at the beginning and end, respectively, and the flood process is finally divided into four segments: DB segment (baseflow segment before flooding), AB segment (flooding segment), AC segment (receding segment), and CE segment (baseflow segment after flooding).

[0033] An exemplary embodiment of the present invention includes performing nonlinear weight extraction: Let the starting point of the rising or receding water level range be 0 and the ending point be 1. Assign a relative ranking value to each data point in the rising or receding water level range according to the time series, resulting in an interval from 0 to 1. Divide this interval into equal parts. The system calculates the weight of each sub-interval based on the tangent function, and allocates the number of points to be extracted in each interval according to the total number of points to be extracted based on the weight of the sub-interval. The sampling step size is calculated based on the number of original data points and the number of points to be extracted in each sub-interval, and data points are extracted according to the sampling step size.

[0034] The calculation of the weights of each sub-interval based on the tangent function includes:

[0035]

[0036] In the formula, The function value, i.e., the constructor value. The corresponding function value after the value is obtained. The independent variable value is given in normalized coordinates, and its range is... , For sub-interval weights, Let be the upper bound of the j-th subinterval y. Let y be the lower bound of the j-th subinterval y.

[0037] The calculation of the sampling step size includes:

[0038] In the formula, The sampling step size, The number of original data points in the sub-interval. This is the number of points to be drawn.

[0039] In one embodiment, non-uniform sampling is performed on segments AB and AC based on the tangent function. Normalized coordinate construction: Let x=0 correspond to point B and x=1 correspond to point A, and assign x a value to each data point in segment AB according to its relative rank in the time series.

[0040] Nonlinear weight allocation: assigning weights to intervals Divide the data into α equal sub-intervals; introduce the tangent function, whose increasing growth rate gives greater weight to sub-intervals closer to the flood peak (x→1); calculate the weight of each sub-interval and allocate it according to the total number of points to be extracted. Allocate the number of points to be sampled in each interval .

[0041] Sampling step size determination and execution: based on the number of original data points and the number of points to be sampled in each sub-interval. The sampling step size is calculated. To ensure that the flood peak point A is preserved, the sampling direction is set to proceed in reverse from the flood peak point A to the starting point A. Within each sub-interval, starting from the right endpoint (near A), the sampling proceeds to the left at intervals of... One data point is extracted from each point.

[0042] AC segment processing: For AC segment (receding water segment), normalized coordinates are constructed so that x=0 corresponds to point C and x=1 corresponds to point A. The same weighting function and calculation process as for AB segment are adopted, and reverse sampling is performed from the flood peak point A to the water drop point C.

[0043] The equal-step method is used to extract points from segments DB and CE. Let the initial number of points in segment DB be... Proposed to extract If there are 1 point, then the step size is 1. Starting from point D, every Take one point from each point.

[0044] Similarly, the original number of points for segment CE is... Proposed to extract If there are 1 point, then the step size is 1. Starting from point C, every Take one point from each point.

[0045] The DB, AB, AC, and CE segments of the original time series are spliced ​​together to form a complete flood process pumping dataset. Anomaly verification is performed on the integrated data to correct issues such as missing sampling points and duplicate points. The output is standardized according to the common hydrological industry format (Excel / TXT / database, etc.), including all core indicators such as time, flow rate, and water level, and supports re-execution of optimization by adjusting parameters (α, number of pumping points, etc.) according to accuracy requirements.

[0046] Parameter feedback optimization provides output evaluation feedback and supports dynamic adjustment of key parameters (number of segments α, number of points to be extracted in each segment) based on visualization effects or subsequent calculation accuracy requirements. / / (Filter window m, order k, threshold coefficient, etc.) automatically re-execute S1 to S4 to achieve a closed-loop iteration of "sucking-evaluation-optimization" until the application requirements are met.

[0047] A flood process data compression system based on feature point recognition and nonlinear pumping, characterized in that it includes the aforementioned flood process data compression method based on feature point recognition and nonlinear pumping, and further includes: The data processing module is configured to preprocess and smooth the acquired raw flood data to obtain a normalized flow sequence; based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into a pre-flood baseflow segment, a flood rise segment, a flood recession segment, and a post-flood baseflow segment; nonlinear weighted pumping is performed on the flood rise segment and the flood recession segment based on the tangent function, and equal-step coarse pumping is performed on the pre-flood baseflow segment and the post-flood baseflow segment using the equal-step method; The output module is configured to splice the pumping data of the baseflow section before flood rise, the flood rise section, the flood recede section, and the baseflow section after flood recede according to the original time series to form a flood process pumping dataset.

[0048] Based on the above scheme, an example is given for further explanation.

[0049] Step 1: Data Preprocessing and Smoothing / Noise Reduction Data Input and Validation: Input the original flood simulation / observation data, validate the data integrity, remove null values ​​and outliers, and extract the core hydrological indicator sequences of time and flow (or water level, using flow as an example here) to ensure that the time dimensions of each sequence correspond one-to-one.

[0050] Savitzky-Golay Filtering Smoothing: The Savitzky-Golay filter is a smoothing and differentiation algorithm based on local polynomial least squares fitting. Its core principle is to traverse data points through a sliding window of preset width, performing least squares fitting on the original data using a polynomial within the window, and using the value of the fitted polynomial at the center point of the window as the smoothed output for that point. Through this "local fitting, center substitution" method, it effectively filters out high-frequency noise while preserving the original characteristics of the signal (such as peak height and peak width) to the maximum extent. The smoothing effect of this method is determined by two key parameters: window width (controlling the smoothing intensity) and polynomial order (controlling the flexibility of the fitted curve).

[0051] SG filtering is applied to the dam-break flood flow sequence to suppress measurement noise and random fluctuations while accurately preserving key morphological features of the flood peak (such as the starting point, peak, and receding curve).

[0052] Data confirmation and parameter initialization: Confirm the traffic series to be processed. Preprocessing (i.e., data input and completion verification) has been completed, and the length is... (generally A one-dimensional array with equal intervals of 0, where the time interval is 0. Set the filter window length ( (where the initial value is odd), and given the steep rise and fall of the flow rate process line, an initial value of [value missing] is recommended. It can be based on the data sampling frequency ( Adjustments should be made to the window time span in principle. It should be smaller than the typical timescale of the flood characteristics to be preserved (such as the peak rise phase); the polynomial order should be set ( To balance smoothness and peak shape preservation, the following recommendations are made for traffic data. (Quadratic polynomial) or (Cubic polynomial), initially usable .

[0053] Sequence boundary extension: The original sequence is extended at its boundaries. This is done using a mirror-symmetric method, extending the boundaries of the original sequence's beginning... Each data point is copied in reverse and added to the beginning of the sequence; the end of the sequence is copied... Each data point is copied in reverse and added to the end of the sequence to generate an extended sequence. .

[0054] Perform filtered convolution calculations: Use the Savitzky-Golay filtering algorithm, based on the set window length. and polynomial order The determined fixed convolution kernel, for the extended sequence Performing a one-dimensional convolution operation directly yields a smooth sequence. .

[0055] Extracting and outputting valid results: from the smoothed extended sequence In the original sequence, N data points of equal length are extracted to obtain the final smoothed flow sequence. The filtering process is then completed.

[0056] Effect verification: The filtered sequence With the original sequence By comparing the changes in key hydrological parameters such as peak flow and peak time, the filtering process was verified to effectively maintain the core shape of the flood hydrograph while removing noise.

[0057] It should be noted that the above steps are mainly for traffic sequences. If there are obvious spikes, Savitzky-Golay filtering is used for smoothing. If the original data is already smooth enough, it can be used directly. .

[0058] Step 2: Automatic feature point identification and segmentation: Change threshold Setting: The effective change threshold is set using either the peak flow percentage method or the baseflow standard deviation method. The method is as follows: ① Based on flood peak: , ① Take 0.005 to 0.01 (0.5% to 1% of the peak flow); ② Based on the standard deviation of the base flow: take the standard deviation of the flow data of the straight section before the flood rise, and multiply it by a coefficient of 2 or 3.

[0059] Peak point A location: Filtering the maximum value in the flow sequence Determine its corresponding index and time coordinate, and denot it as the flood peak point. and with the peak of the flood To serve as a boundary, the sequence is divided into rising water segments ( ) and the receding section ( ), Index for peak flood points.

[0060] Identification of starting point B: Select the first point within the rising water section that simultaneously meets ① ; ② Subsequent continuous points ( 3 to 5 can be selected. The data points that meet both conditions are denoted as the starting point of the price increase. Let the index of this point be... Then the starting point of the rise .

[0061] Waterfall point C identification: Select the last waterfall point within the receding water section that simultaneously meets ① ② All points (or multiple consecutive points) after this point The data points based on the two conditions are denoted as the points where the water falls. Let the index of this point be... Then the time of the point of entry into the water (i.e., the turning point where the slope first enters a plateau phase) Determine its index and time coordinate.

[0062] Determining the first and last points D and E: The first data point of the standardized data sequence is denoted as the first point. The last data point is denoted as the tail point. The complete set of feature points is obtained by integration. .

[0063] Flood process segmentation: Standardized flood data is segmented according to the original time series, dividing the entire flood process into 4 segments, namely... Section (without base flow increase) Section (core segment of initial rise) Section (the core falling segment), Section (slow-falling baseflow section).

[0064] Step 3: Nonlinear weight extraction of the core segment part, This segment represents the core characteristic segment of the flood process. A nonlinear weighting extraction method based on the tangent function is employed, with consistent operational logic, differing only in the direction of normalized coordinate assignment. Taking a segment as an example, it specifically includes: Normalization processing: for Segment construction normalized coordinates ,make =0 corresponds to the starting point of the rise. , =1 corresponds to the flood peak point According to time series Assigning values ​​to each data point in the segment , The value is the data point at Relative ranking on a timeline.

[0065] Nonlinear weight calculation: Sub-interval partitioning: Set the number of segments (Positive integer, default value 5), will The interval [0,1] is divided into equal parts. Divide the subintervals into equal division points. , , … ; Substitute the weight function: Using the tangent function:

[0066] This function is hour ,exist hour And with Increase The accelerated growth gives greater weight to sub-intervals closer to the flood peak, which aligns with the physical intuition that higher sampling density is needed near the flood peak.

[0067] Sub-interval weight calculation: of Equal division points Substitution The function yields the corresponding ( Then the first... Sub-intervals The weight (i.e., the proportion of the number of points to be extracted in this interval to the total number of points extracted in segment AB) is:

[0068] Depend on Therefore, the denominator is always 0. ,and Irrelevant.

[0069] Subinterval point count calculation: Let's assume we want to start from... The total number of points extracted from the segment is (Can be specified by the user, e.g., 1000), then the... The number of points to be extracted from the sub-interval is:

[0070] in This indicates rounding to the nearest integer, adjusting as needed to make the sum equal to the nearest whole number. .

[0071] Sampling step size determination within sub-interval: The first The number of original data points corresponding to the sub-interval is (Assuming the original data is in) The subinterval is approximately uniformly distributed (in practice, it can be allocated proportionally to the time length). Therefore, the extraction step size within this subinterval is:

[0072] like Then take 1. From the right endpoint of the subinterval (closest to...) (Side) Start moving to the left every Taking one point from each location ensures the peak flood point ( Point A is retained. If starting from the left, the starting point needs to be adjusted to ensure that point A is included.

[0073] Sampling Execution and Data Extraction: From the Peak Flood Point Starting from the right endpoint of the sub-interval, proceed in steps towards the left endpoint of the sub-interval. Reverse sampling to ensure the peak flood point Accurately preserved, all sampling points' time, flow rate, water level, and other comprehensive hydrological indicators were extracted. AC section treatment (drainage section): and The differences between the segments are mainly reflected in the normalization direction, time series direction, and sampling direction, as shown in Table 1: Table 1 and Differences between segments

[0074] Operation steps: Construct normalized coordinates: = , (Note that the molecule is) ,make sure =1 in point, =0 in Point); set the number of segments α (can be combined with...) (Segments may be the same or different); calculate weights, allocate points, and determine step size (same as above). (segment method); from Point to Point-direction reverse sampling.

[0075] Step 4: Baseflow section with equal step size coarse suction: DB segment (without base current rise) suction: Pumping point setting: Manually set according to actual hydrological analysis needs. The total number of points to be extracted in the segment It balances the preservation of base flow trends with data simplification.

[0076] Sampling step size calculation: Calculate the sampling step size , This is for rounding down.

[0077] Isometric sampling: from Starting from point D at the beginning of the segment, samples are taken at equal intervals according to the corresponding step size to extract all hydrological indicators such as time, flow rate, and water level from all sampling points.

[0078] CE section (slow-fall baseflow section) suction: same method as DB section, set ,calculate ,from Segment start point To begin, samples are taken at equal intervals according to the corresponding step size, and all hydrological indicators such as time, flow rate, and water level are extracted from all sampling points.

[0079] Step 5: Standardized Output and Consolidation of Results: Data stitching: Based on the time series of the original flood process, the data is stitched together. part, part, part, The pumping data from each segment are spliced ​​together sequentially to form a complete flood process pumping dataset; Data quality verification: The integrated dataset is checked for continuity and integrity, and anomalies such as missing samples, duplicate data, and coordinate offsets are identified and corrected to ensure that the dataset can fully reproduce the trend of the flood process; Multi-format standardized output: The validated dataset is standardized and output according to the common formats in the hydrological industry (Excel / TxT / database format, etc.), retaining all core hydrological indicators such as time, flow, water level, and breach width; Parameter feedback and optimization iteration: If the hydrological analysis requires adjustments to data accuracy or processing efficiency, the number of segments can be flexibly modified. Number of points to be extracted in each segment, filter window length, and effective change threshold. Repeat the above steps with the parameters set to achieve parameter adaptation for different scenarios.

[0080] It mainly includes the following functional modules: The global parameter configuration module centrally manages smoothing filter parameters (window length 11, polynomial order 2), slope threshold parameters (0.5% peak flow), suction control parameters (number of segments α=5, weight allocation for each segment), and plotting display parameters, allowing users to flexibly adjust them according to their actual needs.

[0081] Data I / O module: Enables interactive selection of input and output folders through a graphical dialog box, automatically reads flood data in CSV format and performs preprocessing operations such as data type conversion, missing value cleaning, and time-series sorting.

[0082] The data preprocessing module uses Savitzky-Golay filtering to smooth and reduce noise in the original flow sequence, and simultaneously calculates the first-order difference sequence to provide a reliable data foundation for subsequent feature point identification.

[0083] Feature point recognition module: Based on the slope threshold, it automatically locates the flood peak point A, the rising point B, the sinking point C, and the first and last points D and E, realizing the intelligent four-segment division of the flood process (DB segment, AB segment, AC segment, CE segment).

[0084] The suction algorithm module: Segments DB and CE employ uniform suction with equal step sizes; segments AB and AC introduce the tangent function y= A nonlinear weighting model is constructed, and the interval is divided into several sub-intervals according to the number of segments α. The number of sampling points in each sub-interval is adaptively allocated according to the weight ratio. Back sampling from the flood peak side ensures that the key morphology is completely preserved.

[0085] Visualization module: Generates a comparison curve between the original data and the data after extraction, using differentiated line widths and colors to intuitively verify the extraction effect.

[0086] Main control module: Enables automatic batch file traversal and processing, user input interaction, single file complete process scheduling and exception handling, and finally outputs standardized CSV data files and comparative visualization images.

[0087] This method and system describe the entire process from raw data preprocessing to standardized output. First, Savitzky-Golay filtering is used to smooth and reduce noise in the data. Then, the slope thresholding method is used to automatically identify key feature points such as flood peaks, inflection points, and sinking points, and complete the process. part, part, part, The system is divided into four segments, and a method based on the core fluctuation segment is adopted. Nonlinear weighted sampling using the tangent function enables high-resolution, intensified sampling near flood peaks. For smooth baseflow sections, coarse sampling with equal steps simplifies redundant data. Finally, data stitching, anomaly detection, and industry-standard format output are completed, with the entire process supporting multiple segmentation. With parameters such as the number of pumping points being flexibly adjusted, this standardized and differentiated operating method can significantly compress massive amounts of flood data and improve processing efficiency while accurately preserving core hydrological characteristics such as flood peaks. It adapts to the processing needs of various single-peak flood processes, providing high-quality processed data for hydrological simulation, flood control and disaster reduction.

[0088] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0089] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer software product, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0090] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A flood process data compression method based on feature point recognition and nonlinear pumping, characterized in that, include: The acquired raw flood data is preprocessed and smoothed to reduce noise, resulting in a normalized flow sequence. Based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into the pre-rising baseflow stage, the rising water stage, the receding water stage, and the post-receding baseflow stage. Based on the tangent function, nonlinear weighted pumping is performed on the rising and receding water sections, and equal step size method is used to perform equal step size coarse pumping on the baseflow section before rising and the baseflow section after receding. The pumping data of the pre-flood baseflow stage, the flood rise stage, the receding stage, and the post-receding baseflow stage are spliced ​​together from the original time series to form a flood process pumping dataset.

2. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 1, characterized in that, The preprocessing and smoothing noise reduction include: Savitzky-Golay filtering was used to smooth the data sequence. An odd window length and polynomial order were selected, and data points were fitted within the sliding window.

3. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 2, characterized in that, The feature point identification and segmentation based on the traffic sequence includes: The first-order difference approximate derivative is calculated based on the flow sequence, and a change threshold is set to define the point with the largest peak flow as the peak point. Find the point closest to the flood peak that satisfies the condition that the first-order difference approximate derivative is greater than the change threshold and that the first-order difference approximate derivative of the subsequent L consecutive points is positive, and use this as the starting point for the rise. Find the point farthest from the flood peak that satisfies the first-order difference approximate derivative less than a negative change threshold as the point of impact. The flow sequence is divided into three parts: the starting point to the rising point is the pre-rising baseflow section, the rising point to the peak point is the rising water section, the rising water section to the falling point is the receding water section, and the falling point to the end point of the flow sequence is the post-receding baseflow section.

4. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 3, characterized in that, The calculation of the first-order difference approximate derivative includes: In the formula, It is the first-order difference approximation derivative. for The corresponding traffic at that time for The corresponding traffic at that time for time, for time.

5. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 4, characterized in that, The nonlinear weight extraction includes: Let the starting point of the rising or receding water level be 0 and the ending point be 1. Assign a relative ranking value to each data point in the rising or receding water level according to the time series to obtain the interval from 0 to 1. Divide the interval from 0 to 1 into equal parts. Each sub-interval is weighted based on the tangent function, and the number of points to be extracted in each interval is allocated according to the total number of points to be extracted based on the sub-interval weights. The sampling step size is calculated based on the number of original data points and the number of points to be sampled in each sub-interval, and data points are extracted according to the sampling step size.

6. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 5, characterized in that, The calculation of the weights of each sub-interval based on the tangent function includes: In the formula, The function value, i.e., the constructor value. The corresponding function value after the value is obtained. The independent variable value is given in normalized coordinates, and its range is... , For sub-interval weights, Let be the upper bound of the j-th subinterval y. Let y be the lower bound of the j-th subinterval y.

7. The flood process data compression method based on feature point recognition and nonlinear pumping according to claim 6, characterized in that, The calculation sampling step size includes: In the formula, The sampling step size, The number of original data points in the sub-interval. This is the number of points to be drawn.

8. A flood process data compression system based on feature point recognition and nonlinear pumping, characterized in that, The flood process data compression method based on feature point identification and nonlinear pumping, as described in any one of claims 1-7, further includes: The data processing module is configured to preprocess and smooth the acquired raw flood data to obtain a normalized flow sequence; based on the flow sequence, feature point identification and segmentation are performed to divide the flood process into a pre-flood baseflow segment, a flood rise segment, a flood recession segment, and a post-flood baseflow segment; nonlinear weighted pumping is performed on the flood rise segment and the flood recession segment based on the tangent function, and equal-step coarse pumping is performed on the pre-flood baseflow segment and the post-flood baseflow segment using the equal-step method; The output module is configured to splice the pumping data of the baseflow section before flood rise, the flood rise section, the flood recede section, and the baseflow section after flood recede according to the original time series to form a flood process pumping dataset.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the flood process data compression method based on feature point recognition and nonlinear pumping as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements a flood process data compression method based on feature point recognition and nonlinear pumping as described in any one of claims 1-7.

Images