Wave overtopping amount identification method and device based on image identification and segmentation calculation

Abstract

The embodiment of the invention discloses a wave overtopping amount identification method and device based on image identification and segmentation calculation, and the method comprises the steps: carrying out the marking of a collected wave overtopping data set, carrying out the data enhancement processing of a training data set, improving an initial identification model, and carrying out the training through combining with the training data set, thereby obtaining an identification model. Inputting the test data set into the identification model to output an identification result, and calculating an output wave overtopping amount based on the identification result and the elevation data set; according to the embodiment of the invention, the obvious features of overtopping can be marked by marking the overtopping data set, the diversity of the data and the generalization ability of the model can be improved through the enhancement processing of the data, and the calculation precision of the overtopping amount is improved through the deep learning and data enhancement technology; the initial recognition model is improved to improve the recognition precision of the model, the wave overtopping amount is obtained through segmented calculation, the wave overtopping amount recognition method does not need to make direct contact with sea waves, the measurement cost and risk are reduced, and wave overtopping recognition under different weather conditions and sea wave states can be used.

Description

Technical Field

[0001] The invention relates to the technical field of overtopping amount recognition, and in particular to an overtopping amount recognition method and device based on image recognition and segmented calculation. Background Art

[0002] Overtopping refers to the amount of water that flows over the top of coastal structures such as seawalls and revetments under the action of waves. This parameter is of great significance for evaluating the stability of protective structures such as seawalls, predicting seawater overtopping disasters, and guiding engineering design.

[0003] The study of overtopping volume began in the 1950s. With the continuous development of marine engineering technology, the study of overtopping volume has gradually deepened. Early research mainly relied on physical model experiments to observe and analyze the changing laws of overtopping volume by simulating wave propagation and overtopping processes under different conditions. With the popularization of computer technology and the development of numerical simulation technology, more and more scholars have begun to use numerical simulation methods to study overtopping volume. However, when the overtopping volume is currently estimated based on the empirical formula of overtopping volume, it is necessary to combine different cross-sectional shapes and other influencing factors to finally obtain a suitable average overtopping volume calculation formula. The calculation amount is large and requires a lot of manpower and material resources. In addition, the existing empirical coefficients for overtopping volume estimation based on machine learning need to be repeatedly tested, which is difficult to analyze complex shapes, has high requirements on data quality, and is mostly used as an auxiliary tool for scheme optimization selection, not a practical application. In addition, the existing machine learning overtopping volume recognition is based on text data, and overtopping volume recognition cannot be performed through video data. Summary of the invention

[0004] In view of the above-mentioned defects, an embodiment of the present invention discloses a method and device for identifying overtopping amount based on image recognition and segmented calculation, which can accurately identify and calculate the overtopping amount based on images.

[0005] A first aspect of an embodiment of the present invention discloses a method for identifying overtopping volume based on image recognition and segmented calculation, comprising:

[0006] Collecting a number of overtopping data sets from a site database and annotating the overtopping data sets, wherein the overtopping data sets are historical overtopping image data;

[0007] Dividing the overlanding dataset into a training dataset and a test dataset according to a preset ratio, and performing data enhancement processing on the training dataset;

[0008] Constructing an initial recognition model, improving the initial recognition model, and inputting a training data set into the improved initial recognition model for training to obtain a recognition model;

[0009] The test data set is input into the recognition model to output the recognition results, the overtopping elevation data set is loaded, and the output overtopping amount is calculated based on the recognition results and the elevation data set.

[0010] As an optional implementation manner, in the first aspect of the embodiment of the present invention, performing data enhancement processing on the training data set includes:

[0011] Based on Formula I 1 =I+N(0,σ 2 ) Gaussian noise is injected into each pixel of each training data set, where I is the original image and I 1 is the image injected with Gaussian noise, N(0,σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise;

[0012] Translate and rotate the training data after injecting Gaussian noise;

[0013] Based on Formula I new =I 1 +λ*(I 1 -I 1 *F) Sharpening the training data, where I new is the sharpened training data, F is the sharpening filter, and λ is the strength factor.

[0014] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the performing data enhancement processing on the training data set further includes:

[0015] Determine the data enhancement strategy based on the formula: θ (a|s), where θ is the data enhancement strategy, π θ a is the learning rate, R is the reward, and θ is given by the formula To update, is the gradient of the data augmentation strategy.

[0016] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the initial recognition model is improved, including:

[0017] Identify the feature map of the training dataset, based on the formula A convolution kernel is applied to each channel of the feature map for depthwise convolution; is the dth channel of the feature map after depth convolution, F d is the dth channel of the feature map, K d is the corresponding convolution kernel, * indicates the convolution operation;

[0018] By formula F 11 =F1 *K 1*1 The feature map after deep convolution is convolved point by point using a 1*1 convolution kernel to combine different channels; where F 11 is the feature map after point-by-point convolution, F 1 is the feature map after deep convolution, K 1*1 It is a 1*1 convolution kernel.

[0019] As an optional implementation manner, in the first aspect of the embodiment of the present invention, the initial recognition model is improved, further comprising:

[0020] The feature map of the training dataset is divided into parts, where H is the height of the feature map, W is the width of the feature map, and p is the number of segmentations.

[0021] As an optional implementation, in the first aspect of the embodiment of the present invention, the calculating and outputting the overtopping amount based on the recognition result and the elevation data set includes:

[0022] Traverse each pixel in the recognition result, select the target pixel marked as over-the-wave from all pixels, and match the elevation value corresponding to the target pixel based on the elevation data;

[0023] The identification result is evenly divided into a plurality of parts based on the elevation value, and the overtopping area weight in each part is calculated respectively;

[0024] The overtopping amount is calculated based on the overtopping area weight.

[0025] As an optional implementation manner, in the first aspect of the embodiment of the present invention, respectively calculating the overtopping area weight in each part includes:

[0026] Get the center position of each part and the sight distance corresponding to the center position;

[0027] Get the overtopping width and overtopping height of each part, and calculate the overtopping area weight of each part based on the formula: A i is the area weight of the i-th part, the total area is the area corresponding to the i-th part, w i is the overtopping width, and h is the overtopping height, D i is the line of sight distance corresponding to the ith part, and f is the focal length of the camera.

[0028] A second aspect of an embodiment of the present invention discloses an overtopping amount recognition device based on image recognition and segmented calculation, comprising:

[0029] Data collection module: used to collect a number of overtopping data sets from the site database and annotate the overtopping data sets, wherein the overtopping data sets are historical overtopping image data;

[0030] Data enhancement module: used for dividing the overboard data set into a training data set and a test data set according to a preset ratio, and performing data enhancement processing on the training data set;

[0031] Model building module: used to build an initial recognition model, improve the initial recognition model, and input the training data set into the improved initial recognition model for training to obtain a recognition model;

[0032] Overtopping calculation module: used to input the test data set into the recognition model to output the recognition results, load the overtopping elevation data set, and calculate the output overtopping amount based on the recognition results and the elevation data set.

[0033] As an optional implementation, in the second aspect of the embodiment of the present invention, performing data enhancement processing on the training data set includes:

[0034] Based on Formula I 1 =I+N(0,σ 2 ) Gaussian noise is injected into each pixel of each training data set, where I is the original image and I 1 is the image injected with Gaussian noise, N(0,σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise;

[0035] Translate and rotate the training data after injecting Gaussian noise;

[0036] Based on Formula I new =I 1 +λ*(I 1 -I 1 *F) Sharpening the training data, where I new is the sharpened training data, F is the sharpening filter, and λ is the strength factor.

[0037] As an optional implementation, in the second aspect of the embodiment of the present invention, the performing data enhancement processing on the training data set further includes:

[0038] Determine the data enhancement strategy based on the formula: θ (a|s), where θ is the data enhancement strategy, π θ a is the learning rate, R is the reward, and θ is given by the formula To update, is the gradient of the data augmentation strategy.

[0039] As an optional implementation, in the second aspect of the embodiment of the present invention, the initial recognition model is improved, including:

[0040] Identify the feature map of the training dataset, based on the formula A convolution kernel is applied to each channel of the feature map for depthwise convolution; is the dth channel of the feature map after depth convolution, F d is the dth channel of the feature map, K d is the corresponding convolution kernel, * indicates the convolution operation;

[0041] By formula F 11 =F 1 *K 1*1 The feature map after deep convolution is convolved point by point using a 1*1 convolution kernel to combine different channels; where F 11 is the feature map after point-by-point convolution, F 1 is the feature map after deep convolution, K 1*1 It is a 1*1 convolution kernel.

[0042] As an optional implementation manner, in the second aspect of the embodiment of the present invention, the initial recognition model is improved, further comprising:

[0043] The feature map of the training dataset is divided into parts, where H is the height of the feature map, W is the width of the feature map, and p is the number of segmentations.

[0044] As an optional implementation, in the first aspect of the embodiment of the present invention, the calculating and outputting the overtopping amount based on the recognition result and the elevation data set includes:

[0045] Traverse each pixel in the recognition result, select the target pixel marked as over-the-wave from all pixels, and match the elevation value corresponding to the target pixel based on the elevation data;

[0046] The identification result is evenly divided into a plurality of parts based on the elevation value, and the overtopping area weight in each part is calculated respectively;

[0047] The overtopping amount is calculated based on the overtopping area weight.

[0048] As an optional implementation manner, in the second aspect of the embodiment of the present invention, respectively calculating the overtopping area weight in each part includes:

[0049] Get the center position of each part and the sight distance corresponding to the center position;

[0050] Get the overtopping width and overtopping height of each part, and calculate the overtopping area weight of each part based on the formula: A i is the area weight of the i-th part, the total area is the area corresponding to the i-th part, w i is the overtopping width, and h is the overtopping height, D i is the line of sight distance corresponding to the ith part, and f is the focal length of the camera.

[0051] A third aspect of an embodiment of the present invention discloses an electronic device, comprising: a memory storing an executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the overtopping amount identification method based on image recognition and segmented calculation disclosed in the first aspect of the embodiment of the present invention.

[0052] A fourth aspect of an embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the overtopping amount identification method based on image recognition and segmented calculation disclosed in the first aspect of an embodiment of the present invention.

[0053] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0054] In the embodiment of the present invention, the collected overtopping data set is annotated, the overtopping data set is divided into a training data set and a test data set according to a preset ratio, and the training data set is subjected to data enhancement processing, and then the initial recognition model is improved after the initial recognition model is constructed, and the recognition model is obtained by training in combination with the training data set, and the test data set is input into the recognition model to output the recognition result, and the overtopping elevation data set is loaded, and the overtopping amount is calculated and output based on the recognition result and the elevation data set; the embodiment can mark the significant features of the overtopping data set by annotating the overtopping data set, such as the overtopping height, position, time, etc., and the data enhancement processing can increase The diversity of data and the generalization ability of the model, through deep learning and data enhancement technology, the model can accurately identify the overtopping features in the image, thereby improving the calculation accuracy of the overtopping amount; the initial recognition model is improved to improve the recognition accuracy of the model, and the training data set is input into the improved model for training until the model reaches satisfactory performance. The recognition result can accurately reflect the overtopping situation in the image, and the overtopping amount is obtained by segmented calculation by combining the recognition combination and the elevation data set. The overtopping amount recognition method of the embodiment does not need to directly contact the waves, reduces the measurement cost and risk, and can be used for overtopping recognition under different weather conditions and wave states. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a flow chart of an overtopping amount identification method based on image recognition and segmented calculation disclosed in an embodiment of the present invention;

[0057] Figure 2 It is a structural schematic diagram of an overtopping amount recognition device based on image recognition and segmented calculation provided by an embodiment of the present invention;

[0058] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] It should be noted that the terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention are used to distinguish different objects rather than to describe a specific order. The terms "including" and "having" in the embodiments of the present invention and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0061] The embodiment of the present invention discloses a method, device, electronic device and storage medium for identifying overtopping based on image recognition and segmented calculation. In the embodiment, a collected overtopping data set is annotated, the overtopping data set is divided into a training data set and a test data set according to a preset ratio, and data enhancement processing is performed on the training data set. After an initial recognition model is constructed, the initial recognition model is improved, and the recognition model is obtained by training in combination with the training data set. The test data set is input into the recognition model to output a recognition result, and an overtopping elevation data set is loaded. The overtopping amount is calculated and output based on the recognition result and the elevation data set. The embodiment annotates the overtopping data set to mark significant features of the overtopping, such as the overtopping height. The enhanced processing of the data can increase the diversity of the data and the generalization ability of the model. Through deep learning and data enhancement technology, the model can accurately identify the overtopping features in the image, thereby improving the calculation accuracy of the overtopping amount; the initial recognition model is improved to improve the recognition accuracy of the model, and the training data set is input into the improved model for training until the model reaches satisfactory performance. The recognition result can accurately reflect the overtopping situation in the image. Combined with the recognition combination and elevation data sets, the overtopping amount is obtained by segmented calculation. The overtopping amount recognition method of the embodiment does not need to directly contact the waves, reduces the measurement cost and risk, and can be used for overtopping recognition under different weather conditions and wave states.

[0062] Embodiment 1

[0063] See also Figure 1 , Figure 1 It is a flow chart of a method for overtopping amount identification based on image recognition and segmented calculation disclosed in an embodiment of the present invention. Among them, the execution subject of the method described in the embodiment of the present invention is an execution subject composed of software and / or hardware, and the execution subject can receive relevant information by wired or / and wireless means, and can send certain instructions. Of course, it can also have certain processing functions and storage functions. The execution subject can control multiple devices, such as a remote physical server or cloud server and related software, or it can be a local host or server and related software that performs related operations on a device placed somewhere. In some scenarios, multiple storage devices can also be controlled, and the storage devices can be placed in the same place or different places as the devices. For example Figure 1 As shown, the overtopping amount recognition method based on image recognition and segmented calculation includes the following steps:

[0064] 101. Collect a number of overtopping data sets from a site database and annotate the overtopping data sets, wherein the overtopping data sets are historical overtopping image data.

[0065] Overtopping datasets are collected from the station database, which usually contain overtopping data at different times, weather conditions and wave states, and these data are annotated to mark the significant features of overtopping, such as overtopping height, location, time, etc.

[0066] 102. Divide the overlanding data set into a training data set and a test data set according to a preset ratio, and perform data enhancement processing on the training data set.

[0067] The data set is installed with a preset ratio, for example, 80% for training set and 20% for test set, and the training data set and test data set are divided. And the training data set is enhanced to increase the diversity of the data and the generalization ability of the model.

[0068] Specifically, the training data set is subjected to data enhancement processing, including: based on formula I 1 =I+N(0,σ 2 ) Gaussian noise is injected into each pixel of each training data set, where I is the original image and I 1 is the image injected with Gaussian noise, N(0,σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; translate and rotate the training data after injecting Gaussian noise; based on formula I new =I 1 +λ*(I 1 -I 1 *F) Sharpening the training data, where I new is the sharpened training data, F is the sharpening filter, and λ is the strength factor.

[0069] For each piece of training data (i.e., each image) in the training data set, injecting Gaussian noise into each pixel based on the formula can increase the diversity of the image data and make the model more robust to noise. The training data injected with Gaussian noise is translated, that is, the position of the image on the plane is changed, and at the same time, the image is rotated, that is, the orientation of the image is changed. These transformations can simulate images from different perspectives and help the model learn richer features. The training data is sharpened based on the formula to enhance the edges and details of the image. The sharpening filter H is usually a high-pass filter that can retain high-frequency components (i.e., edges and details) and suppress low-frequency components (i.e., smooth areas).

[0070] Performing data enhancement processing on the training data set also includes: determining a data enhancement strategy based on the formula: θ (a|s), where θ is the data enhancement strategy, π θ a is the learning rate, R is the reward, and θ is given by the formula To update, is the gradient of the data enhancement strategy. In practical applications, the data enhancement strategy π may be a complex function, which selects the enhancement method based on the characteristics of the image, the current state of the model, and other factors. The learning rate a needs to be carefully adjusted to ensure that the strategy can be updated stably without falling into the local optimal solution. By adaptively selecting the data enhancement method, the generalization ability of the model can be more effectively improved. Compared with the traditional static data enhancement method, the enhancement strategy can be dynamically adjusted according to the characteristics of different stages and different data sets to obtain better performance. The introduction of the idea of ​​reinforcement learning makes the data enhancement process more flexible and intelligent, and provides new ideas and methods for the training of deep learning models.

[0071] In practice, data augmentation techniques can be used alone or in combination to maximize the generalization ability of the model. In addition to the above operations, data augmentation can be performed in more detail by injecting noise, perturbing, sharpening images, and using reinforcement learning to enhance data to obtain a better quality training set. Injecting noise refers to adding a certain amount of random noise to the image to enhance the robustness of the model to noise. This time, a random value that follows a Gaussian distribution is added to each pixel to increase the image noise. Perturbation refers to random translation and random rotation; moving the image a small distance horizontally and vertically and randomly rotating the object, respectively. Sharpening the image refers to enhancing the edges and details of the image so that the model pays more attention to these features. A sharpening filter is applied through a convolution operation. This time, a Laplace filter is used. Reinforcement learning is used to automatically determine the most effective data augmentation strategy to improve the performance of the machine learning model. After the basic data augmentation operation is performed on the current image, the reward is calculated based on the performance improvement of the enhanced image on the classification task. If the data augmentation leads to an improvement in the model accuracy, the reward is positive; if the accuracy decreases, the reward is negative. The policy network is updated based on the reward signal, making actions that result in positive rewards more likely to be chosen.

[0072] 103. Construct an initial recognition model, improve the initial recognition model, and input the training data set into the improved initial recognition model for training to obtain a recognition model.

[0073] Improve the initial model to improve the recognition accuracy of the model. Input the training data set into the improved model for training until the model achieves satisfactory performance.

[0074] In this step, the initial recognition model is improved, including: identifying the feature map of the training data set, based on the formula A convolution kernel is applied to each channel of the feature map for depthwise convolution; is the dth channel of the feature map after depth convolution, F d is the dth channel of the feature map, Kd is the corresponding convolution kernel, * indicates the convolution operation; through the formula F 11 =F 1 *K 1*1 The feature map after deep convolution is convolved point by point using a 1*1 convolution kernel to combine different channels; where F 11 is the feature map after point-by-point convolution, F 1 is the feature map after deep convolution, K 1*1 It is a 1*1 convolution kernel.

[0075] First, the model identifies and extracts the feature map of the training dataset. The feature map is the result of the convolution layer output in the convolutional neural network (CNN), which contains the spatial features and channel information of the input image; then, for each channel of the feature map, a corresponding convolution kernel is applied for depth convolution. The "depth" here means that the convolution operation is performed independently on the channel dimension of the feature map, that is, each channel uses a different convolution kernel; let Fd' be the d-th channel of the feature map after depth convolution, Fd be the d-th channel of the feature map, Kd is the corresponding convolution kernel, and * represents the convolution operation. Then the depth convolution can be expressed as Fd'=Fd*Kd. This process traverses all channels and generates a new set of feature maps. The main purpose of point-by-point convolution (also known as 1x1 convolution) is to combine channels and fuse information of the feature map after depth convolution. Through 1x1 convolution, the number of feature map channels can be adjusted, cross-channel information can be integrated, and nonlinear transformation of features can be achieved; let O be the feature map after point-by-point convolution, F' be the feature map after depth convolution, and W be the 1x1 convolution kernel. Then the point-by-point convolution can be expressed as O=F′*W. Here W is a two-dimensional convolution kernel with a size of 1x1, but the number of channels matches the number of channels of the input feature map, and the number of output channels is set as needed. Through the combination of deep convolution and point-by-point convolution, the model can more effectively extract and integrate the feature information of the input image, thereby improving the accuracy and robustness of recognition. Compared with the standard convolution operation, the combination of deep convolution and point-by-point convolution can reduce the amount of calculation and the number of parameters while maintaining high performance, thereby improving the training speed and inference efficiency of the model. The method allows the model to flexibly adjust the number of channels and spatial dimensions of the feature map according to the characteristics of the input data and the requirements of the task to adapt to different application scenarios. By improving the initial recognition model with deep convolution and point-by-point convolution, the feature extraction ability and recognition performance of the model can be significantly improved.

[0076] It also includes splitting the feature map of the training data set into parts, where H is the height of the feature map, W is the width of the feature map, and p is the number of segments. That is, finer-grained segmentation, which is a technique applied when processing images, especially when using models based on the Transformer architecture. It allows the model to capture features in the image more meticulously by dividing the image into smaller parts (i.e., Patches). Increasing the number of Patches, which means reducing the size of each Patch, enables the model to observe more details. More accurate recognition results can be obtained in cases of relatively small overtopping amounts or complex scenes. Assuming the original image size is H*W (height X width), in conventional segmentation, the image is divided into Patches of size P*P, and the entire image is divided into Patches. In finer-grained segmentation, a smaller Patch size can be selected, such as P 1 *P 1 , where P 1 < p, so that the image is divided into Patches. The finer resolution provides a higher image resolution to the model, enabling it to capture more detailed features.

[0077] In addition, it can also include improvements to the attention mechanism. After using finer-grained segmentation, the computational cost of model training increases. Therefore, the attention mechanism is optimized to reduce the training resources of the model. The attention mechanism adopted by Swin-transformer is the local attention mechanism. In local attention, the attention scores are calculated only within a local window of the sequence, reducing the computational cost. The formula is as follows: where Q, K, and V are the query, key, and value vectors respectively, and d k is the dimension of the key vector. However, here Q, K, and V are restricted within the local window S i . To reduce waste of computational resources, this paper adopts replacing the attention mechanism with sparse attention. When calculating attention, only the information of the overtopping part, that is, the overtopping amount, is considered, and the rest of the elements are ignored to reduce the computational complexity. Sparse attention can be formalized as selectively masking the elements in the QK T matrix. The specific formula is as follows: where Q, K, and V are the query, key, and value vectors respectively, and d k is the dimension of the key vector, and M is a masking matrix used to selectively retain or ignore certain elements.

[0078] Furthermore, in order to speed up model training and reduce memory usage, the network results are optimized to mixed precision training. Mixed precision training is a technique for optimizing the deep learning training process, using both single precision (float32) and half precision (float16) data types to speed up training and reduce memory usage. First, float16 is used for most calculations to speed up mathematical operations and reduce memory usage. When higher precision is required to ensure numerical stability during the calculation of loss functions and weight updates, float32 is still used. The specific implementation is: float16 is used when inputting data and performing forward propagation to reduce computing time and memory usage, and float32 is used when performing loss calculations, backpropagation, and updating weights. At the same time, single precision is maintained in the loss layer, and float32 is still used to ensure accuracy. Multiply the gradient by a scaling factor before backpropagation to prevent underflow of the gradient in float16 representation. After calculating the gradient, divide the gradient by the same scaling factor to restore its original value. Among them, the gradient scaling formula is scaled_gradients = gradients × scale_factor, and the gradient update formula is W = W-η×(scaled_gradients / scale_factor), where gradients is the original gradient, scale_factor is the scaling factor, W is the weight, and η is the learning rate.

[0079] 104. Input the test data set into the recognition model to output the recognition result, load the overtopping elevation data set, and calculate the output overtopping amount based on the recognition result and the elevation data set.

[0080] Input the test dataset into the trained recognition model and output the recognition results, which should accurately reflect the overtopping in the image. Load the overtopping elevation dataset, which contains the height information of the waves at different locations. Combine the recognition results and the elevation dataset to calculate the overtopping volume through segmented calculation. The overtopping volume refers to the volume or mass of waves passing over the structure per unit time.

[0081] In this step, the output overtopping amount is calculated based on the recognition result and the elevation data set, including: traversing each pixel in the recognition result, selecting the target pixel marked as overtopping from all pixels, and matching the elevation value corresponding to the target pixel based on the elevation data; dividing the recognition result evenly into several parts based on the elevation value, and calculating the overtopping area weight in each part respectively. The overtopping amount is calculated based on the overtopping area weight. The overtopping area weight in each part is calculated respectively, including: obtaining the center position of each part and the line of sight distance corresponding to the center position; obtaining the overtopping width and overtopping height of each part, and calculating the overtopping area weight of each part based on the formula: Ai is the area weight of the i-th part, the total area is the area corresponding to the i-th part, w i is the overtopping width, and h is the overtopping height, D i is the line of sight distance corresponding to the ith part, and f is the focal length of the camera.

[0082] The embodiment uses numpy to load the elevation data set and cleans the invalid values ​​of the elevation data. Ensure that the elevation data and the house image have the same spatial resolution, traverse each pixel of the overtopping recognition image, and obtain the corresponding elevation value from the elevation data for the pixels marked as overtopping. Create a new data structure to store the fused information. According to the overtopping of the elevation data and the recognized picture, the overtopping area is now evenly divided into 5 parts. The calculation method for calculating how much overtopping each of the 5 parts occupies adopts the principle of perspective projection. In perspective projection, the image size of an object decreases as the distance from the camera increases.

[0083] First, it is necessary to traverse each pixel in the recognition result. The recognition result is usually a two-dimensional array (or image), in which each pixel contains a classification label (such as overboard, non-overboard, etc.). From all the pixels, select those target pixels marked as "overboard", which represent the area where overboard actually occurs in the image. For each selected target pixel, it is necessary to find its corresponding elevation value based on the elevation dataset. The elevation dataset is usually a two-dimensional array of the same size as the recognition result, in which each element represents the elevation information of the corresponding pixel position. By matching the position of the target pixel and the elevation dataset, the elevation value of each overboard pixel can be obtained.

[0084] Based on the obtained elevation values, the recognition results (or more specifically, the overtopping areas) are evenly divided into several parts. The basis for the division can be the range, order of magnitude or other appropriate criteria of the elevation values. The purpose of the division is to better understand and calculate the overtopping conditions at different elevations. For each divided part, the area of ​​the overtopping area is calculated, and its area weight is determined according to the proportion of the area in the entire overtopping area. The area weight reflects the importance of different parts in the calculation of overtopping amount. Finally, the total overtopping amount is calculated based on the calculated area weight and the elevation value of each part (or some statistics of the elevation value, such as the mean, median, etc.). The overtopping amount can be a specific value (such as overtopping volume, overtopping quality, etc.) or a relative measure (such as the classification of overtopping intensity). Through pixel-level processing and matching of elevation values, the accurate calculation of the overtopping amount can be achieved, the detailed information in the image can be captured, and the overtopping conditions at different elevations can be taken into account, so that the quantitative analysis and evaluation of the overtopping conditions can be achieved.

[0085] For each divided part, its center position is first determined, and then the line of sight distance corresponding to the center position is calculated based on the position and posture of the camera (usually known) and the coordinates of the center position. The line of sight distance refers to the straight-line distance from the camera to the center position. The overtopping width can be defined as the maximum horizontal width of the overtopping area in the part, and the overtopping height refers to the maximum vertical distance of the overtopping area in the part relative to a certain reference surface (such as the sea surface), which can be obtained from the elevation data set. The formula takes into account the influence of overtopping area, height, line of sight distance and camera focal length on the area weight. Among them, the larger the overtopping area and height, the greater the area weight; the farther the line of sight distance, the area weight may decrease due to the perspective effect; the focal length is a fixed parameter that affects the camera's ability to resolve distant objects. By extracting the geometric features of each part and applying the formula to calculate the area weight, a quantitative evaluation of the area distribution of the overtopping area under different viewing angles can be achieved. This method provides strong support for the accurate calculation of overtopping volume.

[0086] Embodiment 2

[0087] See also Figure 2 , Figure 2 Schematic diagram of the structure of an overtopping amount recognition device based on image recognition and segmented calculation disclosed in an embodiment of the present invention. Figure 2As shown, the overtopping amount recognition device based on image recognition and segmented calculation may include: a data collection module 201, a data enhancement module 202, a model construction module 203 and an overtopping calculation module 204, wherein the data collection module 201 is used to collect a number of overtopping data sets from the site database and annotate the overtopping data sets, wherein the overtopping data sets are historical overtopping image data; the data enhancement module 202 is used to divide the overtopping data sets into a training data set and a test data set according to a preset ratio, and perform data enhancement processing on the training data set; the model construction module 203 is used to construct an initial recognition model, improve the initial recognition model, and input the training data set into the improved initial recognition model for training to obtain a recognition model; the overtopping calculation module 204 is used to input the test data set into the recognition model to output a recognition result, load the overtopping elevation data set, and calculate and output the overtopping amount based on the recognition result and the elevation data set. In order to deal with the problems faced by calculating overtopping based on empirical formulas, such as large amount of calculation and high requirements for expert skills; high cost and time consumption of numerical simulation and strong limitations; as well as the lack of generalization ability of existing machine learning models in overtopping recognition and difficulty in handling dams with complex shapes, and most of them are based on the optimization of overtopping formulas, we proposed an optimized Transformer algorithm for image semantic segmentation. This method combines the recognized image with elevation data, draws line segments on the image, and accurately calculates the specific overtopping amount according to the calculation method of overtopping amount in different regions.

[0088] In the data enhancement module 202, the training data set is subjected to data enhancement processing, including: based on formula I 1 =I+N(0,σ 2 ) Gaussian noise is injected into each pixel of each training data set, where I is the original image and I 1 is the image injected with Gaussian noise, N(0,σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; translate and rotate the training data after injecting Gaussian noise; based on formula I new =I 1 +λ*(I 1 -I 1 *F) Sharpening the training data, where I new is the training data after sharpening, F is the sharpening filter, and λ is the intensity factor. It also includes determining the data enhancement strategy based on the formula: θ (a|s), where θ is the data enhancement strategy, π θ a is the learning rate, R is the reward, and θ is given by the formula To update, is the gradient of the data augmentation strategy.

[0089] In the model building module 203, the initial recognition model is improved, including: identifying the feature graph of the training data set, based on the formula A convolution kernel is applied to each channel of the feature map for depthwise convolution; is the dth channel of the feature map after depth convolution, F d is the dth channel of the feature map, K d is the corresponding convolution kernel, * indicates the convolution operation; through the formula F 11 =F 1 *K 1*1 The feature map after deep convolution is convolved point by point using a 1*1 convolution kernel to combine different channels; where F 11 is the feature map after point-by-point convolution, F 1 is the feature map after deep convolution, K 1*1 It is a 1*1 convolution kernel. It also includes splitting the feature map of the training data set into parts, where H is the height of the feature map, W is the width of the feature map, and p is the number of segmentations.

[0090] In the overtopping calculation module 204, the overtopping amount is calculated and output based on the recognition result and the elevation data set, including: traversing each pixel in the recognition result, selecting the target pixel marked as overtopping from all pixels, and matching the elevation value corresponding to the target pixel based on the elevation data; dividing the recognition result evenly into several parts based on the elevation value, and calculating the overtopping area weight in each part respectively; calculating the overtopping amount based on the overtopping area weight. Further, the overtopping area weight in each part is calculated respectively, including: obtaining the center position of each part and the line of sight distance corresponding to the center position; obtaining the overtopping width and overtopping height of each part, and calculating the overtopping area weight of each part based on the formula: A i is the area weight of the i-th part, the total area is the area corresponding to the i-th part, w i is the overtopping width, and h is the overtopping height, D i is the line of sight distance corresponding to the ith part, and f is the focal length of the camera.

[0091] Embodiment 3

[0092] See also Figure 3 , Figure 3 Schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may be a computer, a server, etc. Of course, in certain circumstances, it may also be a smart device such as a mobile phone, a tablet computer, a monitoring terminal, and an image acquisition device with processing functions. Figure 3 As shown, the electronic device may include:

[0093] A memory 301 storing executable program codes;

[0094] a processor 302 coupled to the memory 301;

[0095] The processor 302 calls the executable program code stored in the memory 301 to execute part or all of the steps in the overtopping amount identification method based on image recognition and segmented calculation in the first embodiment.

[0096] The embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute part or all of the steps in the overtopping amount identification method based on image recognition and segmented calculation in the first embodiment.

[0097] The embodiment of the present invention further discloses a computer program product, wherein when the computer program product is run on a computer, the computer is enabled to execute part or all of the steps in the overtopping amount identification method based on image recognition and segmented calculation in the first embodiment.

[0098] The embodiment of the present invention further discloses an application publishing platform, wherein the application publishing platform is used to publish a computer program product, wherein when the computer program product is run on a computer, the computer executes part or all of the steps in the overtopping amount identification method based on image recognition and segmented calculation in the first embodiment.

[0099] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the processes does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0100] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed over multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0101] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0102] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the method described in each embodiment of the present invention.

[0103] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0104] A person of ordinary skill in the art can understand that some or all of the steps in the various methods of the embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0105] The above is a detailed introduction to the overtopping amount recognition method, device, electronic device and storage medium based on image recognition and segmented calculation disclosed in the embodiments of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for identifying overtopping flow based on image recognition and segmented calculation, characterized in that: include: Collecting a number of overtopping data sets from a site database and annotating the overtopping data sets, wherein the overtopping data sets are historical overtopping image data; Dividing the overlanding dataset into a training dataset and a test dataset according to a preset ratio, and performing data enhancement processing on the training dataset; Constructing an initial recognition model, improving the initial recognition model, and inputting a training data set into the improved initial recognition model for training to obtain a recognition model; The test data set is input into the recognition model to output the recognition results, the overtopping elevation data set is loaded, and the output overtopping amount is calculated based on the recognition results and the elevation data set.

2. The overtopping amount identification method according to claim 1, characterized in that: Performing data enhancement processing on the training data set includes: Based on the formula I1=I+N(0,σ 2 )Inject Gaussian noise into each pixel of each training data in the training data set, where I is the original image, I1 is the image injected with Gaussian noise, N(0,σ 2 ) means the mean is 0 and the variance is σ 2 Gaussian noise; Translate and rotate the training data after injecting Gaussian noise; Based on Formula I new =I1+λ*(I1-I1*F) to sharpen the training data, where I new is the sharpened training data, F is the sharpening filter, and λ is the strength factor.

3. The overtopping amount identification method according to claim 2, characterized in that: The performing data enhancement processing on the training data set further includes: Determine the data enhancement strategy based on the formula: θ (a|s), where θ is the data enhancement strategy, π θ a is the learning rate, R is the reward, and θ is given by the formula To update, is the gradient of the data augmentation strategy.

4. The overtopping amount identification method according to claim 1, characterized in that: The initial recognition model is improved, including: Identify the feature map of the training dataset, based on the formula A convolution kernel is applied to each channel of the feature map for depthwise convolution; is the dth channel of the feature map after depth convolution, F d is the dth channel of the feature map, K d is the corresponding convolution kernel, * indicates the convolution operation; By formula F 11 =F 1 *K 1*1 The feature map after deep convolution is convolved point by point using a 1*1 convolution kernel to combine different channels; where F 11 is the feature map after point-by-point convolution, F 1 is the feature map after deep convolution, K 1*1 It is a 1*1 convolution kernel.

5. The overtopping amount identification method according to claim 4, characterized in that: Improving the initial recognition model also includes: The feature map of the training dataset is divided into parts, where H is the height of the feature map, W is the width of the feature map, and p is the number of segmentations.

6. The overtopping amount identification method according to claim 1, characterized in that: The method of calculating and outputting the overtopping amount based on the identification result and the elevation data set includes: Traverse each pixel in the recognition result, select the target pixel marked as over-the-wave from all pixels, and match the elevation value corresponding to the target pixel based on the elevation data; The identification result is evenly divided into a plurality of parts based on the elevation value, and the overtopping area weight in each part is calculated respectively; The overtopping amount is calculated based on the overtopping area weight.

7. The overtopping amount identification method according to claim 6, characterized in that: Calculate the overtopping area weight in each part separately, including: Get the center position of each part and the sight distance corresponding to the center position; Get the overtopping width and overtopping height of each part, and calculate the overtopping area weight of each part based on the formula: A i is the area weight of the i-th part, the total area is the area corresponding to the i-th part, w i is the overtopping width, and h is the overtopping height, D i is the line of sight distance corresponding to the ith part, and f is the focal length of the camera.

8. An overtopping volume recognition device based on image recognition and segmented calculation, characterized in that: include: Data collection module: used to collect a number of overtopping data sets from the site database and annotate the overtopping data sets, wherein the overtopping data sets are historical overtopping image data; Data enhancement module: used for dividing the overboard data set into a training data set and a test data set according to a preset ratio, and performing data enhancement processing on the training data set; Model building module: used to build an initial recognition model, improve the initial recognition model, and input the training data set into the improved initial recognition model for training to obtain a recognition model; Overtopping calculation module: used to input the test data set into the recognition model to output the recognition results, load the overtopping elevation data set, and calculate the output overtopping amount based on the recognition results and the elevation data set.

9. An electronic device, characterized in that: include: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the overtopping amount identification method based on image recognition and segmented calculation according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute the overtopping amount identification method based on image recognition and segmented calculation according to any one of claims 1 to 7.

Images