Method for evaluating influence of internal solitary wave on bubble plume motion based on digital image

Abstract

The invention provides a digital image-based method for evaluating the influence of internal solitary waves on bubble plume motion. The method comprises the following steps of: S1, acquiring and processing experimental data; s2, constructing a bubble plume characteristic parameter extraction model under the action of the internal solitary wave; s3, visualization of a bubble plume motion state and a behavior velocity field under the action of an internal solitary wave; s4, establishing a space-time relation between the bubble plume motion and the internal solitary wave action; scientific evaluation of the influence of the internal solitary wave on the bubble plume motion is realized. A plurality of time-space key points that the internal solitary waves influence the motion state and behavior change of the bubble plume are selected in a targeted manner, and in order to solve the problem of extraction of bubble motion characteristic parameters in the bubble plume under the action of the internal solitary waves, a neural network model method is adopted to identify and generate a mask of the bubble. And the overall velocity field of the bubble plume is visualized to describe the differential change of the movement velocity, form and behavior of the bubbles in the bubble plume at different moments and different positions under the influence of the internal solitary waves.

Description

Technical Field

[0001] The present invention relates to the fields of marine ocean observation technology and marine engineering geological technology, and in particular to a method for evaluating the influence of internal solitary waves on bubble plume motion based on digital images. Background Art

[0002] In recent years, researchers have found that bubble plumes often escape from submarine cold springs, mud volcano sediments, and methane hydrate decomposition areas. More than 80% of these bubble plumes are methane and carbon dioxide, and the rest are composed of hydrogen sulfide, nitrogen, and oxygen. Studies have estimated that the global ocean-atmosphere methane flux is in the range of 2.2 to 6.3 Tg / year, and is concentrated in shallow sea areas and continental margins. Carbon dioxide and some methane in the bubble plume will dissolve in seawater, change the chemical composition of the ocean, and affect the ocean acidification process. The methane gas that escapes the sea surface and enters the atmosphere will have an important impact on global climate change.

[0003] Internal solitary waves are nonlinear waves that can propagate over long distances in the ocean while maintaining their waveform and wave speed. They occur frequently, carry huge amounts of energy, and are widely distributed in the world's oceans. Studies have shown that internal solitary waves can produce strong disturbance effects in local water bodies by enhancing the unsteady flow and shear stress of water bodies, changing the motion state and motion behavior of bubble plumes in water bodies, thereby affecting the ocean acidification process and exacerbating the global greenhouse effect. Currently, most studies on internal solitary waves focus on the formation and propagation characteristics of internal solitary waves and their impact on the upper marine ecosystem, while studies on the impact of internal solitary waves on the motion state and behavior of submarine bubble plumes in water bodies have not yet been clarified.

[0004] In recent years, with the advancement of digital image technology, neural network models have been introduced in image processing, which has powerful image recognition and segmentation capabilities and is widely used in many fields. Digital image technology can accurately detect bubble plumes escaping into the water body and extract their characteristic parameters. Based on the method of digital image processing, the study of the influence of internal solitary waves on the motion state and motion behavior of bubble plumes can be realized. It is of great significance to explore the influence of internal solitary waves on the motion state and behavior of bubble plumes. The present invention will reveal the complex motion patterns and scientific laws of bubble plumes under the action of internal solitary waves, and provide a solid scientific basis for responding to future climate change and protecting marine ecosystems. Summary of the invention

[0005] In order to make up for the deficiencies of the prior art, the present invention provides a method for evaluating the influence of internal solitary waves on bubble plume motion based on digital images.

[0006] The present invention is implemented by the following technical solution: a method for evaluating the influence of internal solitary waves on bubble plume motion based on digital images, specifically comprising the following steps: S1. Acquisition and processing of experimental data The acquired experimental data are processed to extract the bottom flow velocity under the action of internal solitary waves, reconstruct the background model of the bubble plume motion video, extract the foreground bubble plume using the calculation formula, perform denoising, morphological processing and binarization on the motion foreground, and construct a sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of internal solitary waves; S2. Construct a model for extracting characteristic parameters of bubble plume under the action of internal solitary waves Based on the different types of bubble plumes and corresponding motion states and behavior sample libraries under the action of internal solitary waves, the improved Mask R-CNN neural network model is used to segment the bubbles pixel by pixel to generate masks, and a bubble feature parameter extraction model under the action of internal solitary waves is constructed; the input of the extraction model is the preprocessed bubble plume image, and the output of the extraction model is the characteristic parameters of the bubble plume under the action of internal solitary waves; specifically: the centroid coordinates, major axis, minor axis and area of ​​the bubbles in the bubble plume under the action of internal solitary waves. The mask generation method is as follows

[0007] Where: is the classification result of each pixel of the mask; is the activation function; is the weight matrix of the network; is the bias, It is the pixel information of the image; S3. Visualization of the motion state and behavior velocity field of bubble plume under the action of internal solitary waves: Based on the sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of the internal solitary wave, the PWC-Net convolutional network system based on the CNN convolutional neural network is fine-tuned, and the pixel-level motion between the continuous image sequences of the bubble plume is calculated by pixel-by-pixel optical flow estimation, and the overall velocity field of the bubble plume is evaluated; It is necessary to determine the optical flow estimation method for bubble plume sequence images. The constraint equation

[0008] is the pixel value of the first frame; is the optical flow in the second frame The pixel value after displacement; Taylor expansion is performed to obtain the optical flow constraint equation as follows

[0009] and For the image and gradients over time; Optimize the constraint equation and determine the minimized loss function to solve the optical flow. The cost function is as follows

[0010] The network architecture of PWC-Net includes two fixed parameter layers consisting of distortion and cost volume, and three trainable parameter layers consisting of feature extraction, velocity field estimator and context network; two consecutive original images are inserted into the feature extraction layer of the convergent convolution network with n levels to obtain "features" of different resolutions, that is, the multiplication of each convolution filter; at the lowest resolution feature, the cost volume layer and the velocity field estimator evaluate the sketch of the velocity field, and finally convert it into velocity field data through the context layer; subsequently, the version of the velocity field data is updated to the next layer and used to deform one of the two features to achieve the prediction of the velocity field; S4. Establishing the spatial and temporal connection between bubble plume motion and internal solitary wave action On the basis of the establishment of the above model, the sequence images of the bubble plume motion at each moment and the bottom flow velocity data caused by the internal solitary wave obtained by the high-speed camera and acoustic Doppler current meter are subjected to the above processing process to analyze the changes in the escape motion of the bubble plume under the action of the internal solitary wave with the change of time and space, and to establish the spatiotemporal connection between the motion state and behavior of different types of bubble plumes and the action of the internal solitary wave.

[0011] As a preferred solution, the method for extracting the bottom flow velocity under the action of the internal solitary wave in step S1 specifically includes the following steps: recording the measured flow velocity components in the east-west, north-south and vertical directions observed by ADCP, and then subtracting the background flow velocity, that is, the average value of the flow velocities in these three directions 10 minutes before the internal solitary wave occurs. Therefore, the bottom flow velocity calculation formula induced by the internal solitary wave is:

[0012]

[0013]

[0014]

[0015] in, , and are the east-west, north-south and vertical components of the flow velocity, ;yes , and are the east-west, north-south and vertical components of the measured flow observed by ADCP; , and They are the east-west, north-south and vertical components of the background flow, respectively.

[0016] As a preferred solution, the method for reconstructing the bubble plume motion video background model in step S1 specifically includes the following steps: Before Assumption The video image of the frame is , after converting these images into grayscale images, calculate the median of each pixel position and sort them to get a new background model , where the calculation formula at each pixel position is

[0017] Then, the moving bubbles in the foreground are extracted by background subtraction, and the foreground area is detected by calculating the difference between the current frame and the background model. The specific formula is:

[0018] Set the gray value of the current frame Gray value of the background model Threshold It is set to 30. When the difference between the two is greater than the threshold, it is considered to be the foreground, otherwise it is the background.

[0019] As a preferred solution, the noise reduction, morphological processing and binarization of the moving foreground in step S1 specifically include the following steps: A 5×5 window is selected for median filtering to process the foreground mask, the median of all pixels in the neighborhood of each pixel in the image is calculated, and the median is used as the new value of the pixel to remove noise in the image. The formula is:

[0020] After correctly identifying the moving bubbles as the moving foreground and denoising, the hole filling formula is used to initially fill the holes in the foreground area and remove noise less than 50 pixels; in the binarization process, morphological operations are performed on the foreground bubbles, first performing a dilation operation to expand the foreground area and fill the small holes in the foreground. The formula is

[0021] Further erosion operation will reduce the foreground area and remove small foreground objects and noise. The formula is:

[0022] A is a binary image and B is a structural element. Finally, binarization is performed to completely separate the foreground and background. The formula is: .

[0023] As a preferred solution, the Mask R-CNN neural network model in step S2 marks adjacent pixels with the same label value through region marking, thereby separating different bubbles, and performs connectivity analysis through a formula to mark complete individual bubbles, as shown below:

[0024] in is the label of pixel (i, j), and connected indicates the connection between pixels. The geometric parameters of each object are extracted from the labeled image, including the center of mass, the length of the major axis, the length of the minor axis, and the area. The center of mass of the bubble is calculated by the weighted average of the pixel coordinates. The formula is: ,

[0025] in is the number of pixels per bubble, The calculated center of mass for the bubble; The principal component analysis method is based on the contour point set of the object, which determines the main direction of the data by calculating the eigenvalues ​​and eigenvectors of the covariance matrix. Calculated principal axis direction and secondary axis direction The corresponding eigenvalues ​​are and That is, the length of the primary and secondary axes, the specific formula is

[0026] in is the covariance matrix of the image contour point set. eigvals calculates the eigenvalues ​​of the covariance matrix. By calculating, the length of the main circumference and the length of the secondary axis can be determined. Find the area of ​​the bubble.

[0027] Furthermore, after obtaining the mask of each bubble on the image in step S1, multiple masks are merged into an overall mask to form a final mask representing all bubbles. In the merged mask, the region of multiple bubbles will be displayed as 1 and the background as 0, which is convenient for comparative analysis of bubble parameters on an image.

[0028] As a preferred solution, the lowest resolution 7th layer correlation field in step S3 is evaluated by the partial cost volume function, the formula is

[0029] in and Indicates the feature and its length, the superscript and subscript represent the image order and the level of the feature respectively; The filled correlation field is converted into the first version of the coarser velocity field through a series of convolution filters; in the next level correlation field, the two features in the feature extraction are Input into the cost volume, the second feature is pre-warped to improve accuracy, the formula is

[0030] in is a pixel position with a fixed depth, The velocity field obtained; Combine the layer 7 correlation field with the first feature And the sampled velocity field Transformed into the sixth-layer velocity field through the velocity field estimator and the context network, the process is repeated until the third-layer velocity field is generated and upsampled by bilinear interpolation to have the same resolution as the bubble image; The training loss during fine-tuning is as follows

[0031] in , and denote the pixel location, feature level, and the set of each learnable parameter contained in the feature extractor, velocity estimator, and context network, respectively; and Indicates The predicted velocity field and the supervised velocity field at the feature level; weight They correspond to 0.005, 0.01, 0.02, 0.08 and 0.32 respectively. 3, 4, 5, 6 and 7; Corresponds to Norm, trade-off weight .

[0032] Due to the adoption of the above technical solutions, the present invention has the following beneficial effects compared with the prior art: The present invention utilizes digital image processing technology and the multi-dimensional feature extraction and dynamic analysis capabilities of the neural network model to achieve a scientific evaluation of the impact of internal solitary waves on the motion of the bubble plume. The present invention specifically selects multiple spatiotemporal key points where the internal solitary wave affects the motion state and behavior changes of the bubble plume. In order to solve the problem of extracting the characteristic parameters of bubble motion in the bubble plume under the action of the internal solitary wave, a neural network model method is used to identify and generate a bubble mask. The present invention also visualizes the overall velocity field of the bubble plume to depict the differentiated changes in the motion speed, morphology and behavior of bubbles in the bubble plume at different times and positions under the influence of the internal solitary wave.

[0033] Additional aspects and advantages of the present invention will become apparent from the following description or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 It is the technical roadmap of the present invention; Figure 2 This is a diagram of bubble plume preprocessing at different times under the action of the internal solitary wave of the present invention; Figure 3 This is a case diagram of extracting characteristic parameters of bubble plume under the action of internal solitary waves described in the present invention; Figure 4 It is a schematic diagram of the overall velocity field of the bubble plume before the wave trough passes under the action of the visualized internal solitary wave of the present invention; Figure 5 It is a schematic diagram of the overall velocity field of the bubble plume when the overall velocity field of the bubble plume passes through the wave trough under the action of the visualized internal solitary wave described in the present invention; Figure 6 It is a schematic diagram of the overall velocity field of the bubble plume after the bubble plume passes through the wave trough under the action of the visualized internal solitary wave according to the present invention; Figure 7 It is a schematic diagram of the change of the bubble plume velocity before the wave trough passes under the action of the internal solitary wave of the present invention; Figure 8 It is a schematic diagram of the change of the bubble plume velocity when the bubble plume passes through the wave trough under the action of the internal solitary wave of the present invention; Fig. 9 It is a schematic diagram of the change in velocity of the bubble plume after passing through the trough under the action of the internal solitary wave of the present invention. DETAILED DESCRIPTION

[0035] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0036] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0037] Combine the following Figures 1 to 5The method for evaluating the influence of internal solitary waves on bubble plume motion based on digital images according to an embodiment of the present invention is described in detail.

[0038] like Figure 1 As shown, the present invention proposes a method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images, which specifically includes the following steps: S1. Acquisition and processing of experimental data The acquired experimental data are processed to extract the bottom flow velocity under the action of internal solitary waves, reconstruct the background model of the bubble plume motion video, extract the foreground bubble plume using the calculation formula, perform denoising, morphological processing and binarization on the motion foreground, and construct a sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of internal solitary waves; The method for extracting the bottom velocity under the action of internal solitary waves specifically includes the following steps: recording the measured flow velocity components in the east-west, north-south and vertical directions observed by ADCP, and then subtracting the background velocity, which is the average value of the flow velocities in these three directions 10 minutes before the internal solitary wave occurs. Therefore, the bottom velocity calculation formula induced by the internal solitary wave is:

[0039]

[0040]

[0041]

[0042] in, , and are the east-west, north-south and vertical components of the flow velocity, ;yes , and are the east-west, north-south and vertical components of the measured flow observed by ADCP; , and They are the east-west, north-south and vertical components of the background flow, respectively.

[0043] The method for reconstructing the bubble plume motion video background model specifically comprises the following steps: Before Assumption The video image of the frame is , after converting these images into grayscale images, calculate the median of each pixel position and sort them to get a new background model , where the calculation formula at each pixel position is

[0044] Then, the moving bubbles in the foreground are extracted by background subtraction, and the foreground area is detected by calculating the difference between the current frame and the background model. The specific formula is:

[0045] Set the gray value of the current frame Gray value of the background model Threshold It is set to 30. When the difference between the two is greater than the threshold, it is considered to be the foreground, otherwise it is the background.

[0046] The denoising, morphological processing and binarization of the moving foreground specifically include the following steps: A 5×5 window is selected for median filtering to process the foreground mask, the median of all pixels in the neighborhood of each pixel in the image is calculated, and the median is used as the new value of the pixel to remove noise in the image. The formula is:

[0047] After correctly identifying the moving bubbles as the moving foreground and denoising, the hole filling formula is used to initially fill the holes in the foreground area and remove noise less than 50 pixels; in the binarization process, morphological operations are performed on the foreground bubbles, first performing a dilation operation to expand the foreground area and fill the small holes in the foreground. The formula is

[0048] Further erosion operation will reduce the foreground area and remove small foreground objects and noise. The formula is:

[0049] A is a binary image and B is a structural element. Finally, binarization is performed to completely separate the foreground and background. The formula is: .

[0050] S2. Construct a model for extracting characteristic parameters of bubble plume under the action of internal solitary waves Based on the different types of bubble plumes and corresponding motion states and behavior sample libraries under the action of internal solitary waves, the improved Mask R-CNN neural network model is used to segment the bubbles pixel by pixel to generate masks, and a bubble feature parameter extraction model under the action of internal solitary waves is constructed; the input of the extraction model is the preprocessed bubble plume image, and the output of the extraction model is the characteristic parameters of the bubble plume under the action of internal solitary waves; specifically: the centroid coordinates, major axis, minor axis and area of ​​the bubbles in the bubble plume under the action of internal solitary waves. The mask generation method is as follows

[0051] Where: is the classification result of each pixel of the mask; is the activation function; is the weight matrix of the network; is the bias, It is the pixel information of the image; The mask generation process of the Mask R-CNN neural network model consists of three parts: feature extraction, boundary positioning, and mask output. Feature extraction refers to the use of multi-layer convolution operations to extract local area features from the original image, and the maximum pooling is used to reduce the spatial dimension but retain the most important feature information; boundary positioning refers to the use of the region proposal network to generate the region border that may contain objects, and the use of bilinear interpolation and fully connected layers to improve the accuracy of feature information and correct the position and size of the region border; mask output refers to the use of the full convolution network to output a A binary mask of size indicating the precise shape of the object within the region.

[0052] The Mask R-CNN neural network model uses region labeling to mark adjacent pixels with the same label value, thereby separating different bubbles, and uses a formula to perform connectivity analysis to mark complete individual bubbles. The specific details are as follows:

[0053] in is the label of pixel (i, j), and connected indicates the connection between pixels. The geometric parameters of each object are extracted from the labeled image, including the center of mass, the length of the major axis, the length of the minor axis, and the area. The center of mass of the bubble is calculated by the weighted average of the pixel coordinates. The formula is: ,

[0054] in is the number of pixels per bubble, The calculated center of mass for the bubble; The principal component analysis method is based on the contour point set of the object, which determines the main direction of the data by calculating the eigenvalues ​​and eigenvectors of the covariance matrix. Calculated principal axis direction and secondary axis direction The corresponding eigenvalues ​​are and That is, the length of the primary and secondary axes, the specific formula is

[0055] in is the covariance matrix of the image contour point set. eigvals calculates the eigenvalues ​​of the covariance matrix. By calculating, the length of the main circumference and the length of the secondary axis can be determined. Find the area of ​​the bubble.

[0056] After obtaining the mask of each bubble on the image in step S1, multiple masks are merged into an overall mask to form a final mask representing all bubbles. In the merged mask, the region of multiple bubbles will be displayed as 1 and the background as 0, which is convenient for comparative analysis of bubble parameters on an image.

[0057] S3. Visualization of the motion state and behavior velocity field of bubble plume under the action of internal solitary waves Based on the sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of the internal solitary wave, the PWC-Net convolutional network system based on the CNN convolutional neural network is fine-tuned, and the pixel-level motion between the continuous image sequences of the bubble plume is calculated by pixel-by-pixel optical flow estimation, and the overall velocity field of the bubble plume is evaluated; It is necessary to determine the optical flow estimation method for bubble plume sequence images. The constraint equation

[0058] is the pixel value of the first frame; is the optical flow in the second frame The pixel value after displacement; Taylor expansion is performed to obtain the optical flow constraint equation as follows

[0059] and For the image and gradients over time; Optimize the constraint equation and determine the minimized loss function to solve the optical flow. The cost function is as follows

[0060] The network architecture of PWC-Net includes two fixed parameter layers consisting of distortion and cost volume, and three trainable parameter layers consisting of feature extraction, velocity field estimator and context network; two consecutive original images are inserted into the feature extraction layer of the convergent convolution network with n levels to obtain "features" of different resolutions, that is, the multiplication of each convolution filter; at the lowest resolution feature, the cost volume layer and the velocity field estimator evaluate the sketch of the velocity field, which is finally converted into velocity field data through the context layer; subsequently, this coarser version of the velocity field data is updated to the next layer and used to deform one of the two features to achieve better prediction of the velocity field; The lowest resolution layer 7 correlation field is evaluated by the partial cost volume function, as

[0061] in and Indicates the feature and its length, the superscript and subscript represent the image order and the level of the feature respectively; The filled correlation field is converted into the first version of the coarser velocity field through a series of convolution filters; in the next level correlation field, the two features in the feature extraction are Input into the cost volume, the second feature is pre-warped to improve accuracy, the formula is

[0062] in is a pixel position with a fixed depth, The velocity field obtained; Combine the layer 7 correlation field with the first feature And the sampled velocity field Transformed into the sixth-layer velocity field through the velocity field estimator and the context network, the process is repeated until the third-layer velocity field is generated and upsampled by bilinear interpolation to have the same resolution as the bubble image; The training loss during fine-tuning is as follows

[0063] in , and denote the pixel location, feature level, and the set of each learnable parameter contained in the feature extractor, velocity estimator, and context network, respectively; and Indicates The predicted velocity field and the supervised velocity field at the feature level; weight They correspond to 0.005, 0.01, 0.02, 0.08 and 0.32 respectively. 3, 4, 5, 6 and 7; Corresponds to Norm, trade-off weight .

[0064] S4. Establishing the spatial and temporal connection between bubble plume motion and internal solitary wave action On the basis of the establishment of the above model, the sequence images of the bubble plume motion at each moment and the bottom flow velocity data caused by the internal solitary wave obtained by the high-speed camera and acoustic Doppler current meter are subjected to the above processing process to analyze the changes in the escape motion of the bubble plume under the action of the internal solitary wave with the change of time and space, and to establish the spatiotemporal connection between the motion state and behavior of different types of bubble plumes and the action of the internal solitary wave.

[0065] Take the observation results of the internal solitary wave affecting the movement of bubble plume in an organic water tank conducted by this research team in September 2024 as an example. The observation equipment is an acoustic Doppler flow meter and a high-speed camera. The flow rate monitoring process is 15 minutes, the acquisition frequency is 32HZ, a total of 28,800 groups of numbers, each group has 16 numbers arranged horizontally. The acquisition frequency of the high-speed camera is 1000HZ, a total of 20,000 bubble plume movement sequence images.

[0066] First follow Figure 1 The technical route shown in the figure is used to preprocess the data, extract the bottom flow velocity under the action of internal solitary waves, complete the classification correspondence between the bottom flow velocity and the motion state of the bubble plume, and establish a sample library. Then, the bubble plume characteristic parameter extraction model is used to obtain the centroid coordinates, major axis, minor axis and area of ​​the bubbles in the bubble plume. Figure 3 shown.

[0067] After fine-tuning the convolutional network system, the visualization is as follows Figure 4 The overall velocity field of the bubble plume is shown, and the temporal and spatial variability of the bubble plume motion under the action of internal solitary waves in the overall velocity field of the bubble plume is analyzed. Combined with the changes in the bubble characteristic parameters, the temporal and spatial connection between the bubble plume motion speed and behavior and the internal solitary wave action is analyzed and established.

[0068] It is summarized that in this example, the center-of-mass velocity of the internal solitary wave undergoes a sudden change when the trough of the internal solitary wave passes through the bubble plume; the influence of the internal solitary wave on the horizontal velocity of the bubble plume is several times that of its vertical velocity, which is the main reason for the obvious change in the center-of-mass velocity of the bubble; the change in the horizontal velocity of the bubble plume is identical to the change in the bottom flow velocity caused by the internal solitary wave.

[0069] Before and after the trough of the inner solitary wave passes, the centroid velocity of the upper bubble plume is greater than that of the middle bubble plume, and the horizontal velocity change of the upper bubble plume is more obvious. When the trough of the inner solitary wave passes, the velocity of the middle bubble exceeds that of the upper bubble, and the horizontal velocity change of the middle bubble is greater than that of the upper bubble.

[0070] After the internal solitary wave starts to move, the bubble plume trajectory begins to deviate and becomes curved, and the curve is at its maximum when the internal solitary wave passes through the trough. During the overall movement of the bubble, the deflection reaches its maximum after the bubble passes through the boundary layer; the bubble deflection caused by the internal solitary wave is identical to the change in the horizontal velocity of the bubble.

[0071] In the description of the present invention, the term "plurality" refers to two or more than two. Unless otherwise clearly defined, the orientation or positional relationship indicated by the terms "upper" and "lower" is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation of the present invention; the terms "connection", "installation", "fixation", etc. should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0072] In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiments", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0073] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images, characterized in that , specifically including the following steps: S1. Acquisition and processing of experimental data The acquired experimental data are processed to extract the bottom flow velocity under the action of internal solitary waves, reconstruct the background model of the bubble plume motion video, extract the foreground bubble plume using the calculation formula, perform denoising, morphological processing and binarization on the motion foreground, and construct a sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of internal solitary waves; S2. Construct a model for extracting characteristic parameters of bubble plume under the action of internal solitary waves Based on the different types of bubble plumes and corresponding motion states and behavior sample libraries under the action of internal solitary waves, the improved Mask R-CNN neural network model is used to segment the bubbles pixel by pixel to generate masks, and a bubble feature parameter extraction model under the action of internal solitary waves is constructed; the input of the extraction model is the preprocessed bubble plume image, and the output of the extraction model is the characteristic parameters of the bubble plume under the action of internal solitary waves; specifically: the centroid coordinates, major axis, minor axis and area of ​​the bubbles in the bubble plume under the action of internal solitary waves. The mask generation method is as follows Where: is the classification result of each pixel of the mask; is the activation function; is the weight matrix of the network; is the bias, It is the pixel information of the image; S3. Visualization of the motion state and behavior velocity field of bubble plume under the action of internal solitary waves Based on the sample library of different types of bubble plumes and corresponding motion states and behaviors under the action of the internal solitary wave, the PWC-Net convolutional network system based on the CNN convolutional neural network is fine-tuned, and the pixel-level motion between the continuous image sequences of the bubble plume is calculated by pixel-by-pixel optical flow estimation, and the overall velocity field of the bubble plume is evaluated; It is necessary to determine the optical flow estimation method for bubble plume sequence images. The constraint equation is the pixel value of the first frame; is the optical flow in the second frame The pixel value after displacement; Taylor expansion is performed to obtain the optical flow constraint equation as follows and For the image and gradients over time; Optimize the constraint equation and determine the minimized loss function to solve the optical flow. The cost function is as follows The network architecture of PWC-Net includes two fixed parameter layers consisting of distortion and cost volume, and three trainable parameter layers consisting of feature extraction, velocity field estimator and context network; two consecutive original images are inserted into the feature extraction layer of the convergent convolution network with n levels to obtain "features" of different resolutions, that is, the multiplication of each convolution filter; at the lowest resolution feature, the cost volume layer and the velocity field estimator evaluate the sketch of the velocity field, which is finally converted into velocity field data through the context layer; subsequently, the version of the velocity field data is updated to the next layer and used to deform one of the two features to achieve the prediction of the velocity field; S4. Establishing the spatial and temporal connection between bubble plume motion and internal solitary wave action On the basis of the establishment of the above model, the sequence images of the bubble plume motion at each moment and the bottom flow velocity data caused by the internal solitary wave obtained by the high-speed camera and acoustic Doppler current meter are subjected to the above processing process to analyze the changes in the escape motion of the bubble plume under the action of the internal solitary wave with the change of time and space, and to establish the spatiotemporal connection between the motion state and behavior of different types of bubble plumes and the action of the internal solitary wave.

2. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 1 is characterized in that The method for extracting the bottom flow velocity under the action of the internal solitary wave in step S1 specifically includes the following steps: recording the measured flow velocity components in the east-west, north-south and vertical directions observed by ADCP, and then subtracting the background flow velocity, that is, the average value of the flow velocities in these three directions 10 minutes before the internal solitary wave occurs. Therefore, the bottom flow velocity calculation formula induced by the internal solitary wave is: in, , and are the east-west, north-south and vertical components of the flow velocity, It is the total flow velocity synthesized by the flow velocity components in these three directions; , and are the east-west, north-south and vertical components of the measured flow observed by ADCP; , and They are the east-west, north-south and vertical components of the background flow, respectively.

3. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 1 is characterized in that The method for reconstructing the bubble plume motion video background model in step S1 specifically includes the following steps: Before Assumption The video image of the frame is , after converting these images into grayscale images, calculate the median of each pixel position and sort them to get a new background model , where the calculation formula at each pixel position is Then, the moving bubbles in the foreground are extracted by background subtraction, and the foreground area is detected by calculating the difference between the current frame and the background model. The specific formula is: The gray value of the current frame Gray value of the background model Threshold It is set to 30. When the difference between the two is greater than the threshold, it is considered to be the foreground, otherwise it is the background.

4. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 1 is characterized in that , the noise reduction, morphological processing and binarization of the moving foreground in step S1 specifically include the following steps: A 5×5 window is selected for median filtering to process the foreground mask, the median of all pixels in the neighborhood of each pixel in the image is calculated, and the median is used as the new value of the pixel to remove noise in the image. The formula is: After correctly identifying the moving bubbles as the moving foreground and denoising, the hole filling formula is used to initially fill the holes in the foreground area and remove noise less than 50 pixels; in the binarization process, morphological operations are performed on the foreground bubbles, first performing a dilation operation to expand the foreground area and fill the small holes in the foreground. The formula is Further erosion operation will reduce the foreground area and remove small foreground objects and noise. The formula is: A is a binary image and B is a structural element. Finally, binarization is performed to completely separate the foreground and background. The formula is: 。 5. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 1 is characterized in that In step S2, the Mask R-CNN neural network model marks adjacent pixels with the same label value through region marking, thereby separating different bubbles, and performs connectivity analysis through the formula to mark the complete individual bubbles, as shown below: in is the label of pixel (i,j), connected indicates the connection between pixels; The geometric parameters of each object are extracted from the marked image, including the center of mass, major axis length, minor axis length and area; the center of mass of the bubble is calculated by weighted average of pixel coordinates, and the formula is: ， in is the number of pixels per bubble, The calculated center of mass for the bubble; The principal component analysis method is based on the contour point set of the object, which determines the main direction of the data by calculating the eigenvalues ​​and eigenvectors of the covariance matrix. Calculated principal axis direction and secondary axis direction The corresponding eigenvalues ​​are and That is, the length of the primary and secondary axes, the specific formula is in is the covariance matrix of the image contour point set. eigvals calculates the eigenvalues ​​of the covariance matrix. By calculating, the length of the main circumference and the length of the secondary axis can be determined. Find the area of ​​the bubble.

6. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 5 is characterized in that In the step S1, after obtaining the mask of each bubble on the image, multiple masks are merged into an overall mask to form a final mask representing all bubbles; in the merged mask, the area of ​​multiple bubbles will be displayed as 1 and the background will be 0, which is convenient for comparative analysis of bubble parameters on an image.

7. The method for evaluating the impact of internal solitary waves on bubble plume motion based on digital images according to claim 1 is characterized in that ,The 7th layer correlation field with the lowest resolution in step S3 is evaluated by the partial cost volume function, and the formula is in and Indicates the feature and its length, the superscript and subscript represent the image order and the level of the feature respectively; The filled correlation field is converted into the first version of the coarser velocity field through a series of convolution filters; in the next level correlation field, the two features in the feature extraction are Input into the cost volume, the second feature is pre-warped to improve accuracy, the formula is in is the pixel position with fixed depth, The velocity field obtained; Combine the layer 7 correlation field with the first feature And the sampled velocity field Transformed into the sixth-layer velocity field through the velocity field estimator and the context network, the process is repeated until the third-layer velocity field is generated and upsampled by bilinear interpolation to have the same resolution as the bubble image; The training loss during fine-tuning is as follows in , and denote the pixel location, feature level, and the set of each learnable parameter contained in the feature extractor, velocity estimator, and context network, respectively; and Indicates The predicted velocity field and the supervised velocity field at the feature level; weight They correspond to 0.005, 0.01, 0.02, 0.08 and 0.32 respectively. 3, 4, 5, 6 and 7; Corresponds to Norm, trade-off weight .

Images