Method for extracting dynamic characteristics of water column

Abstract

The invention provides a method for extracting dynamic characteristics of a water column. The method comprises the following steps of: 1, acquiring a video frame containing a water column target to be detected; 2, carrying out dynamic feature extraction on the video frame, and obtaining the duration, the energy difference ratio and the average growth rate of the water column target to be detected; and step 3, judging whether the water column target to be detected is a real water column or not according to the duration time, the energy difference ratio and the average growth rate. According to the method, dynamic parameters such as the duration time, the energy difference ratio and the average growth rate of a water column are comprehensively considered, and accurate detection of a water column target can be achieved. Meanwhile, the method can also be combined with a support vector machine to make a more comprehensive decision judgment. In addition, the method can also be used as a component of a water column target detection frame, the overall detection precision is effectively improved, and the probability of missing detection, false detection and repeated detection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of target detection, and particularly to a method for extracting dynamic features of water columns. Background Art

[0002] Effectively detecting and tracking water columns using visible light images is the core of realizing automatic detection of sea water columns. The current water column detection methods in the art only rely on static features, which limits the ability of the detection algorithm to resist interference factors in complex environments. For example, objects with shapes similar to water columns such as water splashes and water waves, as well as situations such as multiple water columns intersecting and overlapping, and invalid water columns generated by ricochets, may all interfere with the detection results. Water columns have a clear life cycle and will go through the processes of formation, stability, decline, and disappearance. Making full use of their dynamic features can significantly reduce the probabilities of missed detection, false detection, and repeated detection.

[0003] In view of this, a method is needed to realize the extraction of dynamic features of water columns, thereby reducing the situations of missed detection, false detection, and repeated detection of water columns and improving the accuracy of water column detection. Summary of the Invention

[0004] The present invention proposes a method for extracting dynamic features of water columns, aiming to overcome the problems of missed detection, false detection, and repeated detection caused by only relying on static features for water column detection in the prior art, thereby improving the accuracy of water column detection.

[0005] The present invention provides a method for extracting dynamic features of water columns, including: Step 1, obtaining a video frame containing the water column target to be detected; Step 2, extracting dynamic features from the video frame to obtain the duration, energy difference ratio, and average growth rate of the water column target to be detected; Step 3, making a judgment based on the duration, energy difference ratio, and average growth rate to determine whether the water column target to be detected is a real water column.

[0006] As a specific implementation manner of the present invention, in Step 2, obtaining the energy difference ratio of the water column target to be detected further includes: Step 21, obtaining the current video frame, dividing the current video frame into multiple blocks of a fixed size, and obtaining the current frame energy of each block through calculation; Step 22, collecting historical video frames in the past period of time, dividing each historical video frame into multiple blocks of a fixed size, and estimating the background energy of each block through a background model; Step 23, calculating the energy difference ratio of the water column according to the current frame energy and background energy of each block, and the calculation formula is: wherein, represents at In the video frame at a moment, the block 's current frame energy; Indicates that in the video frame at moment, the block 's background energy.

[0007] Among them, step 21 further includes: Step 211, perform discrete wavelet transform on the current video frame, and decompose it into a compressed image, a horizontal coefficient image , a vertical coefficient image and a diagonal coefficient image ; Step 212, according to the horizontal coefficient image , the vertical coefficient image and the diagonal coefficient image , calculate the current frame energy of each block, and the calculation formula is: Among them, is the current video frame, , and are the vertical, horizontal and diagonal coefficient images respectively.

[0008] Among them, step 22 further includes: Step 221, construct a background model ; Step 222, perform discrete wavelet transform on the historical video frame, and decompose it into a compressed image, a horizontal coefficient image , a vertical coefficient image and a diagonal coefficient image ; Step 223, according to the horizontal coefficient image , the vertical coefficient image and the diagonal coefficient image , calculate the current frame energy of each block in the background model , and the calculation formula is: Among them, is the background model, , and are the vertical, horizontal and diagonal coefficient images respectively.

[0009] As a specific implementation manner of the present invention, in step 2, obtaining the energy difference ratio of the water column target to be detected further includes: Step 21, obtain the pixel coordinates of the water column ; Step 22, convert the pixel coordinates to the image coordinates , and the conversion formula is: where: and respectively represent , the focal lengths in the and directions, and Step 22, convert the height and length of the water column in the pixel coordinate system to the height and length in the camera coordinate system , and the conversion formula is: Step 23, calculate the normalized area S of the water column in the camera coordinate system, and the calculation formula is: Step 24, calculate the average growth rate of the water column, and the calculation formula is: where, , respectively represent the normalized areas of the water column in the pixel coordinate system and the camera coordinate system at time, and respectively represent , the focal lengths in the directions at time, is the number of frames at time, is the number of area sequences,

[0010] wherein, Step 24 further includes: For a camera with a fixed focal length, the calculation formula can be simplified to: where: represents the normalized area of the water column in the pixel coordinate system at time, is the number of frames at time, is the number of area sequences,

[0011] As a specific embodiment of the present invention, Step 3 further includes: Duration determination: Determine the duration corresponding to the water column target to be detected according to a preset duration threshold. When the duration is greater than or equal to the preset duration threshold, retain the water column target to be detected; when the duration is less than the preset duration threshold, delete the water column target to be detected. Energy difference ratio determination: Determine the energy difference ratio corresponding to the water column target to be detected according to a preset energy difference ratio threshold. When the energy difference ratio is greater than or equal to the preset energy difference ratio threshold, retain the water column target to be detected; when the energy difference ratio is less than the preset energy difference ratio threshold, delete the water column target to be detected. Average growth rate determination: Determine the average growth rate corresponding to the water column target to be detected according to a preset average growth rate threshold. When the average growth rate is greater than or equal to the preset average growth rate threshold, retain the water column target to be detected; when the average growth rate is less than the preset average growth rate threshold, delete the water column target to be detected. Water column target determination: After performing any one or more of the duration determination, energy difference ratio determination, and average growth rate determination on the water column target to be detected, determine the retained water column target to be detected as a real water column.

[0012] As another specific embodiment of the present invention, step 3 further includes: Step 31: Construct a support vector machine and select a radial basis kernel function as the kernel function of the support vector machine. Step 32: In the support vector machine, construct a mapping relationship between the duration, energy difference ratio, and average growth rate and whether the water column target is a real water column. The mapping function is: Wherein, is a binary variable representing the tracking target whether it is a water column, represents the duration, k(n) represents the average growth rate, represents the energy difference ratio.

[0013] Step 33: Input video frame samples containing real water column targets and non-water column targets into the support vector machine, and train the support vector machine to approximate the relationship ; Step 34: Input the water column target to be detected and its corresponding duration, energy difference ratio, and average growth rate data into the trained support vector machine to determine whether the water column target to be detected is a real water column.

[0014] The present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. The computer program, when executed by a processor, implements any one of the above water column dynamic feature extraction methods.

[0015] The present invention also provides a computer program product, including a computer program which, when executed by a processor, implements the method for extracting the dynamic characteristics of a water column as described in any one of the above.

[0016] A method for extracting the dynamic characteristics of a water column provided by the present invention includes: first, obtaining a video frame including a water column target to be detected; then, performing dynamic feature extraction on the video frame to obtain the duration, energy difference ratio, and average growth rate of the water column target to be detected; and finally, making a judgment based on the duration, energy difference ratio, and average growth rate to determine whether the water column target to be detected is a real water column.

[0017] Specifically, the method of the present invention proposes a method for extracting the dynamic characteristics of a water column, which can comprehensively consider several dynamic characteristics such as the duration, energy difference ratio, and average growth rate of the water column, thereby avoiding problems of missed detection, false detection, and repeated detection, and improving the accuracy of water column detection. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a schematic diagram of the camera pinhole model provided by the present invention; Figure 2 It is a schematic diagram of the change trend of the area of the water column target with time provided by the present invention; Figure 3 It is a schematic diagram of the change trend of the average growth rate of the water column target with the sequence number provided by the present invention; Figure 4 It is a schematic diagram of the water column detection framework based on dynamic characteristics provided by the present invention; Figure 5 It is an effect diagram of water column detection provided by the present invention. Detailed Embodiments

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention with reference to the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments in the present invention belong to the scope of protection of the present invention.

[0021] Effectively detecting and tracking water columns using visible light images is the core of realizing automatic detection of water columns at sea. The water column detection methods in current technologies only rely on static features, which may cause interference to the detection results in cases such as objects with shapes similar to water columns like water splashes and water waves, as well as invalid water columns generated by the intersection and overlap of multiple water columns and ricochets. Therefore, a detection method capable of extracting the dynamic features of water columns is needed to improve the ability to resist interference factors in complex environments.

[0022] Generally speaking, the method of the present invention first extracts dynamic features from video frames containing the water column target to be detected, obtaining the duration, energy difference ratio, and average growth rate of the water column target to be detected; subsequently, based on the duration, energy difference ratio, and average growth rate, it is determined whether the water column target to be detected is a real water column. Through a series of algorithmic processes, the present invention can achieve precise detection of water columns, providing strong technical support for the task of evaluating the accuracy of sea landing points.

[0023] The present invention provides a method for extracting dynamic features of water columns, including: Step 1, obtaining video frames containing the water column target to be detected; Step 2, extracting dynamic features from the video frames to obtain the duration, energy difference ratio, and average growth rate of the water column target to be detected; Step 3, making a judgment based on the duration, energy difference ratio, and average growth rate to determine whether the water column target to be detected is a real water column.

[0024] As a specific implementation manner of the present invention, Step 2 further includes: First, perform discrete wavelet transform on the input image. This step decomposes the image into multiple sub-images, including the compressed image , the horizontal coefficient image , the vertical coefficient image , and the diagonal coefficient image . Then, divide the image into blocks of a fixed size , and calculate the energy of each block. The energy is obtained by summing the squared contributions of each coefficient image: where is the current input frame, , , and are the vertical, horizontal, and diagonal coefficient images respectively.

[0025] The system uses a background model to estimate the image energy of the background. This background model is updated by collecting image data over a period of time and using it. The background energy is obtained through the same discrete wavelet transform and energy calculation method as the current frame. For the background model , the background energy of each block is calculated as follows: Among them, is the background model, , and are the vertical, horizontal, and diagonal coefficient images respectively.

[0026] To measure the disappearance of the water column edge and local energy attenuation, the energy difference ratio is introduced. The energy difference ratio is calculated by comparing the energy in the current frame and the energy in the background: Among them, represents the energy of the input image in the time block ; represents the energy of the background image in the time block .

[0027] As a specific implementation manner of the present invention, step 2 further includes: To adapt to scenes with different frame rates and different focal lengths, it is necessary to convert the pixel area of the water column into the real area in the camera coordinate system, as Figure 1 shown. First, convert the pixel coordinates to the image coordinates : Among them: and respectively represent the focal lengths in the , directions, and respectively represent the pixel coordinates of the principal point.

[0028] Assume that the height and width of the water column in the pixel coordinate system are , then the corresponding height and width in the camera coordinate system can be expressed as: Then the normalized area S of the water column in the camera coordinate system can be expressed as: Define the average growth rate of the water column as: Among them: , respectively represent the pixel coordinate system and the camera coordinate system at Normalized area of the water column at a moment and respectively represent 、 the focal lengths in the direction at the moment of is the number of frames at the moment of is the number of area sequences is the video frame rate

[0029] For a camera with a fixed focal length, that is and are both constant values, then the average growth rate can be simplified as: The average growth rate evaluates the rising trend of the water column by calculating the ratio of the increment of the water column area between consecutive frames to the time increment

[0030] To verify the index, 10 suspected water column targets are selected here, and their area change curves are as shown in Figure 2 . Among them, numbers 1-4 are real water columns, and 5-10 are background waves. The average growth rates of these 10 targets are calculated respectively, as shown in Figure 3 . From the figure, it can be seen that the average growth rate of the normal water column decreases with the increase of the sequence number. This is because as time goes by, the area of the water column becomes larger, so the growth rate decreases. It should be noted that the average growth rates of the 4 normal water columns are significantly higher than those of the background waves. Therefore, based on the average growth rate, the real water columns can be effectively screened out, while reducing the cases of false detection and duplicate detection

[0031] As a specific implementation manner of the present invention, step 3 further includes: Duration determination: judging the duration corresponding to the water column target to be detected according to a preset duration threshold. When the duration is greater than or equal to the preset duration threshold, the water column target to be detected is retained; when the duration is less than the preset duration threshold, the water column target to be detected is deleted Energy difference ratio determination: judging the energy difference ratio corresponding to the water column target to be detected according to a preset energy difference ratio threshold. When the energy difference ratio is greater than or equal to the preset energy difference ratio threshold, the water column target to be detected is retained; when the energy difference ratio is less than the preset energy difference ratio threshold, the water column target to be detected is deleted Average growth rate determination: judging the average growth rate corresponding to the water column target to be detected according to a preset average growth rate threshold. When the average growth rate is greater than or equal to the preset average growth rate threshold, the water column target to be detected is retained; when the average growth rate is less than the preset average growth rate threshold, the water column target to be detected is deleted Water column target determination: After performing any one or more of the duration determination, energy difference ratio determination, and average growth rate determination on the water column target to be detected, the remaining water column target to be detected is determined as a real water column.

[0032] As another specific embodiment of the present invention, step 3 further includes: Using a Support Vector Machine (SVM) to separate and classify real water columns and pseudo water columns. Support Vector Machine is a machine learning algorithm commonly used to solve problems of limited sample size, non-linearity, and high-dimensional pattern recognition. Here, the SVM is constructed as a two-class classifier, namely the water column class and the non-water column class. To solve the problem of linear inseparability, a kernel function is introduced in the SVM to map the samples into a high-dimensional space. Considering that the radial basis kernel function can better balance the operation time and prediction effect and improve the classification speed, the radial basis kernel function is selected here. The form of the radial basis kernel function is: Where, is the parameter of the kernel function, is the vector and is the Euclidean distance between them.

[0033] Through the dynamic feature acquisition in the previous step, a feature vector of length 3 is obtained: Assuming the binary variable represents whether the tracking target is a water column, The relationship with the extracted features is defined as: Using an image training set containing water column scenarios and non-water column scenarios to train the support vector machine to approximate the relationship .

[0034] As Figure 4 shown, a method for extracting dynamic features of a water column provided by the present invention can also be applied to a water column tracking detection framework to improve the overall detection accuracy. Among them, the water column tracking detection framework can include the following four parts: Water column static feature extraction network, using YOLO, SSD, etc. as the water column static feature extraction network, and outputting the position and classification confidence information of the suspected water column; Multi-object tracker, inputting the above detection results into the ByteTrack multi-object tracker, and tracking the target through the association of upper and lower frame information to obtain the suspected water column object; Water column dynamic feature extractor, analyzing the dynamic characteristics of each suspected water column object according to the above method for extracting dynamic features of a water column; The support vector machine decision - maker comprehensively makes decisions on the information of the water column to obtain the detection result.

[0035] Since the water column is dynamically changing, during the detection process, the water column static feature extraction network may have the situation of repeatedly identifying the water column. Although the ByteTrack algorithm can track the severely occluded water column objects under complex sea conditions, it will generate new tracking trajectories for the high - score box water columns that are repeatedly identified and cannot eliminate them. Through the water column dynamic feature extractor, according to the water column dynamic feature extraction method provided by the present invention, analyze the dynamic characteristics of each suspected water column object, and through the comprehensive decision - making of the support vector machine, it helps to reduce the false detection rate of the water column tracking and detection framework.

[0036] It can be seen that the water column detection method based on dynamic features proposed by the present invention effectively utilizes several dynamic features of the water column, such as the duration, energy difference ratio, and average growth rate. Figure 5 The detection effects based on static features and dynamic features of the test video are respectively shown. The test video is shot from an aerial perspective, with a large perspective change and a large contrast in color between the water column and the background. It can be seen from the figure that due to the relatively bad sea conditions, there are many white waves on the sea surface interfering with the recognition, and the forms of some splashing waves are similar to those of the water column. It is very difficult for the detection method based on static features to exclude them. For example, the water column with ID 21 is a typical mis - identified wave; for the detection method based on dynamic features, after sufficient training, the SVM can exclude the interference of background waves and reduce the false detection rate.

[0037] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program. The computer program can be stored on a non - transient computer - readable storage medium. When the computer program is executed by a processor, the computer can execute a water column detection method provided by each of the above - mentioned methods. The method includes: Step 1, obtain a video frame containing the water column target to be detected; Step 2, perform dynamic feature extraction on the video frame to obtain the duration, energy difference ratio, and average growth rate of the water column target to be detected; Step 3, make a judgment according to the duration, energy difference ratio, and average growth rate to determine whether the water column target to be detected is a real water column.

[0038] On another aspect, the present invention also provides a non - transient computer - readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements a water column detection method provided by each of the above - mentioned methods. The method includes: Step 1, obtain a video frame containing the water column target to be detected; Step 2, perform dynamic feature extraction on the video frame to obtain the duration, energy difference ratio, and average growth rate of the water column target to be detected; Step 3, make a judgment according to the duration, energy difference ratio, and average growth rate to determine whether the water column target to be detected is a real water column.

[0039] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative labor.

[0040] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting dynamic characteristics of a water column, characterized in that: include: Step 1, obtaining a video frame containing a water column target to be detected; Step 2, extracting dynamic features from the video frame to obtain the duration, energy difference ratio and average growth rate of the water column target to be detected; Step 3: judging whether the water column target to be detected is a real water column based on the duration, energy difference ratio and average growth rate.

2. The method for extracting dynamic characteristics of a water column according to claim 1, characterized in that: In step 2, obtaining the energy difference ratio of the water column includes the following steps: Step 21, obtaining a current video frame, dividing the current video frame into a plurality of blocks of a fixed size, and obtaining current frame energy of each block; Step 22, collecting historical video frames, dividing each of the historical video frames into a plurality of blocks of a fixed size, and estimating the background energy of each block through a background model; Step 23, according to the current frame energy and background energy of each block, the energy difference ratio of the water column is calculated, and the calculation formula is: in, Indicated in In the video frame at time instant, the block The current frame energy of Indicated in In the video frame at time instant, the block background energy.

3. A method for extracting dynamic characteristics of a water column according to claim 2, characterized in that: Step 21 further comprises: Step 211, performing discrete wavelet transform on the current video frame to decompose it into a compressed image, a horizontal coefficient image , vertical factor image and the diagonal coefficient image ; Step 212: based on the horizontal coefficient image , vertical factor image and the diagonal coefficient image , calculate the current frame energy of each block, the calculation formula is: in, is the current video frame, , and are the vertical, horizontal and diagonal coefficient images respectively.

4. The method for extracting dynamic characteristics of a water column according to claim 2, characterized in that: Step 22 further comprises: Step 221, construct background model ; Step 222: Perform discrete wavelet transform on the historical video frame to decompose it into a compressed image, a horizontal coefficient image, and a , vertical factor image and the diagonal coefficient image ; Step 223, based on the horizontal coefficient image , vertical factor image and the diagonal coefficient image , calculate the background model In the current frame energy of each block, the calculation formula is: in, is the background model, , and are the vertical, horizontal and diagonal coefficient images respectively.

5. The method for extracting dynamic characteristics of a water column according to claim 1, characterized in that: In step 2, obtaining the average growth rate of the water column includes the following steps: Step 21, get the pixel coordinates of the water column ; Step 22: Set the pixel coordinates Convert to image coordinates , the conversion formula is: in: and Respectively represent , Focal length in direction, and Represent the pixel coordinates of the principal point respectively; Step 22: The height of the water column in the pixel coordinate system Convert to the length and height in the camera coordinate system , the conversion formula is: Step 23, calculate the normalized area S of the water column in the camera coordinate system, and the calculation formula is: Step 24, calculate the average growth rate of the water column, the calculation formula is: in, , Respectively represent the pixel coordinate system and the camera coordinate system The normalized area of ​​the water column at time , and Respectively , Direction The focal length of the moment, for The number of frames at a time, is the number of area sequences, is the video frame rate.

6. The method for extracting dynamic characteristics of a water column according to claim 1, characterized in that: For a camera with a fixed focal length, the calculation formula in step 24 can be simplified to: in: Indicates the pixel coordinate system The normalized area of ​​the water column at time , for The number of frames at a time, is the number of area sequences, is the video frame rate.

7. A method for extracting dynamic characteristics of a water column according to any one of claims 1 to 5, characterized in that: Step 3 also includes: Duration determination: the duration corresponding to the water column target to be detected is determined according to a preset duration threshold, when the duration is greater than or equal to the preset duration threshold, the water column target to be detected is retained; when the duration is less than the preset duration threshold, the water column target to be detected is deleted; Energy difference ratio determination, judging the energy difference ratio corresponding to the water column target to be detected according to a preset energy difference ratio threshold, when the energy difference ratio is greater than or equal to the preset energy difference ratio threshold, retaining the water column target to be detected; when the energy difference ratio is less than the preset energy difference ratio threshold, deleting the water column target to be detected; Average growth rate determination: judging the average growth rate corresponding to the water column target to be detected according to a preset average growth rate threshold, when the average growth rate is greater than or equal to the preset average growth rate threshold, retaining the water column target to be detected; when the average growth rate is less than the preset average growth rate threshold, deleting the water column target to be detected; Water column target determination: after performing any one or more of duration determination, energy difference ratio determination and average growth rate determination on the water column target to be detected, the retained water column target to be detected is determined to be a real water column.

8. A method for extracting dynamic characteristics of a water column according to any one of claims 1 to 5, characterized in that: Step 3 also includes: Step 31, constructing a support vector machine, and selecting a radial basis kernel function as the kernel function of the support vector machine; Step 32, in the support vector machine, construct a mapping relationship between the duration, energy difference ratio and average growth rate and whether the water column target is a real water column, and the mapping function is: in, is a binary variable, indicating the tracking target Is it a water column? represents the duration, k(n) represents the average growth rate, Represents the energy difference ratio. Step 33, inputting the video frame samples containing the real water column target and the non-water column target into the support vector machine, training the support vector machine so that it approximates the relationship ; Step 34, inputting the water column target to be detected and its corresponding duration, energy difference ratio and average growth rate data into the trained support vector machine to determine whether the water column target to be detected is a real water column.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for extracting dynamic characteristics of a water column as described in any one of claims 1 to 8 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for extracting dynamic characteristics of a water column as described in any one of claims 1 to 8 is implemented.

Images