Method, device, computer and storage medium for detecting sea-sky line of infrared sea surface image sequence

Abstract

The present invention discloses a method for detecting a sea-sky line in an infrared sea image sequence, comprising: inputting a sea infrared image sequence; performing a coarse sea-sky line detection on a subsequence in the sea infrared image sequence to obtain the upper and lower boundaries of the sea-sky line; determining a sea-sky line fine detection region according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, performing a fine detection on the sea-sky line in the region to obtain the sea-sky line boundary region of the current frame, and performing a straight line fitting on the sea-sky line boundary region to obtain the sea-sky line; determining a sea-sky line fine detection region of the next adjacent frame according to the sea-sky line of the current frame, and performing a sea-sky line fine detection on the sea-sky line region of the next frame, and recursively in sequence, finally realizing the sea-sky line detection of the entire sequence. The method can effectively solve the sea-sky line detection problem in complex scenarios such as when the sea-sky line is disturbed by sea fog and sea clutter near the sea-sky line, or when the sky is disturbed by low-altitude clouds.

Description

Technical Field

[0001] The invention belongs to the technical field of image enhancement, and in particular to a method, a device, a computer and a storage medium for detecting a sea-sky line in an infrared sea surface image sequence. Background Art

[0002] In the target detection of marine infrared images, the targets are mostly located at the sea-sky boundary in the infrared image or in the sea water below the sea-sky line. Therefore, sea-sky line detection is often used as a preprocessing step for marine target detection to determine the main target detection area. In addition, the existence of the sky background increases the amount of calculation during target detection. On the other hand, in an environment with a more complex sky background, it is more difficult to detect marine targets, and the accuracy of marine target detection decreases. To solve the above problems, there are currently a variety of sea-sky line detection methods for marine infrared image sequences, which can be mainly divided into the following two categories:

[0003] Edge detection and straight line fitting algorithms. This type of algorithm first extracts the gradient features or edge features of the image, and then uses a straight line fitting algorithm to extract the sea-sky line. However, this type of algorithm is prone to misdetecting the sea-sky line when the gray value of the obstacle is significantly higher than that of other areas, the sea water fluctuates significantly, and there is interference from low-altitude clouds in the sky.

[0004] Image segmentation algorithm. This type of algorithm usually extracts the features of the sky and sea water respectively, then builds models for the two, and then classifies the sea water and sky based on the difference between the two models. The classification boundary between the sea water and the sky is the sea-sky line. However, the existing algorithms of this type are easily interfered by white clouds, sea fog, etc., which reduces the accuracy of sea-sky line detection.

[0005] In summary, the existing algorithms still have the following problems: when the sea surface and the sky background are dark and interfered by bright obstacles, the sea-sky line detection will be affected; when there is sea fog interference near the sea-sky line, the sea-sky line false detection rate is high. Summary of the invention

[0006] In order to solve the above technical problems, an embodiment of the present invention provides a method for detecting a sea-sky line in an infrared sea surface image sequence, which is characterized by comprising:

[0007] Input marine infrared image sequence;

[0008] Performing rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line;

[0009] Determine a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, perform fine detection on the sea-sky line in the sea-sky line fine detection area to obtain a sea-sky boundary area of ​​the current frame, and perform straight line fitting on the sea-sky boundary area to obtain the sea-sky line;

[0010] According to the sea-sky-line of the current frame, the sea-sky-line fine detection area of ​​the next adjacent frame is determined, and the sea-sky-line fine detection is performed on the sea-sky-line area of ​​the next frame, and the sea-sky-line fine detection is performed recursively in sequence, and finally the sea-sky-line detection of the entire sequence is realized

[0011] Further, performing a rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line comprises:

[0012] Extracting a subsequence from the marine infrared image sequence, and performing three-dimensional space-time strip division from top to bottom to determine an initial training sample area and a test sample area;

[0013] Extracting training sample sub-blocks and testing sample sub-blocks from the initial training sample area and the testing sample area respectively in a sub-block manner;

[0014] Performing sample feature transformation and feature vector construction on the training sample sub-block and the test sample sub-block respectively to obtain a sky feature set, a seawater feature set and a middle strip test sample feature set;

[0015] The Fisher linear discriminant analysis method is used to perform feature dimensionality reduction on the sky feature set, the seawater feature set and the intermediate strip test sample set, and a sky-seawater probability classification model is constructed using the reduced-dimensional sky feature set and the reduced-dimensional seawater feature set;

[0016] The training sample set is updated by using the test sample set after dimensionality reduction. When the training sample set is not updated, the upper boundary and the lower boundary of the final sea-sky line are determined, wherein the training sample set is a sky feature set and a seawater feature set.

[0017] Furthermore, the sample feature transformation and feature vector construction are performed on the training sample sub-block and the test sample sub-block to obtain a sky feature set, a seawater feature set and a middle strip test sample feature set, including:

[0018] Perform three-dimensional discrete cosine DCT transform on the three-dimensional space-time sub-images respectively;

[0019] Feature extraction is performed based on the transformed three-dimensional discrete cosine spectrum to obtain the sky feature set, seawater feature set and middle strip test sample feature set.

[0020] Furthermore, the updating of the training sample set by using the test sample set after dimensionality reduction includes:

[0021] According to the sky sub-block probability model and the seawater sub-block probability model, the middle strip test sample area of ​​the sea surface infrared image is classified to obtain a seawater-sky classification binary image;

[0022] Post-processing the test sample area, and using a sliding window scanning method to determine the sea area and sky area of ​​the classified sea surface infrared image, as well as the upper boundary and lower boundary of the sea-sky line;

[0023] The neighborhood strip successive approximation method is used to divide the classified sea area and sky area into strips of preset sizes to obtain an updated training sample set.

[0024] Further, determining a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection of the sea-sky line, performing sea-sky line fine detection on one frame of the sea-sky line fine detection area to obtain a sea-sky boundary area of ​​the current frame, and performing straight line fitting on the sea-sky boundary area to obtain the sea-sky line, including:

[0025] The row coordinates of the upper and lower boundaries of the sea-sky line area obtained by the rough detection are moved according to a preset distance to obtain the sky initial training sample area, the seawater initial training sample area and the sea-sky line sample detection area respectively;

[0026] The two-dimensional image sub-blocks of preset sizes are moved and sampled in the sky initial training sample area, the seawater initial training sample area and the sea-sky line sample detection area, and are sampled into a sky training sample set, a seawater training sample set and a sea-sky line detection area sample set according to the areas where the two-dimensional image sub-block reference points are located.

[0027] Performing a two-dimensional discrete cosine transform on the two-dimensional image sub-block to obtain a discrete cosine spectrum, extracting the discrete cosine spectrum transform domain features to obtain a feature vector, and establishing an initial sky sample training set, an initial seawater sample training set, and a sea-sky line detection area test sample set;

[0028] Using the initial sky sample training set and the initial seawater sample training set to construct and train a logistic regression model for seawater-sky classification;

[0029] The sea-sky line test sample set is classified by using the sea-sky classification logistic regression model, and the sky sample training set and the sea-sky line training set are updated until the results converge to determine the finely detected sea-sky line, and the least squares method is used for straight line fitting.

[0030] Further, completing the sea-sky-line detection of a next frame adjacent to the current frame according to the sea-sky-line of the current frame includes:

[0031] Obtain the sea-sky line equation of the current frame of fine detection, and predict the sea-sky line area of ​​the next adjacent frame;

[0032] The position of the sample of the last iteration of the current frame is used as the sampling position of the initial training sample of the adjacent next frame, and the sea-sky-line detection of the adjacent next frame is completed according to the initial training sample, and the sea-sky-line detection of the entire sequence is realized recursively.

[0033] A device for detecting a sea-sky line of an infrared sea surface image sequence, comprising:

[0034] An input module, used for inputting marine infrared image sequences;

[0035] A processing module, used for performing a rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line;

[0036] The processing module determines a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, performs fine detection on the sea-sky line in the sea-sky line fine detection area to obtain a sea-sky boundary area of ​​the current frame, and performs straight line fitting on the sea-sky boundary area to obtain the sea-sky line;

[0037] The execution module is used to determine the sea-sky-line fine detection area of ​​the next adjacent frame according to the sea-sky-line of the current frame, and perform sea-sky-line fine detection on the sea-sky-line area of ​​the next frame, and perform recursive detection in sequence to finally realize the sea-sky-line detection of the entire sequence.

[0038] Furthermore, the processing module includes:

[0039] The first input submodule is used to extract a subsequence from the marine infrared image sequence, and determine an initial training sample area and a test sample area by performing three-dimensional space-time strip division from top to bottom;

[0040] A first processing submodule, configured to extract training sample subblocks and test sample subblocks from the initial training sample region and the test sample region respectively in a subblock manner;

[0041] The second processing submodule is used to perform sample feature transformation and feature vector construction on the training sample subblock and the test sample subblock to obtain a sky feature set, a seawater feature set and a middle strip feature test set;

[0042] The third processing submodule is used to perform feature dimensionality reduction on the sky feature set, the seawater feature set and the intermediate strip test set respectively by using the Fisher linear discriminant analysis method, and to construct a sky-seawater probability classification model by using the reduced-dimensional sky feature set and the reduced-dimensional seawater feature set;

[0043] The first execution submodule is used to update the training sample set using the reduced-dimensional middle strip test sample set, and determine the upper and lower boundaries of the sea-sky line when the training sample set is not updated, wherein the training sample set is a sky feature set and a seawater feature set.

[0044] Furthermore, the second processing submodule includes:

[0045] The fourth processing submodule is used to perform three-dimensional discrete cosine DCT transform on the three-dimensional space-time sub-images respectively;

[0046] The second execution submodule is used to obtain a sky feature set, a seawater feature set and a middle strip test sample feature set by performing feature extraction on the three-dimensional discrete cosine spectrum of the transformed three-dimensional sub-block.

[0047] Furthermore, the first execution submodule includes:

[0048] The second acquisition submodule is used to classify the test sample area of ​​the sea surface infrared image according to the sky sub-block probability model and the sea water sub-block probability model to obtain a binary image after sea water-sky classification;

[0049] A fifth processing submodule, used for post-processing the test sample area, and determining the sea area and sky area of ​​the classified sea surface infrared image, as well as the upper boundary and the lower boundary of the sea-sky line by using a sliding window scanning method;

[0050] The third execution submodule is used to adopt a neighborhood strip successive approximation method to divide the classified sea area and sky area into strips of preset sizes to obtain an updated training sample set.

[0051] Furthermore, the processing module includes:

[0052] The third acquisition submodule is used to move the row coordinates of the upper boundary and the lower boundary of the sea-sky line area obtained by the rough detection according to a preset distance to obtain the sky initial training sample area, the sea water initial training sample area and the sea-sky line detection area respectively;

[0053] The fourth execution submodule is used to move and sample the two-dimensional image sub-blocks of preset sizes in the sky initial training sample area, the seawater initial training sample area and the sea-sky line sample detection area, and sample them into a sky training sample set, a seawater training sample set and a sea-sky line detection area test sample set according to the areas where the two-dimensional image sub-block reference points are located.

[0054] Furthermore, the processing module includes:

[0055] The fourth acquisition submodule is used to perform a two-dimensional discrete cosine transform on the two-dimensional image sub-block to obtain a discrete cosine spectrum, extract the discrete cosine spectrum transform domain features to obtain a feature vector, and establish an initial sky sample training set, an initial seawater sample training set and a sea-sky line detection area sample set;

[0056] A sixth processing submodule, for constructing and training a logistic regression model for seawater-sky classification using the initial sky sample training set and the initial seawater sample training set;

[0057] The fifth execution submodule is used to classify the sea-sky line test sample set using the sea-sky line classification logistic regression model, and update the sky sample training set and the sea-sky line training set until the results converge to determine the linear equation of the finely detected sea-sky line.

[0058] Furthermore, the execution module includes:

[0059] The fifth acquisition submodule is used to input the sea-sky line equation of the current frame detected in detail and predict the sea-sky line area of ​​the next adjacent frame;

[0060] The sixth execution submodule is used to use the position of the sample of the last iteration of the current frame as the sampling position of the initial training sample of the adjacent next frame, and complete the sea-sky-line detection of the adjacent next frame according to the initial training sample.

[0061] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the above-mentioned method for detecting a sea-sky line in an infrared sea surface image sequence.

[0062] A storage medium storing computer-readable instructions, when the computer-readable instructions are executed by one or more processors, enables the one or more processors to execute the steps of the method for detecting a sea-sky line in an infrared sea surface image sequence as described above.

[0063] The embodiment of the present invention includes sea-sky-line coarse detection, sea-sky-line fine detection and sea-sky-line detection of sequence images. In the sea-sky-line coarse detection stage, in the space-time three-dimensional discrete cosine domain of the infrared image sequence, the training sample area and the extended sample area are initialized a priori, and the probability classification model and the training sample set are continuously iteratively updated by the training sample recursive self-learning method until the coarsely detected sea-sky-line is obtained after the iteration is terminated; in the sea-sky-line fine detection stage, according to the sea-sky-line obtained by the coarse detection of the sea-sky-line, the finely detected sea-sky-line area is predicted, in the two-dimensional discrete cosine domain of the infrared image, the new training sample area and the extended sample area are initialized a priori, and the logistic regression model and the training sample set are iteratively updated by the training sample self-learning method to obtain the optimal logistic regression model, the position of the sea-sky-line of the frame is determined by sea-sky classification, and the sea-sky-line equation is determined by linear fitting using the least squares method; in the sea-sky-line detection stage of the sequence image, the sea-sky-line area of ​​the current frame is predicted according to the sea-sky-line position determined in the previous frame, and then the sea-sky-line fine detection method of the above-mentioned sea-sky-line is used to detect the sea-sky-line of the current frame, and the sea-sky-line detection of each frame in the sequence image is finally realized by recursion. The method can effectively solve the sea-sky-line detection problem in complex scenarios such as when the sea-sky-line is disturbed by sea fog and sea clutter, or when the sky is disturbed by low-altitude clouds. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0065] Figure 1 A schematic diagram of the flow of a method for detecting a sea-sky line in an infrared sea surface image sequence provided by an embodiment of the present invention;

[0066] Figure 2 A schematic diagram of extracting a test sample sub-block provided by an embodiment of the present invention;

[0067] Figure 3 (a), 3(b) and 3(c) are schematic diagrams of the shapes of the non-edge misclassified sub-block, the left edge misclassified sub-block and the right edge misclassified sub-block provided by the embodiments of the present invention respectively;

[0068] Figure 4 Schematic diagram of laterally continuously staggered sub-blocks provided by an embodiment of the present invention

[0069] Figure 5 A schematic diagram after classification provided for an embodiment of the present invention;

[0070] Figure 6A schematic diagram of a fine detection sea-sky line area provided in an embodiment of the present invention;

[0071] Figure 7 A schematic diagram of transform domain features provided by an embodiment of the present invention;

[0072] Figure 8 A schematic diagram of a sea-sky-line binary value provided by an embodiment of the present invention;

[0073] Fig. 9 A schematic diagram of a fine-detection sea-sky line fitting diagram provided in an embodiment of the present invention;

[0074] Fig.10 A schematic diagram of a fine detection sea-sky line area provided in an embodiment of the present invention;

[0075] Fig.11 A basic structural block diagram of a detection device for a sea-sky line of an infrared sea surface image sequence provided by an embodiment of the present invention;

[0076] Fig.12 A basic structural block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0078] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0079] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0080] Please refer to Figure 1 , Figure 1A method for detecting a sea-sky line in an infrared sea surface image sequence is provided in an embodiment of the present invention. Figure 1 As shown, the method specifically comprises the following steps:

[0081] S1, input marine infrared image sequence;

[0082] S2, performing a rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line;

[0083] Input the original infrared image subsequence I, the size is M×N×Q, where M represents the row of the image in the sequence, N represents the column of the image in the sequence, and Q represents the number of frames of the image in the sequence. In the sea-sky line detection of the marine infrared image sequence of the embodiment of the present invention, the size of the infrared image is 480 rows and 640 columns, and a 32-frame continuous image sequence is selected as the subsequence input, that is, M=480, N=640, Q=32.

[0084] In an embodiment of the present invention, performing a rough sea-sky-line detection on a subsequence in the marine infrared image sequence and obtaining an upper boundary and a lower boundary of the sea-sky-line comprises the following steps:

[0085] Step 1: extracting a subsequence from the marine infrared image sequence, and performing three-dimensional space-time strip division from top to bottom to determine an initial training sample area and a test sample area;

[0086] Step 2: extracting training sample sub-blocks and testing sample sub-blocks from the initial training sample area and the testing sample area respectively in a sub-block manner;

[0087] The subsequence of S1 input is divided into three-dimensional space-time strips from top to bottom in rows, and the size of each strip is Q×N×Q, and they do not overlap. The first three-dimensional space-time strip and the last three-dimensional space-time strip are used as the initial training sample areas of the sky and sea respectively, and the middle three-dimensional space-time strip is used as the initial test sample area.

[0088] In order to ensure the diversity of training samples, the present invention performs the initial three-dimensional sky strip R sky The initial three-dimensional space-time strip of seawater R sea The space-time 3D image sub-blocks of size Q×Q×Q are obtained by sliding along the column direction with an interval of two pixels as sample sub-blocks; test Then, the three-dimensional sub-blocks are adjacent and non-overlapping to obtain Q×Q×Q space-time three-dimensional image sub-blocks as test sample sub-blocks. Figure 2 shown.

[0089] Step 3: Perform sample feature transformation and feature vector construction on the training sample sub-blocks and the test sample sub-blocks respectively to obtain a sky feature set, a sea water feature set, and an intermediate strip test feature set;

[0090] Specifically, perform three-dimensional discrete cosine DCT transformation on the three-dimensional spatio-temporal sub-image blocks respectively; perform feature extraction based on the transformed three-dimensional discrete cosine spectrum to obtain a sky feature set, a sea water feature set, and a test sample set.

[0091] Assume a spatio-temporal three-dimensional sub-image block f(i, j, t), where i = 0, 1, …, Q - 1, j = 0, 1, …, Q - 1, t = 0, 1, …, Q - 1. Perform three-dimensional discrete cosine transform 3D-DCT on it to obtain the three-dimensional discrete cosine spectrum F(u, v, w) of this sub-block, as shown below.

[0092]

[0093] Among them, u = 0, 1, …, Q - 1, v = 0, 1, …, Q - 1, w = 0, 1, …, Q - 1.

[0094]

[0095] In the above formula, (p, q) = (i, u), (j, v), (t, w).

[0096] Along the three frequency directions of the three-dimensional discrete cosine spectrum F(u, v, w), define cut-off frequencies U1, U2, V1, V2, W1, W2 respectively. Among them, 0 < U1 < U2 < Q - 1, 0 < V1 < V2 < Q - 1, 0 < Q1 < W2 < Q - 1. In this embodiment, select U1 = V1 = W1 = 4, U2 = V2 = W2 = 16, calculate and obtain the 11-dimensional feature vector of this three-dimensional spatio-temporal sample sub-block:

[0097] E = (F 00 , F 11 , F 12 , F 13 , F 21 , F 22 , F 23 , F 31 , F 32 , F 33 , F 40 ) T

[0098] Among them, T is matrix transpose, and the calculation of each component in the vector E is as follows:

[0099] F 00 = |F(0, 0, 0)|

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110] In the above formulas, |·| represents absolute value operation. The above 11-dimensional feature extraction is performed on each 3D space-time sub-block. Three sample sets are obtained for the 3D space-time strip of the sky, the 3D space-time strip of the sea, and the 3D space-time strip of the middle:

[0111] S sky ={E k ,k=1,2,…,n sky}

[0112] S sea ={E k ,k=1,2,…,n sea}

[0113] S test ={E k ,k=1,2,…,n test}

[0114] Among them, S sky The sample set corresponding to the three-dimensional space-time strip of the sky, n sky is the number of samples in the set; S sea The sample set corresponding to the three-dimensional space-time strip of seawater, n sea is the number of samples in the set; S test is the sample set obtained from the middle three-dimensional space-time strip, n test is the number of samples in the set.

[0115] Step 4: using Fisher linear discriminant analysis method to perform feature dimensionality reduction on the sky feature set, the seawater feature set and the intermediate strip test set, and using the reduced-dimensional sky feature set and the reduced-dimensional seawater feature set to construct sky and seawater probability classification models;

[0116] Fisher discriminant analysis is used to perform feature dimensionality reduction as follows. The mean vector of the sky feature set is calculated,

[0117]

[0118] Compute the intra-class scatter matrix of the sky feature set,

[0119]

[0120] Calculate the mean vector of the seawater feature set,

[0121]

[0122] Compute the intra-class scatter matrix of the sky feature set,

[0123]

[0124] Calculate the total intra-class scatter matrix of the sky and sea feature sets,

[0125] S W =S0+S1

[0126] Calculate the projection vector,

[0127]

[0128] In the above formulas, T is the transposition operator. Yes S W Find the inverse matrix, ‖·‖ is the operator for calculating the vector length norm.

[0129] Sky Training Set S sky , seawater training set S sea and the middle strip test set S test Each sample vector in is reduced in dimension to obtain the one-dimensional Fisher feature set of the sky, sea water and intermediate test set, which are represented by Y0, Y1, and Y t Expressed as Eq.

[0130] Y0={y k =w T E k |E k ∈S sky}

[0131] Y1={y k =wT E k |E k ∈S sea}

[0132] Y t ={y k =w T E k |E k ∈S test}

[0133] Construct the probability classification model of the sky and sea water after dimensionality reduction. Construct Gaussian probability models for the initial training sample sets Y0 and Y1 of the sky and sea water after dimensionality reduction, as shown in the formula

[0134]

[0135] in,

[0136]

[0137] In the above formula, l is the label corresponding to the two categories. When l=0, it represents the probability model of the sky; when l=1, it represents the probability model of the sea water.

[0138] Step 5: Use the reduced-dimensional middle strip test sample set to update the training sample set. When the training sample set is not updated, determine the upper and lower boundaries of the sea-sky line area, where the training sample set is a sky feature set and a seawater feature set.

[0139] Specifically include:

[0140] The middle strip test sample area of ​​the sea surface infrared image is classified according to the sky sub-block probability model and the sea water sub-block probability model to obtain a sea water-sky classified binary image; the test sample area is post-processed, and a sliding window scanning method is used to determine the sea water area and the sky area of ​​the classified sea surface infrared image, as well as the upper boundary and the lower boundary of the sea-sky line; and a neighborhood strip successive approximation method is used to divide the classified sea water training area and the sky training area into strips of preset sizes to obtain an updated training sample set.

[0141] For the aforementioned infrared image middle area, each space-time three-dimensional sub-image block is divided into a sample, denoted by f k is the kth space-time three-dimensional sub-image block, and its 11-dimensional feature vector in 3D-DCT space is E k , the one-dimensional feature after dimensionality reduction is y k ,Right now,

[0142] f k →E k →y k ∈Yt

[0143] Probabilistic model using sky sub-blocks and the probability model of seawater For each f k The classification is as follows:

[0144]

[0145] Where l(·) is a label function. In this implementation case, l(f k )=0, indicating f k Classified as sky sub-block; l(f k )=1 means f k is classified as a seawater sub-block, assuming f k The strip R described in S21 test In the divided i r OK, J c For convenience, it is also written as f (ir,jc) =f k ,like Figure 2 . k ) According to f k The two-dimensional relationship in the original image space is stored in the r×c two-dimensional array B f (i r ,j c ), i r =1,2,…,r,j c =1,2,…,c, where r and c are calculated as follows:

[0146] r=(M-2Q) / Q

[0147] c=N / Q

[0148] Where r is a two-dimensional array B f (i r ,j c ), c is the number of rows of the two-dimensional array B f The number of columns.

[0149] After classification, R test There may be a problem of misclassification of sub-blocks in the region, that is, a problem of misclassification of sub-blocks. To effectively avoid the problem of misclassification of sub-blocks after classification, the embodiment of the present invention adopts the following two post-processing methods:

[0150] 1. For isolated misclassified blocks. Figure 3 As shown in , there are three different cases: non-edge misclassified sub-blocks, left edge misclassified sub-blocks, and right edge misclassified sub-blocks. Non-edge misclassified sub-blocks refer to misclassified sub-blocks that are not located in the first or last column, such as Figure 3As shown in (a); for this type of misclassified sub-block, check whether the four neighborhoods of the sub-block above, below, left, and right are samples of another type; if so, the sub-block is determined to be the category of its neighborhood sample, otherwise, the category of the sample sub-block remains unchanged. The left edge misclassified sub-block refers to the misclassified sub-block located in the first column, as shown in Figure 3 (b) As shown; for this type of misclassified sub-block, check whether the upper, lower and right neighboring sub-blocks of the sub-block are samples of another type; if so, the sub-block is determined to be of the sample category of its three neighbors, otherwise, the category of the sample sub-block remains unchanged. The right edge misclassified sub-block refers to the misclassified sub-block located in the last column, such as Figure 3 As shown in (c), for the right edge misclassified sub-block, detect whether the upper and lower right neighboring sub-blocks of the sub-block are samples of another type; if so, determine the sub-block as the category of its three neighboring samples; otherwise, the category of the sample sub-block remains unchanged.

[0151] 2. For horizontally continuous misclassified sub-blocks. Horizontally continuous misclassified sub-blocks refer to misclassified sub-blocks that are adjacent and continuously distributed, such as Figure 4 As shown. For this type of misaligned sub-block, the embodiment of the present invention first detects whether the upper and lower sub-blocks of the misaligned sub-block and the left and right sub-blocks of the upper and lower sub-blocks are of another category. If so, the sub-block will be determined as another sample category; otherwise, the sample sub-block category remains unchanged.

[0152] The sliding window scanning method is used to determine the position of the classified sea-sky line. After the classified samples are processed, the specific implementation process of determining the position of the sea-sky line in the rough detection of the sea-sky line in the embodiment of the present invention is as follows:

[0153] Use a sliding window of size 1×3 to map B f (i r ,j c ) Slide row by row to obtain the sliding window values ​​of the first 3 columns and the last 3 columns. From top to bottom, calculate the sliding window with all three labels being sky (i.e. B f (i r ,j c )=0) rl0 、i rr0 , as shown below.

[0154]

[0155]

[0156] Calculate the image block f(i rl0 ,0) and f(i rr0 ,0)The row coordinates of the lower boundary:

[0157] m skyL =(i rl0 +1)*Q

[0158] m skyR=(i rr0 +1)*Q

[0159] From bottom to top, calculate the sliding window with three labels all containing seawater (i.e., B f (i r ,j c )=1) rl1 、i rr1 , as shown below.

[0160]

[0161]

[0162] Calculate the image block f(i rl1 ,0) and f(i rr1 ,0) is the row coordinate of the upper boundary:

[0163] m seaL =i rl1 *Q+1

[0164] m seaR =i rr1 *Q+1

[0165] Calculate the row coordinates of the upper boundary of the coarsely classified sea-sky line area:

[0166]

[0167] The lower boundary row coordinates of the coarsely classified sea-sky line area:

[0168]

[0169] Figure 5 The following is a schematic diagram of the classification results after post-processing. Assume that m0 represents the sky training area R before the above classification. sky0 and seawater training sample area R sea0 Before the first classification, m0 = Q, assuming that m1 is region R sky0 The row coordinate value of the bottom row in the region R sea0 The row coordinate value of the top row in m ssu With m ssd YesR test The coordinate values ​​of the two boundary rows of the sea-sky line determined after sample classification.

[0170] Due to the interference of clouds or seawater, in complex sea conditions, the characteristics of different areas of the sky and different areas of the sea are quite different. To solve this problem, the present invention adopts the neighborhood strip successive approximation method to classify the R sky and R sea The regions are divided into strips of size Q×N.

[0171] The specific update method is: Let R sky0 and R sea0 The width of the training area is m0, let R sky0 Upper boundary row coordinate m 10 , R sea0 The lower boundary row coordinate m 20 Initially m 10 =0,m 20 =M-1. sky0 Update. As follows:

[0172]

[0173] R sea0 Update as follows:

[0174]

[0175] After the above update, the width m0 of the training area is updated to:

[0176] m0=min{m1-m 10 ,m 20 -m2}

[0177] R sky0 The upper boundary of is updated to:

[0178] m 10 =m1-m0

[0179] R sea0 The lower boundary is updated to:

[0180] m 20 =m2+m0

[0181] In the updated area R sky0 and R sea0 In , the training sample set is constructed in the same way.

[0182] Repeat the above process until the training sample is no longer updated to obtain the updated training sample area R sky0 , R sea0 And the sample area to be detected R test After that, we first obtain the projection vector w of the updated sample; then we construct the sky and seawater probability classification model after dimensionality reduction; finally, we perform the R test The samples in the region are classified and the model training sample set is updated. This is repeated until R sky0 With R sea0 No longer updated.

[0183] Output the rough detected sea-sky line. Update the training samples, and then use the final row coordinate values ​​m of the upper and lower boundaries of the sea-sky line to obtain ssu With m ssd , i.e. the upper and lower boundaries of the coarsely detected sea-sky line area.

[0184] S3, determining a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, performing fine detection on the sea-sky line in the area to obtain a sea-sky boundary area of ​​the current frame, and performing straight line fitting on the sea-sky boundary area to obtain a sea-sky line;

[0185] The specific steps include:

[0186] Step 1: Move the row coordinates of the upper and lower boundaries of the sea-sky line area obtained by coarse detection according to a preset distance to obtain a sky initial training sample area, a seawater initial training sample area, and a sea-sky line sample detection area respectively;

[0187] The initial image frame for sea-sky-line fine detection is determined, and the upper and lower boundary positions of the sea-sky-line in the sea-sky-line coarse detection are input. The fine detection sea-sky-line area is determined by the upper and lower boundaries of the coarse detection sea-sky-line area.

[0188] In the present invention, one of the frames I0 in the sequence used for the sea-sky-line coarse detection stage is selected as the initial image frame for fine classification. The upper boundary row coordinate value m of the sea-sky-line obtained in S2 ssu And the lower boundary row coordinate value m ssd , m ssu The distance Q is moved up and m ssd The distance area of ​​Q is moved down as the sea-sky line area L1 of the current frame, such as Figure 6 shown.

[0189] Determine the initial image frame I0 and the sea-sky line area, determine the sky initial training sample area L0, the sea water initial training sample area L2, and the sea-sky line detection sample area L1 in I0, such as Figure 6 shown.

[0190] The initial sky training sample area L0 is a Q×N strip, and its upper boundary row coordinates are:

[0191] n 10 =m ssu -2Q

[0192] The row coordinates of the lower boundary are:

[0193] n1=m ssu -Q

[0194] The initial seawater training sample area L2 is also a Q×N strip, and its upper boundary row coordinates are:

[0195] n2=mssd +Q

[0196] The row coordinates of the lower boundary are:

[0197] n 20 =m ssd +2Q

[0198] The sea-sky-line detection sample area L1 is one (m ssd -m ssu +2Q)×N strip, with upper and lower boundary row coordinates n1 and n2 respectively.

[0199] Step 2: Move and sample the two-dimensional image sub-blocks of preset size in the sky initial training sample area, the seawater initial training sample area and the sea-sky-line sample detection area, and sample the sky training sample set, the seawater training sample set and the sea-sky-line detection area test sample set according to the area where the two-dimensional image sub-block reference point is located. The sample is extracted as follows: In this embodiment, the sample is a two-dimensional image sub-block of size H×H, H=Q / 2=16, and the reference point of each sample sub-block is selected as the pixel in the lower right corner of the sub-block. In the areas L0, L1 and L2, a square window of size H×H is used to slide and sample pixel by pixel from left to right and from top to bottom. The image sub-block in I0 covered by the H×H square window is regarded as a sample for each sliding pixel. In the above sliding sampling, the sub-block reference point, that is, the pixel in the lower right corner of the sub-block falls in L0, which is composed of all sub-blocks as the training sample set of the sky class, the sub-block reference point falls in L1, which is composed of all sub-blocks as the sea-sky-line detection sample set, and the sub-block reference point falls in L2, which constitutes the training sample set of the seawater class.

[0200] Step 3: Perform a two-dimensional discrete cosine transform on the two-dimensional sub-block image to obtain a discrete cosine spectrum, and obtain a feature vector for the discrete cosine spectrum transform domain features, and establish an initial sky sample training set, an initial seawater sample training set, and a sea-sky line test sample set;

[0201] The calculation formula of the discrete cosine spectrum F(u,v) of the two-dimensional sub-block image f(i,j), i=0,1,…,H-1, j=0,1,…,H-1, is

[0202]

[0203] Among them, u=0,1,…,H-1,v=0,1,…,H-1.

[0204]

[0205] In the above formula, (p,q)=(i,u), (j,v).

[0206] Construct feature vectors and establish an initial sample set. The discrete cosine transform domain features constructed by the present invention include a DC component F DC , vertical AC component F V , horizontal AC component F U and the mixed AC component F M Four parts, such as Figure 7 As shown. Calculate the feature vector of each sub-block:

[0207] X=(1,F DC ,F V ,F U ,F M )

[0208] F DC =|F(0,0)|

[0209]

[0210]

[0211]

[0212] Establish the initial training sample set and test sample set. For the image block f n , define a mapping T(·), which means:

[0213]

[0214] Constructing the initial training sample set of the sky area

[0215]

[0216] And order, Constructing the initial training sample set of seawater area

[0217]

[0218] And order, Constructing the initial training sample set of sea-skyline area

[0219]

[0220] In the above set, d n is a sample X n Tags in the collection d in n =0, d in n =1, d in n To be determined.

[0221] Step 4: constructing and training a logistic regression model for seawater-sky classification using the initial sky sample training set and the initial seawater sample training set;

[0222] The logistic regression model for seawater-sky is:

[0223]

[0224] Among them, w = (w0, w1, w2, w3, w4) is the parameter of the model, T is the vector transpose operation, w T X n is defined as follows:

[0225]

[0226] Define the cost function:

[0227]

[0228] in, N c Is a collection The number of sample elements in , ∪ is the union operator. The learning of the model is to find the optimal parameter w, that is:

[0229]

[0230] The learning of logistic regression models and parameters is a relatively mature technology in the field of machine learning, and there are many iterative learning methods. The gradient descent GD iterative learning formula used in this embodiment is given below:

[0231]

[0232] Where 0<α<1 is the learning rate, w(t) refers to the w parameter at the tth iteration, and w(t+1) refers to the updated parameter w after the tth iteration. as follows:

[0233]

[0234] N c Is a collection The total number of sample elements in , and

[0235]

[0236] The initial value of the parameter w(t=0) can be randomly selected, or w(t=0)=(0,0,0,0,0) T .

[0237] When the above iterations converge, the trained parameters are obtained.

[0238] w=w(t+1)

[0239] Step 5: Classify the sea-sky line test sample set using the sea-sky line classification logistic regression model, and update the sky sample training set and the sea-sky line training set until the results converge to determine the boundary area of ​​the finely detected sea-sky line.

[0240] Using the trained model, The samples in are labeled using the following formula.

[0241]

[0242] Among them, in the above formula δ is a relaxation parameter, in this embodiment, δ = 0.45. Construct the subset to be updated:

[0243]

[0244]

[0245] Let r1 = |ΔL0| be the number of elements in the set ΔL0; r2 = |ΔL2| be the number of elements in the set ΔL2; m is the minimum value of r1 and r2, as shown below:

[0246] r m =min{r1,r2}

[0247] Eliminate r1-r in ΔL0 m samples, forming a new set ΔL 0m , remove r2-r in ΔL2 m samples, forming a new set ΔL 0m , where the elimination process can adopt random method and deterministic method. The new model training sample set is updated as:

[0248]

[0249]

[0250] Where ∪ represents the set union operation.

[0251] Repeat the above steps until the results converge. After the logistic regression model classification, the binary image B(i,j) of seawater and sky classification is calculated for the image in L1. Specifically, let (i,j)∈L1 be the sub-block f n The base pixel position, is its eigenvector, then:

[0252]

[0253] Create a one-dimensional array Z[N] of sea-sky-line boundaries and assign values ​​to it as follows:

[0254] Z(j)=max{i|B(i,j)=0},j=0,1,…,N-1

[0255] If Z t and Z t+1 are the sea-sky-line boundaries of the tth and t+1th times respectively. If:

[0256] e=max{|Z t (j)-Z t+1 (j)|,j=0,1,…,N-1}<ε

[0257] The iteration is considered to have converged. ε is a constant greater than 0.

[0258] According to the boundary results of sea-sky fine detection, a binary image is obtained by linear fitting of the sea-sky boundary, as shown in Figure 8 As shown, the row coordinates of the jth sea-sky line are stored in a one-dimensional array Z[j]. Let z j =Z[j], the equation of the sea-sky line is:

[0259] z j =K(j+1)+b

[0260] Among them, K represents the slope and b represents the intercept. By the least squares method, we can get:

[0261]

[0262] in,

[0263] In this implementation case, the straight line fitting effect of the sea-sky line is as follows: Fig. 9 shown.

[0264] S4. According to the sea-sky-line of the current frame, determine the sea-sky-line fine detection area of ​​the next adjacent frame, and perform the sea-sky-line fine detection of S3 on the area to obtain the sea-sky-line, and perform the above recursion in sequence to finally realize the sea-sky-line detection of the entire sequence.

[0265] Specifically, the steps include:

[0266] Step 1: input the sea-sky line equation of the current frame detected in detail, and predict the sea-sky line area of ​​the next adjacent frame;

[0267] Input the sea-sky line equation z in the current frame sea-sky line fine detection, and the upper and lower boundaries m of the current frame sea-sky line are ssd and m ssuThe row coordinate value of is used to predict the sea-sky line area of ​​the next frame.

[0268] Get the equation of the sea-sky line z, and calculate the values ​​of the upper and lower boundaries of the sea-sky line z0 and z N-1 , the following are the upper and lower boundaries of the sea-sky line m ssu and m ssd The row coordinate value calculation method is:

[0269] m ssu =min{z0,z N-1}

[0270] m ssd =max{z0,z N-1}

[0271] M ssu The distance of moving H up is m ssd The area moved down by a distance H is used as the sea-sky line area of ​​the current frame, such as Fig.10 shown.

[0272] Step 2: Using the position of the sample of the last iteration of the current frame as the sampling position of the initial training sample of the adjacent next frame, and completing the sea-sky line fine detection of the adjacent next frame according to the initial training sample.

[0273] Determine the initial training sample of the next frame, and extract the sample in the sea-sky line area. The present invention uses the training sample position of the last iteration of the current frame as the sampling position of the initial training sample of the next frame, and the test sample area is the determined sea-sky line area. In this embodiment, the next frame uses a similar sample extraction method to extract the sample of the next frame. Perform the above steps until the next frame sea-sky line detection is completed.

[0274] After determining the initial training sample area for the next frame and extracting the samples, firstly, the feature transformation and feature vector construction are performed on the test samples of the next frame; secondly, the test sample set for the next frame is established; then, a logistic regression model for sea-sky classification is established and trained; then, the samples in the test set for the next frame are classified, and the training set is updated at the same time; finally, the sea-sky line of the next frame is detected and output.

[0275] Repeat the above process for subsequent sequence image frames until the complete sequence sea-sky-line detection is achieved. Determine the sea-sky-line area of ​​the current frame, then determine the initial training samples of the current frame and extract the samples. Finally, detect the sea-sky-line of the current frame and detect the sea-sky-line of the next frame.

[0276] The embodiment of the present invention includes sea-sky-line coarse detection, sea-sky-line fine detection and sea-sky-line detection of sequence images. In the sea-sky-line coarse detection stage, in the space-time three-dimensional discrete cosine domain of the infrared image sequence, the training sample area and the extended sample area are initialized a priori, and the probability classification model and the training sample set are continuously iteratively updated by the training sample recursive self-learning method until the coarsely detected sea-sky-line is obtained after the iteration is terminated; in the sea-sky-line fine detection stage, according to the sea-sky-line obtained by the coarse detection of the sea-sky-line, the finely detected sea-sky-line area is predicted, and in the two-dimensional discrete cosine domain of the infrared image, the new training sample area and the extended sample area are initialized a priori, and the logistic regression model and the training sample set are iteratively updated by the training sample self-learning method to obtain the optimal logistic regression model, and the position of the sea-sky-line of the frame is determined by sea and sky classification, and the sea-sky-line equation is determined by linear fitting using the least squares method; in the sea-sky-line detection stage of the sequence image, the sea-sky-line area of ​​the current frame is predicted according to the sea-sky-line position determined in the previous frame, and then the sea-sky-line fine detection method of the above-mentioned sea-sky-line is used to detect the sea-sky-line of the current frame, and the sea-sky-line detection of each frame in the sequence image is finally realized by recursion. The method can effectively solve the sea-sky-line detection problem in complex scenarios such as when the sea-sky-line is disturbed by sea fog and sea clutter, or when the sky is disturbed by low-altitude clouds.

[0277] like Fig.11 As shown, in order to solve the above problem, an embodiment of the present invention further provides a sea-sky-line detection device for an infrared sea surface image sequence, comprising: an input module 2100, a processing module 2200 and an execution module 2300, wherein the input module 2100 is used to input a marine infrared image sequence; the processing module 2200 is used to perform a coarse sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line; the processing module 2200 is used to determine a sea-sky-line fine detection region according to the upper boundary and the lower boundary of the sea-sky-line obtained by the coarse detection, perform a fine detection on the sea-sky-line in the region to obtain a sea-sky boundary region of the current frame, and perform a straight line fitting on the sea-sky boundary region to obtain a sea-sky-line; the execution module 2300 determines a sea-sky-line fine detection region of an adjacent next frame according to the sea-sky-line of the current frame, and performs a sea-sky-line fine detection on the sea-sky-line region of the next frame, and recursively performs a sea-sky-line fine detection on the sea-sky-line region of the next frame, and finally realizes the sea-sky-line detection of the entire sequence.

[0278] The embodiment of the present invention includes sea-sky-line coarse detection, sea-sky-line fine detection and sea-sky-line detection of sequence images. In the sea-sky-line coarse detection stage, in the space-time three-dimensional discrete cosine domain of the infrared image sequence, the training sample area and the extended sample area are initialized a priori, and the probability classification model and the training sample set are continuously iteratively updated by the training sample recursive self-learning method until the coarsely detected sea-sky-line is obtained after the iteration is terminated; in the sea-sky-line fine detection stage, according to the sea-sky-line obtained by the coarse detection of the sea-sky-line, the finely detected sea-sky-line area is predicted, in the two-dimensional discrete cosine domain of the infrared image, the new training sample area and the extended sample area are initialized a priori, and the logistic regression model and the training sample set are iteratively updated by the training sample self-learning method to obtain the optimal logistic regression model, the position of the sea-sky-line of the frame is determined by sea-sky classification, and the sea-sky-line equation is determined by linear fitting using the least squares method; in the sea-sky-line detection stage of the sequence image, the sea-sky-line area of ​​the current frame is predicted according to the sea-sky-line position determined in the previous frame, and then the sea-sky-line fine detection method of the above-mentioned sea-sky-line is used to detect the sea-sky-line of the current frame, and the sea-sky-line detection of each frame in the sequence image is finally realized by recursion. The method can effectively solve the sea-sky-line detection problem in complex scenarios such as when the sea-sky-line is disturbed by sea fog and sea clutter, or when the sky is disturbed by low-altitude clouds.

[0279] In some embodiments, the processing module includes: a first acquisition submodule, used to extract subsequences from the marine infrared image sequence, and determine the initial training sample area and the test sample area according to three-dimensional space-time strip division from top to bottom; a first processing submodule, used to extract training sample subblocks and test sample subblocks from the initial training sample area and the test sample area in a subblock manner; a second processing submodule, used to perform sample feature transformation and feature vector construction on the training sample subblocks and the test sample subblocks to obtain a sky feature set, a seawater feature set, and an intermediate strip test sample feature set; a third processing submodule, used to use the Fisher linear discriminant analysis method to perform feature dimensionality reduction on the sky feature set, the seawater feature set, and the intermediate strip test sample feature set, and use the reduced sky feature set and the reduced seawater feature set to construct a sky-seawater probability classification model; a first execution submodule, used to update the training sample set using the reduced test sample set, and determine the upper boundary and lower boundary of the sea-sky line area when the training sample set is not updated, wherein the training sample set is the sky feature set and the seawater feature set.

[0280] In some embodiments, the second processing submodule includes: a fourth processing submodule, used to perform three-dimensional discrete cosine DCT transform on the three-dimensional space-time sub-images respectively; a second execution submodule, used to extract features according to the transformed three-dimensional discrete cosine spectrum to obtain a sky feature set, a sea water feature set and a middle strip test sample feature set.

[0281] In some embodiments, the first execution submodule includes: a second acquisition submodule, used to classify the middle strip test sample area of ​​the sea surface infrared image according to the sky sub-block probability model and the sea water sub-block probability model to obtain a sea water-sky classification binary image; a fifth processing submodule, used to post-process the test sample area, and use a sliding window scanning method to determine the sea water area and sky area of ​​the classified sea surface infrared image, as well as the upper boundary and lower boundary of the sea-sky line; a third execution submodule, used to use a neighborhood strip successive approximation method to divide the classified sea water area and sky area into strips of preset sizes respectively to obtain an updated training sample set.

[0282] In some embodiments, the processing module includes: a third acquisition submodule, used to move the row coordinates of the upper boundary and the lower boundary of the sea-sky line area obtained by coarse detection according to a preset distance to respectively obtain the sky initial training sample area, the sea water initial training sample area and the sea-sky line sample detection area; a fourth execution submodule, used to move and sample two-dimensional image sub-blocks of preset sizes in the sky initial training sample area, the sea water initial training sample area and the sea-sky line sample detection area, and sample them into a sky training sample set, a sea water training sample set, and a sea-sky line detection area test sample set according to the area where the two-dimensional image sub-block reference point is located.

[0283] In some embodiments, the processing module includes: a fourth acquisition submodule, used to perform a two-dimensional discrete cosine transform on a two-dimensional image sub-block to obtain a discrete cosine spectrum, and extract the discrete cosine spectrum transform domain features to obtain a feature vector, and establish an initial sky sample training set, an initial seawater sample training set, and a sea-sky line detection area test sample set; a sixth processing submodule, used to use the initial sky sample training set and the initial seawater sample training set to construct and train a seawater-sky classification logistic regression model; a fifth execution submodule, used to classify the sea-sky line test sample set using the seawater-sky classification logistic regression model, and update the sky sample training set and the seawater sample training set until the results converge and the sea-sky line equation of the fine detection is determined.

[0284] In some embodiments, the execution module includes: a fifth acquisition submodule, used to acquire the sea-sky-line straight line equation of the current frame of fine detection, and predict the sea-sky-line area of ​​the adjacent next frame; a sixth execution submodule, used to use the training sample position of the last iteration of the current frame as the initial training sample sampling position of the adjacent next frame, and complete the sea-sky-line detection of the adjacent next frame according to the initial training sample, and recursively implement the sea-sky-line detection of the entire sequence.

[0285] To solve the above technical problems, the embodiment of the present invention also provides a computer device. Fig.12 , Fig.12 This is a basic structural block diagram of the computer device in this embodiment.

[0286] like Fig.12 As shown in Figure 1, the internal structure diagram of a computer device. Fig.12 As shown, the computer device includes a processor, a non-volatile storage medium, a memory and a network interface connected via a system bus. Among them, the non-volatile storage medium of the computer device stores an operating system, a database and computer-readable instructions, and the database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor can implement an image processing method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor can execute an image processing method. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art can understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0287] In this embodiment, the processor is used to execute Fig.11 The specific contents of the acquisition module 2100, the processing module 2200 and the execution module 2300 are stored in the memory, and the program code and various data required to execute the above modules are stored in the memory. The network interface is used to transmit data between the user terminal or the server. The memory in this embodiment stores the program code and data required to execute all sub-modules in the image processing method, and the server can call the program code and data of the server to execute the functions of all sub-modules.

[0288] The present invention also provides a storage medium storing computer-readable instructions, and when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image processing method described in any of the above embodiments.

[0289] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0290] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0291] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting sea-sky line in infrared sea surface image sequence, characterized in that: include: Input marine infrared image sequence; Performing rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line; Determine a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, perform fine detection on the sea-sky line in the sea-sky line fine detection area to obtain a sea-sky boundary area of ​​the current frame, and perform straight line fitting on the sea-sky boundary area to obtain the sea-sky line; According to the sea-sky-line of the current frame, determining the sea-sky-line fine detection area of ​​the next adjacent frame, and performing sea-sky-line fine detection on the sea-sky-line area of ​​the next frame, and recursively perform sea-sky-line detection in sequence, and finally achieving the sea-sky-line detection of the entire sequence; The step of performing a rough sea-sky-line detection on the marine infrared image sequence to determine the upper and lower boundaries of the sea-sky-line comprises: Extracting a subsequence from the marine infrared image sequence, and performing three-dimensional space-time strip division from top to bottom to determine an initial training sample area and a test sample area; Extracting training sample sub-blocks and testing sample sub-blocks from the initial training sample area and the testing sample area respectively in a sub-block manner; Performing sample feature transformation and feature vector construction on the training sample sub-block and the test sample sub-block respectively to obtain a sky feature set, a seawater feature set and a middle strip test sample feature set; The Fisher linear discriminant analysis method is used to perform feature dimensionality reduction on the sky feature set, the seawater feature set and the intermediate strip test set, and a sky-seawater probability classification model is constructed using the reduced-dimensional sky feature set and the reduced-dimensional seawater feature set; The training sample set is updated by using the intermediate strip test sample set after dimensionality reduction, and the final upper boundary and lower boundary of the sea-sky line are determined when the training sample set is not updated, wherein the training sample set is a sky feature set and a seawater feature set; The method of determining a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection of the sea-sky line, performing a sea-sky line fine detection on one frame of the sea-sky line fine detection area to obtain the sea-sky boundary area of ​​the current frame, and performing a straight line fitting on the sea-sky boundary area to obtain the sea-sky line comprises: The row coordinates of the upper and lower boundaries of the sea-sky line obtained by the rough detection are moved according to a preset distance to obtain the sky initial training sample area, the seawater initial training sample area and the sea-sky line detection area respectively; The two-dimensional image sub-blocks of a preset size are sampled in the sky initial training sample area, the seawater initial training sample area and the sea-sky line detection area by movement, and the areas where the two-dimensional image sub-block reference points are located are sampled into a sky training sample set, a seawater training sample set and a sea-sky line detection area sample set respectively; Performing a two-dimensional discrete cosine transform on the two-dimensional image sub-block to obtain a discrete cosine spectrum, extracting the discrete cosine spectrum transform domain features to obtain a feature vector, and establishing an initial sky sample training set, an initial seawater sample training set, and a sea-sky line detection area test sample set; Using the initial sky sample training set and the initial seawater sample training set to construct and train a logistic regression model for seawater-sky classification; The sea-sky line test sample set is classified by using the sea-sky classification logistic regression model, and the sky sample training set and the sea-sky line training set are updated until the results converge to determine the finely detected sea-sky line, and the least squares method is used for straight line fitting.

2. The method according to claim 1, characterized in that The step of performing sample feature transformation and feature vector construction on the training sample sub-block and the test sample sub-block to obtain a sky feature set, a seawater feature set and a middle strip test sample feature set comprises: Performing three-dimensional discrete cosine DCT transform on the three-dimensional space-time sub-image blocks respectively; The sky feature set, the sea water feature set and the middle strip test sample feature set are obtained by extracting the feature of the three-dimensional discrete cosine spectrum of the transformed three-dimensional sub-image block.

3. The method according to claim 1, characterized in that The method of updating the training sample set by using the test sample set after dimensionality reduction includes: According to the sky sub-block probability model and the seawater sub-block probability model, the middle strip test sample area of ​​the sea surface infrared image is classified to obtain a seawater-sky classification binary image; Post-processing the test sample area, and using a sliding window scanning method to determine the sea area and sky area of ​​the classified sea surface infrared image, as well as the upper boundary and lower boundary of the sea-sky line; The neighborhood strip successive approximation method is used to divide the classified sea area and sky area into strips of preset sizes to obtain an updated training sample set.

4. The method according to claim 1, characterized in that: The completing the sea-sky-line detection of a next frame adjacent to the current frame according to the sea-sky-line of the current frame comprises: Obtain the sea-sky line equation of the current frame of fine detection, and predict the sea-sky line area of ​​the next adjacent frame; The position of the sample of the last iteration of the current frame in the image is used as the sampling position of the initial training sample of the adjacent next frame, and the sea-sky-line detection of the adjacent next frame is completed according to the initial training sample, and the sea-sky-line detection of the entire sequence is realized by recursion.

5. A device for detecting sea-sky line in infrared sea surface image sequence, characterized in that: include: An input module, used for inputting marine infrared image sequences; A processing module, used for performing a rough sea-sky-line detection on a subsequence in the marine infrared image sequence to obtain an upper boundary and a lower boundary of the sea-sky-line; The processing module determines a sea-sky line fine detection area according to the upper and lower boundaries of the sea-sky line obtained by the coarse detection, performs fine detection on the sea-sky line in the sea-sky line fine detection area to obtain a sea-sky boundary area of ​​the current frame, and performs straight line fitting on the sea-sky boundary area to obtain the sea-sky line; An execution module is used to determine the sea-sky-line fine detection area of ​​the next adjacent frame according to the sea-sky-line of the current frame, and perform sea-sky-line fine detection on the sea-sky-line area of ​​the next frame, and perform recursive detection in sequence to finally realize the sea-sky-line detection of the entire sequence; Wherein, the processing module includes: The first input submodule is used to extract a subsequence from the marine infrared image sequence, and determine an initial training sample area and a test sample area by performing three-dimensional space-time strip division from top to bottom; A first processing submodule, configured to extract training sample subblocks and test sample subblocks from the initial training sample region and the test sample region respectively in a subblock manner; The second processing submodule is used to perform sample feature transformation and feature vector construction on the training sample subblock and the test sample subblock to obtain a sky feature set, a seawater feature set and a middle strip feature test set; The third processing submodule is used to perform feature dimensionality reduction on the sky feature set, the seawater feature set and the intermediate strip test set respectively by using the Fisher linear discriminant analysis method, and to construct a sky-seawater probability classification model by using the reduced-dimensional sky feature set and the reduced-dimensional seawater feature set; The first execution submodule is used to update the training sample set by using the intermediate strip test sample set after dimensionality reduction, and determine the upper boundary and the lower boundary of the sea-sky line when the training sample set is not updated, wherein the training sample set is a sky feature set and a seawater feature set; Wherein, the processing module includes: The third acquisition submodule is used to move the row coordinates of the upper boundary and the lower boundary of the sea-sky line area obtained by the rough detection according to a preset distance to obtain the sky initial training sample area, the sea water initial training sample area and the sea-sky line detection area respectively; a fourth execution submodule, for moving and sampling a two-dimensional image subblock of a preset size in the sky initial training sample area, the seawater initial training sample area, and the sea-sky-line sample detection area, and sampling the area according to the position of the two-dimensional image subblock reference point into a sky training sample set, a seawater training sample set, and a sea-sky-line detection area test sample set; The fourth acquisition submodule is used to perform a two-dimensional discrete cosine transform on the two-dimensional image sub-block to obtain a discrete cosine spectrum, extract the discrete cosine spectrum transform domain features to obtain a feature vector, and establish an initial sky sample training set, an initial seawater sample training set and a sea-sky line detection area sample set; A sixth processing submodule, for constructing and training a logistic regression model for seawater-sky classification using the initial sky sample training set and the initial seawater sample training set; The fifth execution submodule is used to classify the sea-sky line test sample set using the sea-sky line classification logistic regression model, and update the sky sample training set and the sea-sky line training set until the results converge to determine the finely detected sea-sky line, and use the least squares method for straight line fitting.

6. A computer device comprising a memory and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes the steps of the method for detecting a sea-sky line in an infrared sea surface image sequence as claimed in any one of claims 1 to 4.

7. A storage medium storing computer-readable instructions, wherein when the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the method for detecting the sea-sky line in an infrared sea surface image sequence as claimed in any one of claims 1 to 4.

Images