Image stabilization method and device of shipborne camera, computer equipment and storage medium

Abstract

The invention provides a shipborne camera image stabilization scheme, which comprises the following steps of: firstly, acquiring a target shipborne video, providing original data for subsequent processing, extracting image features of adjacent picture frames to form feature vectors, converting image changes into analyzable vector data, and judging the similarity of the image features of the adjacent frames by calculating the correlation degree, so as to realize image stabilization of the shipborne camera. And a decision basis is provided for subsequent processing. When the correlation degree meets a condition, constructing a target function taking a displacement field as a variable, converting image motion analysis into an optimization problem, solving to obtain a target displacement field, and accurately determining an image pixel displacement amount; and finally, adjusting the second picture frame according to the target displacement field, and compensating picture deviation caused by ship movement. According to the scheme, the image jitter of the shipborne video caused by the shaking of the ship body is effectively and quickly eliminated, the stability and definition of the video image are improved, high-quality video information is provided for applications such as ship navigation monitoring and maritime data acquisition, and the availability and reliability of shipborne video data are enhanced.

Description

Technical Field

[0001] This application relates to the technical field of ships, and particularly to a method, device, computer device, and storage medium for image stabilization of an on-board camera. Background Art

[0002] The high-speed development of ship intelligence requires a fine perception of external environment information and its own state. The main perception means is an on-board camera. Stable video images can provide accurate environmental perception information for the crew and assist the safe navigation of the ship. During the navigation of the ship, the imaging of the on-board camera is affected by external interferences such as wind, waves, and swells, as well as the hull vibration generated by the operation of ship equipment, resulting in problems such as jitter, offset, and blurring of the perceived images.

[0003] As an important supporting technology in the traditional monitoring industry, a large number of research results have been achieved in electronic image stabilization. Due to its complex installation environment, the image jitter of on-board cameras is affected by multiple factors such as the marine environment, on-board mechanical equipment, and ship navigation state. The mechanical image stabilization is the main technology, and the electronic image stabilization is the secondary technology for the image stabilization solution. The mechanical image stabilization technology uses a pan-tilt head with three degrees of freedom (DOFs) and a robust controller. The electronic image stabilization technology uses an image cropping scheme based on the sea horizon. However, these methods still have problems such as missing details and poor real-time performance. Summary of the Invention

[0004] The purpose of this application aims to at least solve one of the above technical defects, especially the problems of missing details and poor real-time performance in the image stabilization of on-board cameras in the prior art.

[0005] In a first aspect, this application provides a method for image stabilization of an on-board camera, including:

[0006] Obtain a target on-board video;

[0007] For adjacent first and second picture frames in the target on-board video, respectively extract the image features of the first and second picture frames to obtain a first feature vector and a second feature vector;

[0008] Determine the correlation degree between the first feature vector and the second feature vector;

[0009] If the correlation degree is greater than a first threshold, then construct an objective function with the displacement field as a variable based on the first feature vector and the second feature vector; the objective function is used to reflect the difference between the second feature vector adjusted by the displacement field and the first feature vector;

[0010] Solve with the goal of minimizing the objective function to obtain a target displacement field;

[0011] Adjust the second picture frame according to the target displacement field.

[0012] In one embodiment, the image features of the first picture frame and the second picture frame are respectively extracted to obtain a first feature vector and a second feature vector, including:

[0013] Input the first picture frame into the feature extraction model to obtain a first feature vector;

[0014] Input the second picture frame into the feature extraction model to obtain a second feature vector.

[0015] In one embodiment, the feature extraction model includes a visually geometric group neural network unit, a feature enhancement unit, and a size reduction unit that are connected to each other.

[0016] In one embodiment, determining the correlation degree between the first feature vector and the second feature vector includes:

[0017] Determine the correlation degree according to the dot product of the first feature vector and the second feature vector.

[0018] In one embodiment, constructing an objective function with the displacement field as a variable according to the first feature vector and the second feature vector includes:

[0019] Adjust the second feature vector by using the displacement field;

[0020] Obtain the objective function according to the Frobenius norm between the adjusted second feature vector and the first feature vector.

[0021] In one embodiment, adjusting the second picture frame according to the target displacement field includes:

[0022] Determine the corresponding translation amount and rotation angle according to the displacement field;

[0023] Obtain the transformation matrix according to the translation amount and the rotation angle;

[0024] Adjust the second picture frame according to the transformation matrix.

[0025] In one embodiment, the image stabilization method further includes:

[0026] If the correlation degree is not greater than the first threshold, keep the second picture frame unchanged.

[0027] In a second aspect, the present application provides an image stabilization device for a shipborne camera, including:

[0028] A video acquisition module, configured to acquire a target shipborne video;

[0029] A feature extraction module, configured to respectively extract the image features of the first picture frame and the second picture frame in the target shipborne video for adjacent first picture frame and second picture frame to obtain a first feature vector and a second feature vector;

[0030] An association determination module, configured to determine the association degree between the first feature vector and the second feature vector;

[0031] A target function construction module, configured to, if the association degree is greater than a first threshold, construct a target function with a displacement field as a variable based on the first feature vector and the second feature vector; the target function is used to reflect the difference between the second feature vector adjusted by the displacement field and the first feature vector;

[0032] A solution module, configured to solve with the goal of minimizing the target function to obtain a target displacement field;

[0033] An adjustment module, configured to adjust the second picture frame according to the target displacement field.

[0034] In a third aspect, an embodiment of the present application provides a computer device, including one or more processors and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the steps of the image stabilization method of the on-board camera in any of the above embodiments are executed.

[0035] In a fourth aspect, an embodiment of the present application provides a storage medium, in which computer-readable instructions are stored. When the computer-readable instructions are executed by one or more processors, one or more processors are caused to execute the steps of the image stabilization method of the on-board camera in any of the above embodiments.

[0036] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0037] Based on the on-board camera image stabilization solution in this embodiment, first, an on-board target video is obtained to provide raw data for subsequent processing. Then, the image features of adjacent picture frames are extracted to form feature vectors, converting image changes into analyzable vector data. By calculating the association degree, the similarity of adjacent frame image features is judged, providing a decision basis for subsequent processing. When the association degree meets the condition, a target function with a displacement field as a variable is constructed, converting image motion analysis into an optimization problem, solving to obtain the target displacement field, and accurately determining the image pixel displacement amount. Finally, the second picture frame is adjusted according to the target displacement field to compensate for the picture deviation caused by ship motion. This solution effectively and quickly eliminates the picture jitter in the on-board video caused by hull shaking, improves the stability and clarity of the video picture, provides high-quality video information for applications such as ship navigation monitoring and maritime data collection, and enhances the usability and reliability of on-board video data. Description of the Drawings

[0038] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0039] Figure 1 Flow chart of the image stabilization method for an on-ship camera provided by an embodiment of the present application;

[0040] Figure 2 Flow chart of adjusting the second picture frame in an embodiment of the present application;

[0041] Figure 3 Internal structure diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0043] The embodiments of the present application provide an image stabilization method for an on-ship camera. Please refer to Figure 1 , including steps S102 to S110.

[0044] S102, obtain a target on-ship video.

[0045] It can be understood that in the shipborne camera image stabilization method, the target shipborne video refers to the video data collected by a camera installed on a ship, and this video data contains the visual information of the surrounding environment during the ship's navigation. When a ship sails at sea, it will experience movements such as bumps and shakes due to factors such as sea waves and sea winds. These movements cause the video images collected by the camera to jitter, affecting the viewing and subsequent processing of the video. As an image acquisition device, the camera converts the optical information in the scene into an electrical signal based on the principle of photoelectric conversion, and then forms a digital video signal through processes such as analog-to-digital conversion. In the shipborne environment, due to the motion characteristics of the hull, the collected video images are unstable. Obtaining the target shipborne video is the starting step of the entire image stabilization process, and subsequent image stabilization operations are based on this video. The existing shipborne camera equipment can be used to obtain the target shipborne video by setting appropriate video acquisition parameters, such as resolution, frame rate, etc. For example, a common shipborne high-definition camera can be set to a resolution of 1920×1080 and a frame rate of 30 frames per second to ensure that the obtained video can meet the clarity requirements and completely record the changes in the images during the ship's movement.

[0046] S104. For the adjacent first picture frame and second picture frame in the target shipborne video, extract the image features of the first picture frame and the second picture frame respectively to obtain a first feature vector and a second feature vector.

[0047] It can be understood that a picture frame is the basic component unit of a video, and a video is composed of a series of consecutive picture frames. In a shipborne video, each picture frame reflects the image information of the ship's surrounding environment at a certain moment. Image features refer to the information that can characterize the essential attributes and characteristics of an image. It can be local features such as edges, corner points, and textures in the image, or global features that reflect the overall structure and content of the image. A feature vector is a vector form formed by quantifying and organizing the extracted image features for subsequent analysis and processing. The first picture frame and the second picture frame are two adjacent picture frames, and the first picture frame is before the second picture frame. The entire target shipborne video will use the image of the previous frame to correct and stabilize the image of the next frame. Extracting the image features of adjacent picture frames and forming feature vectors is to analyze the changes between adjacent frames by comparing the feature vectors, and then determine the motion information of the image. Due to the ship's movement causing the video images to jitter, there will be certain changes such as displacement, rotation, or scaling between adjacent frames. By extracting image features and forming feature vectors, the complex changes in the image can be transformed into the differences between feature vectors, which is convenient for subsequent quantitative analysis and calculation. This step is an important basis for subsequent determining the correlation degree of feature vectors, constructing the objective function, and solving the displacement field. Only by accurately extracting image features and forming effective feature vectors can reliable data basis be provided for subsequent image stabilization processing.

[0048] S106. Determine the correlation degree between the first eigenvector and the second eigenvector.

[0049] It can be understood that the correlation degree is an index used to measure the similarity or correlation between two eigenvectors, which reflects the matching degree of image features between adjacent picture frames. By calculating the correlation degree, the image change situation between adjacent frames can be judged, providing a basis for subsequent processing. Determining the correlation degree between the first eigenvector and the second eigenvector is to evaluate the similarity between adjacent picture frames, and then judge whether the movement of the image is continuous and consistent. If the correlation degree is high, it indicates that the image feature change between adjacent frames is small and the movement of the image is relatively stable; on the contrary, if the correlation degree is low, it means that the image feature difference between adjacent frames is large, and there may be large movement or scene changes. This step plays a connecting role in the entire image stabilization method. It is calculated based on the previously extracted eigenvectors, providing a decision-making basis for whether to construct the objective function and how to solve the displacement field subsequently. Only by accurately determining the correlation degree can the subsequent image stabilization processing operations be reasonably carried out, avoiding poor image stabilization effects caused by misjudging the image change situation.

[0050] S108. If the correlation degree is greater than the first threshold, construct an objective function with the displacement field as a variable between the first eigenvector and the second eigenvector. The objective function is used to reflect the difference between the second eigenvector adjusted by the displacement field and the first eigenvector.

[0051] It can be understood that the first threshold is a pre-set numerical standard used to judge whether the correlation degree between the first eigenvector and the second eigenvector reaches a certain level, so as to decide whether to perform subsequent objective function construction and displacement field solution operations. The displacement field refers to the displacement change situation of each pixel point in the image, which describes the motion transformation of the image from one state to another. The objective function is a mathematical function constructed with the displacement field as a variable, and its purpose is to minimize the difference between the second eigenvector adjusted by the displacement field and the first eigenvector by adjusting the displacement field, so as to achieve image stabilization. When the correlation degree is greater than the first threshold, it indicates that the image feature change between adjacent picture frames is relatively small, with a certain degree of similarity and continuity. At this time, the objective function can be constructed to further analyze and calculate the motion situation of the image to achieve the purpose of image stabilization. The objective function takes the displacement field as a variable, and by quantifying the difference between the second eigenvector adjusted by the displacement field and the first eigenvector, it provides an optimization goal for solving the displacement field. By continuously adjusting the displacement field to minimize the value of the objective function, the displacement field that can best align the adjacent frame images can be found, thereby eliminating the image jitter caused by ship movement.

[0052] S110. Solve with the goal of minimizing the objective function to obtain the target displacement field.

[0053] It can be understood that the target displacement field is obtained by minimizing the objective function, which can minimize the image difference between adjacent picture frames, thereby realizing the displacement change situation for image stabilization. It contains the optimal displacement amount of each pixel point in the image, which is used to adjust the second picture frame to align it as much as possible with the first picture frame. Solving for the target displacement field with the goal of minimizing the objective function is the core step of the entire image stabilization method. Through the objective function constructed in the previous steps, the motion analysis of the image is transformed into a mathematical optimization problem. By solving the minimum value of the objective function, the optimal displacement field is found, enabling the images between adjacent picture frames to be better aligned, thereby eliminating the jitter of the picture caused by ship motion and realizing the stabilization of the video picture.

[0054] S112. Adjust the second picture frame according to the target displacement field.

[0055] It can be understood that the target displacement field is the result obtained by minimizing the objective function in step S110. It contains the accurate displacement change data of each pixel point in the image, which is used to describe the motion transformation required for the image to change from the current state to the stable state. The second picture frame is a frame image adjacent to the first picture frame in the target shipborne video. In the acquired video sequence, it has jitter due to ship motion. Adjusting the second picture frame is to change the positions of the pixels in the second picture frame according to the pixel displacement information provided by the target displacement field, so that it can achieve a better alignment effect with the first picture frame visually, thereby realizing the stabilization of the picture.

[0056] Based on the shipborne camera image stabilization scheme in this embodiment, first, obtain the target shipborne video to provide raw data for subsequent processing. Then, extract the image features of adjacent picture frames to form feature vectors, transform the image change into analyzable vector data, and judge the similarity of the image features of adjacent frames by calculating the correlation degree to provide a decision-making basis for subsequent processing. When the correlation degree meets the conditions, construct an objective function with the displacement field as a variable, transform the image motion analysis into an optimization problem, solve for the target displacement field, and accurately determine the image pixel displacement amount; finally, adjust the second picture frame according to the target displacement field to compensate for the picture deviation caused by ship motion. This scheme effectively and quickly eliminates the picture jitter of the shipborne video caused by the hull shaking, improves the stability and clarity of the video picture, provides high-quality video information for applications such as ship navigation monitoring and maritime data collection, and enhances the usability and reliability of shipborne video data.

[0057] In one embodiment, the image features of the first picture frame and the second picture frame are respectively extracted to obtain a first feature vector and a second feature vector, including: inputting the first picture frame into a feature extraction model to obtain a first feature vector; inputting the second picture frame into the feature extraction model to obtain a second feature vector. It can be understood that the feature extraction model is a trained algorithm model that can extract representative and discriminative image features from the input image and convert them into the form of feature vectors. The feature vector is a mathematical quantization representation of the image features, storing the key information of the image in vector form for subsequent image analysis and processing. In the scene of image stabilization of an on-board camera, the first feature vector contains important feature information related to ship movement, scene structure, etc. in the first picture frame, and the second feature vector contains important feature information related to ship movement, scene structure, etc. in the second picture frame. When the ship is sailing, the video collected by the camera has changes such as displacement and rotation between frames due to the hull vibration, and the image features can reflect these changes. By processing the first picture frame and the second picture frame respectively through the feature extraction model, the complex visual information in the image can be converted into feature vectors for easy analysis.

[0058] In one embodiment, the feature extraction model includes a visually geometric group neural network unit, a feature enhancement unit, and a size reduction unit that are interconnected. The visually geometric group neural network unit (Visual Geometry Group Neural Network Unit) is the starting part of the feature extraction model and is designed by drawing on the architecture idea of the Visual Geometry Group (VGG) network. This unit is composed of multiple convolutional layers and pooling layers stacked together. Through convolutional operations, it extracts features from the input picture frame and performs pixel calculations by sliding the convolutional kernel on the image. Specifically, the visually geometric group neural network unit can include at least two convolutional layers with 3x3 convolutional kernels, with a size of 224x224x64. Then, through a max pooling layer, the size is compressed to half of the original, becoming 112x112x64. Then, through at least two convolutional layers with 3x3 convolutional kernels, with a size of 112x112x64. Then, through a max pooling layer, the size is compressed to half of the original, becoming 56x56x64. Then, through at least two convolutional layers with 3x3 convolutional kernels, with a size of 56x56x128. Then, through a max pooling layer, the size is further compressed to half of the original, becoming 28x28x128.

[0059] The feature enhancement unit further processes the compressed feature map output by the size compression unit to enhance the feature representation related to key information such as ship motion and scene structure in the image. Through specific algorithms or network structures, it screens, enhances, and fuses the features in the feature map, highlighting the important features in the image and suppressing irrelevant or noisy features, making the feature representation more discriminative and representative. In shipborne camera image stabilization, this unit can enhance the features related to ship sway and changes in the navigation environment, providing a more effective feature basis for accurately judging the motion changes of the image, constructing the objective function, and solving the displacement field in the subsequent steps. Specifically, it can be two ECA (Efficient Channel Attention) layers with a size of 28x28x128 and a non-linear activation layer with a 1x1 convolution kernel, with a size of 28x28x256.

[0060] The size restoration unit is the last link of the feature extraction model, and its function is to restore the enhanced but size-compressed feature map output by the feature enhancement unit to an appropriate size for conversion into the final feature vector. After being processed by the previous units, the size and dimension of the feature map have changed. To meet the requirements of the subsequent operations such as calculating the correlation degree and constructing the objective function for the format of the feature vector, it is necessary to restore the feature map to a specific size through the size restoration unit. Specifically, it can include a Softmax layer and a Reshape layer.

[0061] In one embodiment, determining the correlation degree between the first feature vector and the second feature vector includes: determining the correlation degree according to the dot product of the first feature vector and the second feature vector. It can be understood that the first feature vector and the second feature vector are the results obtained by performing feature extraction on adjacent first picture frames and second picture frames in the target shipborne video. They contain the image feature information related to the ship navigation scene in their respective picture frames and are stored quantitatively in the form of vectors. The dot product, also known as the inner product, is a method of operating on two vectors in a vector space. The result obtained through the dot product operation, after certain processing and mapping, can be transformed into a numerical value representing the similarity degree between the two. According to the properties of the vector dot product, the dot product value of two vectors is related to the cosine value of their included angle and the vector norm. In the application scenario of feature vectors, when the directions of two feature vectors are similar, their dot product value is larger, indicating a high similarity degree of the image features in adjacent picture frames and less change in the picture; conversely, a smaller dot product value indicates a large difference in the directions of the feature vectors and obvious changes in adjacent frame images. For example, the first feature vector can be expressed as and the second feature vector can be expressed as , where H / W / D are the sizes of the first feature vector and the second feature term in three dimensions. The correlation degree calculation can be expressed as:

[0062]

[0063] Among them, i, j, and h are the length, width, and number of channels of the first feature vector, and k, l, and h are the length, width, and number of channels of the second feature vector.

[0064] In one embodiment, a target function with the displacement field as a variable is constructed based on the first feature vector and the second feature vector, including: adjusting the second feature vector using the displacement field. The target function is obtained according to the Frobenius norm between the adjusted second feature vector and the first feature vector. Adjusting the second feature vector is to apply the displacement field to the second feature vector, causing it to undergo corresponding transformations in the feature space to compensate for the image changes brought about by ship motion. Denoting the displacement field as, the adjusted second feature vector can be expressed as . Adjusting the second feature vector through the displacement field is actually simulating this spatial transformation in the feature space. From a mathematical perspective, the displacement field can be regarded as a kind of coordinate transformation that maps the pixels in the second picture frame to the corresponding positions in the first picture frame. When this transformation is applied to the feature vector, it can make the second feature vector closer to the first feature vector in the feature space, thereby eliminating the feature differences caused by ship motion. This adjustment is the basis for constructing the target function because only when the second feature vector is reasonably adjusted can the similarity between the two frames be accurately measured, providing a basis for solving the optimal displacement field subsequently. The Frobenius Norm is a type of matrix norm. For a matrix, its Frobenius norm is defined as the square root of the sum of the squares of all elements of the matrix. In the application of feature vectors, the feature vector can be regarded as a special matrix. By calculating the Frobenius norm between the adjusted second feature vector and the first feature vector, a scalar value measuring the difference between the two is obtained, and its value reflects the similarity degree between the adjusted second feature vector and the first feature vector under the given displacement field. The goal is to find the displacement field that minimizes the value of this function. The target function obtained based on this is: . The target displacement field can be expressed as: .

[0065] In one embodiment, the second picture frame is adjusted according to the target displacement field. Please refer to Figure 2 , including steps S202 to S206.

[0066] S202, determining the corresponding translation amount and rotation angle according to the displacement field.

[0067] It can be understood that the translation amount is the linear movement distance of pixels in the horizontal (x-axis) and vertical (y-axis) directions, reflecting the overall translation trend of the image; the rotation angle is the angle by which the image rotates around a reference point (usually the center of the image), reflecting the degree of image rotation caused by ship sway. To stabilize the image sequence based on the displacement field, two affine transformations of translation and rotation must be performed. To simplify the adjustment process, it is best to combine the two transformations into one affine transformation. Therefore, it is necessary to determine the corresponding translation amount and rotation angle based on the displacement field. Due to the movement of the hull in shipborne videos, there are rigid body transformations such as translation and rotation between adjacent frames, and the displacement field contains the comprehensive information of these transformations. Analyzing the displacement field and separating the translation and rotation components is the key to converting the complex displacement field into operable geometric transformation parameters. Mathematically, the displacement field can be decomposed into components related to the translation vector and rotation matrix. Using calculus and geometric transformation knowledge, the sampling points of the displacement field are calculated and fitted to extract the translation amount and rotation angle.

[0068] S204. Obtain the transformation matrix according to the translation amount and rotation angle.

[0069] It can be understood that in image geometric transformation, the transformation matrix is a mathematical matrix used to describe geometric transformations such as image translation, rotation, and scaling. In the shipborne image stabilization scenario, the transformation matrix constructed by combining the translation amount and rotation angle can map the pixel coordinates of the second picture frame to the corrected coordinates, realizing image pose adjustment. It is a mathematical tool connecting the motion parameters and the image adjustment operation. Its specific expression can be:

[0070]

[0071] where the translation amount and rotation angle are respectively and .

[0072] S206. Adjust the second picture frame according to the transformation matrix.

[0073] It can be understood that this step means using the transformation matrix to transform the image representation matrix of the second picture frame, correcting the image geometric deviation (such as translation and rotation) caused by ship movement, making the adjusted image more stable visually, and having the same relative position and pose as the reference frame (such as the first picture frame). It is the final execution link of the image stabilization process.

[0074] In one embodiment, the image stabilization method further includes: if the correlation degree is not greater than the first threshold, keep the second picture frame unchanged.

[0075] The present application provides a video stabilization device for an on-ship camera, including: a video acquisition module for acquiring a target on-ship video; a feature extraction module for respectively extracting the image features of a first picture frame and a second picture frame adjacent in the target on-ship video to obtain a first feature vector and a second feature vector; a correlation determination module for determining the correlation degree between the first feature vector and the second feature vector; a target function construction module for, if the correlation degree is greater than a first threshold, constructing a target function with a displacement field as a variable based on the first feature vector and the second feature vector, where the target function is used to reflect the difference between the second feature vector adjusted by the displacement field and the first feature vector; a solution module for solving with the goal of minimizing the target function to obtain a target displacement field; and an adjustment module for adjusting the second picture frame according to the target displacement field.

[0076] For the specific limitations of the video stabilization device for an on-ship camera, reference may be made to the limitations of the video stabilization method for an on-ship camera in the above text, which will not be elaborated here. Each module in the above video stabilization device for an on-ship camera can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or stored in a memory in a computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above-mentioned modules. It should be noted that the division of modules in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0077] The present application provides a computer device, including one or more processors and a memory. Computer-readable instructions are stored in the memory. When the computer-readable instructions are executed by one or more processors, the video stabilization method for an on-ship camera in any of the above embodiments is executed.

[0078] Schematically, as Figure 3 shown, Figure 3 is a schematic internal structure diagram of a computer device provided by an embodiment of the present application. Referring to Figure 3 , the computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by a memory 301 for storing instructions executable by the processing component 302, such as application programs. The application programs stored in the memory 301 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 302 is configured to execute instructions to perform the steps of the video stabilization method for an on-ship camera in any of the above embodiments.

[0079] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300, a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305.

[0080] Those skilled in the art can understand that Figure 3 the structure shown in is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0081] This application provides a storage medium storing computer-readable instructions that, when executed by one or more processors, cause the one or more processors to execute the image stabilization method of the on-vehicle camera in any of the above embodiments.

[0082] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0083] The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for stabilizing the image of a shipborne camera, characterized in that, Including: Obtain a target on-ship video; For adjacent first and second picture frames in the target on-ship video, respectively extract the image features of the first picture frame and the second picture frame to obtain a first feature vector and a second feature vector; Determine the correlation degree between the first feature vector and the second feature vector; If the correlation degree is greater than a first threshold, then construct an objective function with a displacement field as a variable between the first feature vector and the second feature vector; the objective function is used to reflect the difference between the second feature vector adjusted by the displacement field and the first feature vector; Solve with the goal of minimizing the objective function to obtain a target displacement field; Adjust the second picture frame according to the target displacement field.

2. The image stabilization method according to claim 1, wherein The step of respectively extracting the image features of the first picture frame and the second picture frame to obtain a first feature vector and a second feature vector includes: Input the first picture frame into a feature extraction model to obtain the first feature vector; Input the second picture frame into the feature extraction model to obtain the second feature vector.

3. The image stabilization method according to claim 2, wherein The feature extraction model includes a visually geometric group neural network unit, a feature enhancement unit, and a size reduction unit connected to each other.

4. The image stabilization method according to claim 1, characterized in that The step of determining the correlation degree between the first feature vector and the second feature vector includes: Determine the correlation degree according to the dot product of the first feature vector and the second feature vector.

5. The image stabilization method according to claim 1, wherein The step of constructing an objective function with a displacement field as a variable between the first feature vector and the second feature vector includes: Adjust the second feature vector by using the displacement field; Obtain the objective function according to the Frobenius norm between the adjusted second feature vector and the first feature vector.

6. The image stabilization method according to claim 1, wherein The step of adjusting the second picture frame according to the target displacement field includes: Determine the corresponding translation amount and rotation angle according to the displacement field; Obtain a transformation matrix according to the translation amount and the rotation angle; Adjust the second picture frame according to the transformation matrix.

7. The image stabilization method according to claim 1, wherein It also includes: If the correlation degree is not greater than the first threshold, then keep the second picture frame unchanged.

8. An image stabilization device for a shipborne camera, characterized in that, Including: A video acquisition module for acquiring a target on-ship video; A feature extraction module for, for adjacent first and second picture frames in the target on-ship video, respectively extract the image features of the first picture frame and the second picture frame to obtain a first feature vector and a second feature vector; A correlation determination module for determining the correlation degree between the first feature vector and the second feature vector; An objective function construction module for, if the correlation degree is greater than a first threshold, then construct an objective function with a displacement field as a variable between the first feature vector and the second feature vector; the objective function is used to reflect the difference between the second feature vector adjusted by the displacement field and the first feature vector; A solving module for solving with the goal of minimizing the objective function to obtain a target displacement field; An adjustment module for adjusting the second picture frame according to the target displacement field.

9. A computer device, characterized in that, Comprising one or more processors and a memory, wherein computer-readable instructions are stored in the memory, and when the computer-readable instructions are executed by the one or more processors, the steps of the image stabilization method of the on-vehicle camera according to any one of claims 1-7 are executed.

10. A storage medium, characterized in that, Computer-readable instructions are stored in the storage medium, and when the computer-readable instructions are executed by one or more processors, the one or more processors are caused to execute the steps of the image stabilization method of the on-vehicle camera according to any one of claims 1-7.

Images