Image stabilization method, device, computer equipment and storage medium for shipborne camera

Abstract

The present application provides a shipborne camera image stabilization solution. First, the target shipborne video is acquired to provide raw data for subsequent processing. Then, the image features of adjacent picture frames are extracted to form feature vectors. The image changes are converted into analyzable vector data. The similarity of the image features of adjacent frames is judged by calculating the correlation degree, providing a decision basis for subsequent processing. When the correlation degree meets the conditions, an objective function with the displacement field as the variable is constructed, the image motion analysis is converted into an optimization problem, the target displacement field is solved, and the image pixel displacement is accurately determined; finally, the second picture frame is adjusted according to the target displacement field to compensate for the image deviation caused by the ship's motion. This solution effectively and quickly eliminates the image jitter caused by the shaking of the ship's hull, improves the stability and clarity of the video, provides high-quality video information for applications such as ship navigation monitoring and maritime data collection, and enhances the availability and reliability of shipborne video data.

Description

Technical Field

[0001] The present application relates to the field of ship technology, and in particular to a method, device, computer equipment, and storage medium for stabilizing an image of a shipborne camera. Background Art

[0002] The rapid development of intelligent ships requires precise perception of both the external environment and their own status, primarily through shipboard cameras. Stable video images provide crew members with accurate environmental information, assisting with safe navigation. During navigation, camera images are subject to external interference from wind, waves, surges, and other external factors, as well as hull vibrations caused by the operation of ship equipment. These can cause jitter, offset, and blurring of the perceived images.

[0003] Electronic image stabilization, a key supporting technology in the traditional surveillance industry, has yielded extensive research results. Due to the complex installation environments of shipboard cameras, image jitter is affected by a variety of factors, including the marine environment, onboard mechanical equipment, and the vessel's navigational state. Mechanical image stabilization is the primary technology, with electronic image stabilization serving as a secondary stabilization solution. Mechanical image stabilization utilizes a pan / tilt (PTZ) with three degrees of freedom (DOF) and a robust controller. Electronic image stabilization utilizes a sea-skyline-based image cropping scheme. However, these methods still suffer from issues such as loss of detail and poor real-time performance. Summary of the Invention

[0004] The purpose of this application is to solve at least one of the above-mentioned technical defects, especially the problems of missing details and poor real-time performance of image stabilization of shipborne cameras in the prior art.

[0005] In a first aspect, the present application provides a method for stabilizing an image of a shipborne camera, comprising:

[0006] Acquire target shipborne video;

[0007] For adjacent first and second picture frames in the target shipborne video, extract image features of the first and second picture frames respectively to obtain a first feature vector and a second feature vector;

[0008] determining a correlation between the first eigenvector and the second eigenvector;

[0009] If the correlation degree is greater than the first threshold, constructing an objective function with the displacement field as a variable based on the relationship between the first eigenvector and the second eigenvector; the objective function is used to reflect the difference between the second eigenvector adjusted by the displacement field and the first eigenvector;

[0010] Solve the problem with the goal of minimizing the objective function to obtain the target displacement field;

[0011] The second image frame is adjusted according to the target displacement field.

[0012] In one embodiment, extracting image features of the first picture frame and the second picture frame respectively to obtain a first feature vector and a second feature vector includes:

[0013] Inputting the first image frame into the feature extraction model to obtain a first feature vector;

[0014] The second image frame is input into the feature extraction model to obtain a second feature vector.

[0015] In one embodiment, the feature extraction model includes a visual geometry group neural network unit, a feature enhancement unit and a size restoration unit that are interconnected.

[0016] In one embodiment, determining the correlation between the first eigenvector and the second eigenvector includes:

[0017] The degree of association is determined according to a dot product of the first eigenvector and the second eigenvector.

[0018] In one embodiment, constructing an objective function with the displacement field as a variable based on the relationship between the first eigenvector and the second eigenvector includes:

[0019] Using the displacement field to adjust the second eigenvector;

[0020] The objective function is obtained according to the Frobenius norm between the adjusted second eigenvector and the first eigenvector.

[0021] In one embodiment, adjusting the second image frame according to the target displacement field includes:

[0022] Determine the corresponding translation amount and rotation angle according to the displacement field;

[0023] Get the transformation matrix according to the translation amount and rotation angle;

[0024] The second image frame is adjusted according to the transformation matrix.

[0025] In one embodiment, the image stabilization method further includes:

[0026] If the correlation degree is not greater than the first threshold, the second picture frame remains unchanged.

[0027] In a second aspect, the present application provides an image stabilization device for a shipborne camera, comprising:

[0028] Video acquisition module, used to acquire target shipborne video;

[0029] A feature extraction module is used to extract image features of a first picture frame and a second picture frame adjacent to each other in the target shipborne video, respectively, to obtain a first feature vector and a second feature vector;

[0030] an association determination module, configured to determine a degree of association between the first eigenvector and the second eigenvector;

[0031] an objective function construction module, configured to construct an objective function with the displacement field as a variable based on the relationship between the first eigenvector and the second eigenvector if the correlation degree is greater than a first threshold; the objective function is configured to reflect the difference between the second eigenvector adjusted by the displacement field and the first eigenvector;

[0032] A solution module is used to solve the problem with the goal of minimizing the objective function to obtain the target displacement field;

[0033] The adjustment module is used to adjust the second image frame according to the target displacement field.

[0034] In a third aspect, an embodiment of the present application provides a computer device comprising one or more processors and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the steps of the image stabilization method for a shipborne camera in any of the above embodiments are executed.

[0035] In a fourth aspect, an embodiment of the present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the image stabilization method for the shipborne camera in any of the above embodiments.

[0036] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0037] Based on the shipborne camera stabilization solution in this embodiment, first, the target shipborne video is acquired to provide raw data for subsequent processing. Then, the image features of adjacent picture frames are extracted to form feature vectors, and the image changes are converted into analyzable vector data. The similarity of the image features of adjacent frames is determined by calculating the correlation degree, providing a decision basis for subsequent processing. When the correlation degree meets the conditions, an objective function with the displacement field as the variable is constructed, and the image motion analysis is converted into an optimization problem. The target displacement field is solved and the image pixel displacement is accurately determined. Finally, the second picture frame is adjusted according to the target displacement field to compensate for the image deviation caused by the ship's motion. This solution effectively and quickly eliminates the image jitter caused by the shaking of the hull in the shipborne video, improves the stability and clarity of the video image, provides high-quality video information for applications such as ship navigation monitoring and maritime data collection, and enhances the availability and reliability of shipborne video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0039] Figure 1 A schematic flow chart of an image stabilization method for a shipborne camera provided in one embodiment of the present application;

[0040] Figure 2 This is a schematic diagram of a process for adjusting the second picture frame in one embodiment of the present application;

[0041] Figure 3 A diagram of the internal structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0043] The present application embodiment provides a method for stabilizing the image of a shipborne camera. Figure 1 , including steps S102 to S110.

[0044] S102, obtaining a target shipborne video.

[0045] It can be understood that in the shipborne camera image stabilization method, the target shipborne video refers to the video data captured by a camera installed on the vessel. This video data contains visual information of the vessel's surrounding environment during navigation. When a ship is at sea, it is subject to the influence of factors such as waves and wind, causing it to pitch and shake. This motion causes jitter in the video captured by the camera, affecting video viewing and subsequent processing. As an image acquisition device, a camera converts optical information in the scene into electrical signals based on the principle of photoelectric conversion. This is then processed through analog-to-digital conversion and other processes to form a digital video signal. In a shipborne environment, the captured video is unstable due to the motion characteristics of the ship. Acquiring the target shipborne video is the initial step in the entire image stabilization process, and subsequent image stabilization operations are based on this video. The target shipborne video can be acquired by utilizing existing shipborne camera equipment by setting appropriate video acquisition parameters such as resolution and frame rate. For example, a common shipborne HD camera can be set to a resolution of 1920×1080 and a frame rate of 30 frames per second to ensure that the captured video meets the required clarity and fully records the image changes during the ship's motion.

[0046] S104 , for a first picture frame and a second picture frame adjacent to each other in the target shipborne video, extract image features of the first picture frame and the second picture frame respectively to obtain a first feature vector and a second feature vector.

[0047] As you can understand, an image frame is the basic unit of video. Video consists of a series of consecutive frames. In shipborne video, each frame reflects the image information of the ship's surroundings at a specific moment. Image features are information that characterizes the essential attributes and characteristics of an image. They can be local features such as edges, corners, and textures, or global features that reflect the overall structure and content of the image. A feature vector is a vector formed by quantifying and organizing the extracted image features for subsequent analysis and processing. The first and second frames are adjacent frames. The first frame precedes the second frame. The entire target shipborne video uses the image of the previous frame to correct and stabilize the image of the next frame. Extracting image features from adjacent frames and forming feature vectors is done to analyze the changes between adjacent frames by comparing the feature vectors and thereby determine image motion information. Due to the jitter caused by ship motion, adjacent frames may exhibit certain variations such as displacement, rotation, and scaling. By extracting image features and forming feature vectors, complex image changes can be converted into differences between feature vectors, facilitating subsequent quantitative analysis and calculation. This step is an important basis for subsequently determining the correlation degree of eigenvectors, constructing the objective function, and solving the displacement field. Only by accurately extracting image features and forming effective eigenvectors can a reliable data basis be provided for subsequent image stabilization processing.

[0048] S106: Determine the correlation between the first eigenvector and the second eigenvector.

[0049] As can be understood, correlation is a metric used to measure the similarity or correlation between two eigenvectors. It reflects the degree of matching between image features in adjacent frames. By calculating the correlation, we can determine the image changes between adjacent frames, providing a basis for subsequent processing. Determining the correlation between the first and second eigenvectors is to assess the similarity between adjacent frames and, therefore, determine whether the image motion is continuous and consistent. A high correlation indicates that the image features between adjacent frames vary little and the image motion is relatively stable. Conversely, a low correlation indicates that the image features between adjacent frames differ significantly, possibly indicating significant motion or scene changes. This step serves as a link between the previous and next steps in the image stabilization method. Based on the previously extracted eigenvectors, it provides a basis for subsequent decision-making on whether to construct the objective function and how to solve the displacement field. Only by accurately determining the correlation can subsequent image stabilization processing be performed appropriately, avoiding poor stabilization results caused by misjudgment of image changes.

[0050] S108: If the correlation is greater than the first threshold, construct an objective function with the displacement field as a variable based on the first eigenvector and the second eigenvector. The objective function is used to reflect the difference between the second eigenvector adjusted by the displacement field and the first eigenvector.

[0051] It can be understood that the first threshold is a pre-set numerical standard used to determine whether the correlation between the first and second eigenvectors reaches a certain level, thereby determining whether to proceed with the subsequent objective function construction and displacement field solution. The displacement field refers to the displacement change of each pixel in the image, describing the motion transformation of the image from one state to another. The objective function is a mathematical function constructed using the displacement field as a variable. Its purpose is to minimize the difference between the second eigenvector and the first eigenvector after adjustment, thereby achieving image stabilization. When the correlation is greater than the first threshold, it indicates that the image features between adjacent frames have relatively small changes, showing a certain degree of similarity and continuity. In this case, the objective function can be constructed to further analyze and calculate the image motion to achieve image stabilization. The objective function uses the displacement field as a variable and quantifies the difference between the second eigenvector and the first eigenvector after adjustment, providing an optimization target for solving the displacement field. By continuously adjusting the displacement field to minimize the value of the objective function, the displacement field that best aligns adjacent frames can be found, thereby eliminating image jitter caused by ship motion.

[0052] S110, solving with the goal of minimizing the objective function to obtain a target displacement field.

[0053] It can be understood that the target displacement field refers to the displacement change obtained by minimizing the objective function, which can minimize the image difference between adjacent picture frames and thus achieve image stabilization. It contains the optimal displacement of each pixel in the image, which is used to adjust the second picture frame to align it with the first picture frame as much as possible. Solving the target displacement field with the goal of minimizing the objective function is the core step of the entire image stabilization method. The objective function constructed in the previous steps converts the motion analysis of the image into a mathematical optimization problem. By solving the minimum value of the objective function, the optimal displacement field is found, so that the images between adjacent picture frames can be better aligned, thereby eliminating the picture jitter caused by the movement of the ship and achieving video stabilization.

[0054] S112: Adjust the second image frame according to the target displacement field.

[0055] It can be understood that the target displacement field is the result obtained by minimizing the objective function in step S110. It contains the precise displacement change data for each pixel in the image and is used to describe the motion transformation required to move the image from its current state to a stable state. The second image frame is an image frame in the target shipborne video adjacent to the first image frame. In the acquired video sequence, it exhibits jitter due to the ship's motion. Adjusting the second image frame involves changing the position of each pixel in the second image frame based on the pixel displacement information provided by the target displacement field, so that it is visually aligned with the first image frame, thereby achieving image stabilization.

[0056] Based on the shipborne camera stabilization solution in this embodiment, first, the target shipborne video is acquired to provide raw data for subsequent processing. Then, the image features of adjacent picture frames are extracted to form feature vectors, and the image changes are converted into analyzable vector data. The similarity of the image features of adjacent frames is determined by calculating the correlation degree, providing a decision basis for subsequent processing. When the correlation degree meets the conditions, an objective function with the displacement field as the variable is constructed, and the image motion analysis is converted into an optimization problem. The target displacement field is solved and the image pixel displacement is accurately determined. Finally, the second picture frame is adjusted according to the target displacement field to compensate for the image deviation caused by the ship's motion. This solution effectively and quickly eliminates the image jitter caused by the shaking of the hull in the shipborne video, improves the stability and clarity of the video image, provides high-quality video information for applications such as ship navigation monitoring and maritime data collection, and enhances the availability and reliability of shipborne video data.

[0057] In one embodiment, extracting image features from a first image frame and a second image frame to obtain a first feature vector and a second feature vector includes: inputting the first image frame into a feature extraction model to obtain a first feature vector; and inputting the second image frame into the feature extraction model to obtain a second feature vector. It can be understood that a feature extraction model is a trained algorithmic model that extracts representative and discriminative image features from an input image and converts them into feature vectors. A feature vector is a mathematically quantified representation of image features, storing key image information in vector form for subsequent image analysis and processing. In a shipborne camera image stabilization scenario, the first feature vector contains important feature information related to ship motion, scene structure, etc. in the first image frame, and the second feature vector contains important feature information related to ship motion, scene structure, etc. in the second image frame. When a ship is sailing, the video captured by the camera may experience changes such as displacement and rotation between frames due to the ship's hull shaking, and the image features can reflect these changes. By processing the first and second image frames separately through the feature extraction model, the complex visual information in the images can be converted into feature vectors that are easy to analyze.

[0058] In one embodiment, the feature extraction model includes a connected visual geometry group neural network unit, a feature enhancement unit, and a scale reduction unit. The visual geometry group neural network unit (VGG) is the starting point of the feature extraction model and is designed based on the architectural principles of the visual geometry group (VGG) network. This unit consists of multiple stacked convolutional and pooling layers. It extracts features from the input image frame through convolution operations and performs pixel calculations by sliding the convolution kernel across the image. The VGG neural network unit can specifically include at least two convolutional layers with 3x3 convolution kernels, with a size of 224x224x64. A max pooling layer compresses the size to half, to 112x112x64. At least two convolutional layers with 3x3 convolution kernels, with a size of 112x112x64, and a max pooling layer compresses the size to half, to 56x56x64. After passing through at least two convolutional layers with 3x3 convolution kernels, the size is 56x56x128. After passing through a maximum pooling layer, the size is further compressed to half of the original size, becoming 28x28x128.

[0059] The feature enhancement unit further processes the compressed feature map output by the size compression unit to enhance the feature representation related to key information in the image, such as ship motion and scene structure. It uses a specific algorithm or network structure to filter, enhance, and fuse features in the feature map, highlighting important features in the image and suppressing irrelevant or noisy features, making the feature representation more discriminative and representative. In shipborne camera image stabilization, this unit can enhance features related to ship motion and changes in the navigation environment, providing a more effective feature basis for subsequent accurate judgment of image motion changes, construction of objective functions, and solution of displacement fields. Specifically, it can be two ECA (Efficient Channel Attention) layers of size 28x28x128 and a nonlinear activation layer with a 1x1 convolution kernel, with a size of 28x28x256.

[0060] The resizing unit is the final step in the feature extraction model. Its function is to restore the enhanced but compressed feature map output by the feature enhancement unit to a suitable size for conversion into the final feature vector. After processing by the previous units, the size and dimensionality of the feature map have changed. To meet the feature vector format requirements for subsequent operations such as calculating correlation and constructing the objective function, the resizing unit is required to restore the feature map to a specific size. This can include a Softmax layer and a Reshape layer.

[0061] In one embodiment, determining the degree of correlation between the first eigenvector and the second eigenvector includes: determining the degree of correlation based on the dot product of the first eigenvector and the second eigenvector. It can be understood that the first eigenvector and the second eigenvector are the results obtained by performing feature extraction on the adjacent first and second picture frames in the target shipborne video. They contain image feature information related to the ship's navigation scene in their respective picture frames, which are quantized and stored in the form of vectors. The dot product, also known as the inner product, is a method of operating two vectors in a vector space. The result obtained by the dot product operation can be converted into a numerical value representing the degree of similarity between the two after certain processing and mapping. According to the properties of the vector dot product, the dot product value of the two vectors is related to the cosine value of their angle and the vector modulus. In the application scenario of the eigenvector, when the directions of the two eigenvectors are similar, their dot product values ​​are large, which means that the image features of the adjacent picture frames are highly similar and the picture changes are small; conversely, a small dot product value indicates that the directions of the eigenvectors are very different and the image changes of adjacent frames are obvious. For example, the first eigenvector can be expressed as , the second eigenvector can be expressed as , H / W / D are the sizes of the first eigenvector and the second eigenterm in three dimensions. The correlation calculation can be expressed as:

[0062]

[0063] Among them, i, j, h are the length, width and number of channels of the first eigenvector, and k, l, h are the length, width and number of channels of the second eigenvector.

[0064] In one embodiment, constructing an objective function with a displacement field as a variable based on the relationship between the first eigenvector and the second eigenvector includes: adjusting the second eigenvector using the displacement field. Obtaining the objective function based on the Frobenius norm between the adjusted second eigenvector and the first eigenvector. Adjusting the second eigenvector involves applying the displacement field to the second eigenvector so that it undergoes a corresponding transformation in the feature space to compensate for image changes caused by the ship's motion. Represents the displacement field, and the adjusted second eigenvector can be expressed as Adjusting the second eigenvector using the displacement field actually simulates this spatial transformation in the feature space. From a mathematical perspective, the displacement field can be viewed as a coordinate transformation, mapping pixels in the second image frame to corresponding positions in the first image frame. When applied to the eigenvector, this transformation brings the second eigenvector closer to the first eigenvector in the feature space, eliminating the characteristic differences caused by the ship's motion. This adjustment is the basis for constructing the objective function, as only when the second eigenvector is properly adjusted can the similarity between the two frames be accurately measured, providing a basis for subsequently solving the optimal displacement field. The Frobenius norm is a type of matrix norm. For a matrix, the Frobenius norm is defined as the square root of the sum of the squares of all the matrix elements. In the application of eigenvectors, the eigenvector can be treated as a special matrix. By calculating the Frobenius norm between the adjusted second eigenvector and the first eigenvector, a scalar value is obtained that measures the difference between the two. This value reflects the degree of similarity between the adjusted second eigenvector and the first eigenvector under a given displacement field. The goal is to find the displacement field that minimizes this function. The objective function obtained based on this is: The target displacement field can be expressed as: .

[0065] In one embodiment, the second image frame is adjusted according to the target displacement field, see Figure 2 , including steps S202 to S206.

[0066] S202: Determine the corresponding translation amount and rotation angle according to the displacement field.

[0067] As can be understood, translation is the linear displacement of pixels in the horizontal (x-axis) and vertical (y-axis) directions, reflecting the overall translational tendency of the image. Rotation is the angle of rotation of the image around a reference point (usually the image center), reflecting the degree of image rotation caused by ship motion. Compensating an image sequence based on the displacement field requires two affine transformations: translation and rotation. Image stabilization is achieved through these two transformations. To simplify the adjustment process, it is best to combine these two transformations into a single affine transformation. Therefore, it is necessary to determine the corresponding translation and rotation angles based on the displacement field. Due to ship motion, rigid body transformations such as translation and rotation occur between adjacent frames in shipborne video. The displacement field contains the combined information of these transformations. Analyzing the displacement field and separating the translation and rotation components is key to converting the complex displacement field into operational geometric transformation parameters. Mathematically, the displacement field can be decomposed into components related to the translation vector and the rotation matrix. Using calculus and geometric transformations, the displacement field sampling points are calculated and fitted to extract the translation and rotation angles.

[0068] S204: Obtain a transformation matrix according to the translation amount and the rotation angle.

[0069] It can be understood that in image geometric transformation, a transformation matrix is ​​a mathematical matrix used to describe geometric transformations such as image translation, rotation, and scaling. In shipborne image stabilization scenarios, the transformation matrix, constructed by combining the translation amount and rotation angle, can map the pixel coordinates of the second image frame to the corrected coordinates, achieving image pose adjustment. It is a mathematical tool that connects motion parameters with image adjustment operations. Its specific expression can be:

[0070]

[0071] The translation and rotation angle are and .

[0072] S206: Adjust the second picture frame according to the transformation matrix.

[0073] It can be understood that this step refers to using the transformation matrix to transform the image representation matrix of the second picture frame, correcting the image geometric deviation (translation, rotation, etc.) caused by the movement of the ship, making the adjusted image more visually stable and consistent with the relative position and posture of the reference frame (such as the first picture frame). It is the final execution link of the image stabilization process.

[0074] In one embodiment, the image stabilization method further includes: if the correlation degree is not greater than a first threshold, keeping the second image frame unchanged.

[0075] The present application provides an image stabilization device for a shipborne camera, comprising: a video acquisition module for acquiring a target shipborne video. A feature extraction module for extracting the image features of the first picture frame and the second picture frame, respectively, from adjacent first and second picture frames in the target shipborne video to obtain a first eigenvector and a second eigenvector. An association determination module for determining the degree of correlation between the first eigenvector and the second eigenvector. An objective function construction module for constructing an objective function with a displacement field as a variable based on the first eigenvector and the second eigenvector if the degree of correlation is greater than a first threshold. The objective function is used to reflect the difference between the second eigenvector and the first eigenvector after adjustment of the displacement field. A solution module for solving with the goal of minimizing the objective function to obtain a target displacement field. An adjustment module for adjusting the second picture frame according to the target displacement field.

[0076] For the specific definition of the image stabilization device of the shipborne camera, please refer to the definition of the image stabilization method of the shipborne camera above, which will not be repeated here. The various modules in the above-mentioned image stabilization device of the shipborne camera can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0077] The present application provides a computer device comprising one or more processors and a memory, wherein the memory stores computer-readable instructions. When the computer-readable instructions are executed by the one or more processors, the image stabilization method for the shipborne camera in any of the above embodiments is executed.

[0078] Schematically, as Figure 3 As shown, Figure 3 This is a schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. Figure 3 Computer device 300 includes a processing component 302, which further includes one or more processors, and memory resources represented by memory 301 for storing instructions executable by processing component 302, such as application programs. The application programs stored in memory 301 may include one or more modules, each corresponding to a set of instructions. Furthermore, processing component 302 is configured to execute the instructions to perform the steps of the shipborne camera image stabilization method described in any of the above-described embodiments.

[0079] The computer device 300 may further include a power supply component 303 configured to perform power management of the computer device 300 , a wired or wireless network interface 304 configured to connect the computer device 300 to a network, and an input / output (I / O) interface 305 .

[0080] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0081] The present application provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the image stabilization method for a shipborne camera in any of the above embodiments.

[0082] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referenced to each other.

[0083] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for stabilizing an image of a shipborne camera, characterized in that: include: Acquire target shipborne video; For a first picture frame and a second picture frame adjacent to each other in the target shipborne video, extract image features of the first picture frame and the second picture frame respectively to obtain a first feature vector and a second feature vector; determining a correlation between the first eigenvector and the second eigenvector; If the correlation degree is greater than a first threshold, constructing an objective function with a displacement field as a variable based on the first eigenvector and the second eigenvector; the objective function is used to reflect the difference between the second eigenvector and the first eigenvector adjusted by the displacement field; Solving the problem with the goal of minimizing the objective function to obtain a target displacement field; adjusting the second picture frame according to the target displacement field; If the correlation degree is not greater than the first threshold, the second picture frame remains unchanged.

2. The image stabilization method according to claim 1, wherein: The extracting the image features of the first picture frame and the second picture frame respectively to obtain a first feature vector and a second feature vector includes: Inputting the first picture frame into a feature extraction model to obtain the first feature vector; The second image frame is input into the feature extraction model to obtain the second feature vector.

3. The image stabilization method according to claim 2, wherein: The feature extraction model includes a visual geometry group neural network unit, a feature enhancement unit and a size restoration unit that are interconnected.

4. The image stabilization method according to claim 1, wherein: The determining the degree of association between the first feature vector and the second feature vector includes: The degree of association is determined according to a dot product of the first eigenvector and the second eigenvector.

5. The image stabilization method according to claim 1, wherein: The constructing of an objective function with the displacement field as a variable based on the first eigenvector and the second eigenvector includes: adjusting the second eigenvector using the displacement field; The objective function is obtained according to the adjusted Frobenius norm between the second eigenvector and the first eigenvector.

6. The image stabilization method according to claim 1, wherein: The adjusting the second picture frame according to the target displacement field includes: Determining corresponding translation amounts and rotation angles according to the displacement field; Obtaining a transformation matrix according to the translation amount and the rotation angle; The second picture frame is adjusted according to the transformation matrix.

7. An image stabilization device for a shipborne camera, characterized in that: include: Video acquisition module, used to acquire target shipborne video; A feature extraction module is configured to extract image features of a first picture frame and a second picture frame adjacent to each other in the target shipborne video, respectively, to obtain a first feature vector and a second feature vector; an association determination module, configured to determine a degree of association between the first eigenvector and the second eigenvector; an objective function construction module, configured to construct an objective function with a displacement field as a variable based on the relationship between the first eigenvector and the second eigenvector if the correlation degree is greater than a first threshold; the objective function is configured to reflect the difference between the second eigenvector and the first eigenvector adjusted by the displacement field; A solution module, configured to solve the problem with the goal of minimizing the objective function to obtain a target displacement field; an adjustment module, configured to adjust the second image frame according to the target displacement field; The image stabilization device is further configured to keep the second picture frame unchanged if the correlation degree is not greater than the first threshold.

8. A computer device, characterized in that: The device comprises one or more processors and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the one or more processors, the steps of the image stabilization method for a shipborne camera according to any one of claims 1 to 6 are performed.

9. A storage medium, characterized in that: The storage medium stores computer-readable instructions, which, when executed by one or more processors, enable the one or more processors to perform the steps of the image stabilization method for a shipborne camera as described in any one of claims 1 to 6.

Images