Marine moving target tracking method and system based on image visual guidance

Abstract

The invention discloses a sea moving target tracking method and system based on image visual guidance, and relates to the technical field of moving target tracking. A moving target to be tracked is positioned, a plurality of positioning points are selected, and then initial relative position parameters of the positioning points are determined; in the monitoring process of the moving target, new sea image frames are continuously obtained, the positions of positioning points in the sea image frames are recognized, then the positioning points are verified, error positioning caused by obstacles is eliminated, and it is ensured that all the positioning points belong to the same tracking target; and after the positioning points belong to the same tracking target, matching the tracking target by using a course template library established by a plurality of ship 3D models so as to fit and determine the course of the tracking target, determining the relative distance between the tracking target and a photoelectric observation value by establishing a ship body projection model, and tracking the tracking target in real time by using a proportional guidance method. Accurate positioning, stable tracking and attitude prediction are realized, and the tracking accuracy and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mobile target tracking, and in particular to a method and system for tracking a mobile target at sea based on image vision guidance. Background Art

[0002] With the increasing frequency of maritime transportation, fisheries, marine resource development and other activities, the safe management and tracking of maritime targets have become crucial. For example, real-time tracking of merchant ships, fishing boats, etc. can help avoid collision accidents and ensure maritime traffic safety. Tracking targets of illegal activities at sea is conducive to safeguarding maritime rights and security.

[0003] Some traditional maritime target tracking methods mainly rely on positioning technologies such as GPS, which are prone to positioning errors and even target loss in complex environments. In addition, some existing tracking methods do not fully consider the impact of environmental factors such as waves on moving targets, resulting in reduced tracking accuracy in maritime environments. At the same time, traditional methods are easily interfered with by factors such as obstacles when identifying target positioning points, resulting in mispositioning.

[0004] Therefore, in response to the above problems, there is an urgent need for a method and system for tracking mobile targets at sea based on image vision guidance. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a method and system for tracking mobile targets at sea based on image vision guidance, which solves the problems of mobile target tracking in complex marine environments being easily interfered with by obstacles, leading to mispositioning and difficulty in accurately determining the target heading and relative distance.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for tracking a mobile target at sea based on image vision guidance, comprising the following steps: S1, obtaining an initial image of a marine scene, outlining the edge contours of the object in the initial image, and then locating the mobile target to be tracked; S2, selecting multiple positioning points for the located mobile target, and identifying the relative position relationship of the positioning points in the mobile target based on the initial position and posture information of the mobile target, and then determining the initial relative position parameters of the positioning points; S3, during the monitoring process of the mobile target, continuously obtaining new marine image frames, identifying the positions of the positioning points in the marine image frames, and then verifying the positioning points to eliminate mispositioning caused by obstacles and ensure that the positioning points all belong to the same tracking target; S4, after ensuring that the positioning points all belong to the same tracking target, fitting to determine the heading of the tracking target, and determining the relative distance between the tracking target and the photoelectric observation value by establishing a hull projection model, and then determining the position coordinates of the tracking target; S5, based on the determined position coordinates of the tracking target, using the proportional guidance method to track the tracking target in real time.

[0007] Furthermore, the specific steps of S2 are: selecting and marking no less than three positioning points based on the block feature score of the moving target image; setting the center of mass of the moving target as the origin of the target coordinate system, defining the coordinate axes of the target coordinate system according to the structure and movement characteristics of the moving target, and then using the origin and coordinate axes of the target coordinate system as a reference, using geometric relationships and trigonometric functions to convert the coordinates of the positioning points in the geographic coordinate system into coordinates in the target coordinate system; identifying the relative position vector of each positioning point relative to the origin of the target coordinate system based on the coordinate difference, and then recording the identified relative position vector of the positioning point as the initial relative position parameter.

[0008] Furthermore, the specific method of selecting and marking no less than three positioning points based on the block feature score of the moving target image is: extracting the moving target image based on the initial image, and then dividing the moving target image into multiple rectangular blocks using a uniform grid; identifying the feature information of each block, the feature information including color features, texture features and shape features, and setting corresponding weights for the feature information according to the needs and characteristics of mobile target tracking, and then obtaining a comprehensive feature score for each block according to its feature information and the corresponding weight; sorting all blocks in descending order according to the comprehensive feature score, screening out a preset number of blocks with the highest score rankings as candidate blocks, and marking the center of mass point of each candidate block as a positioning point.

[0009] Furthermore, the specific steps of S3 are: using the multi-target tracking method to track the positioning points in real time and determine the real-time position parameters of the positioning points; combining the real-time position parameters of the positioning points with the initial relative position parameters to output the movement mode of the positioning points; obtaining the reference movement mode of the moving target based on the movement characteristics of the moving target, and then comparing the movement mode of each positioning point with the reference movement mode of the moving target, and outputting the error value of each positioning point and the reference movement of the moving target; comparing the error value of each positioning point and the reference movement of the moving target with the error threshold, when the error values of all positioning points and the reference movement of the moving target are lower than the error threshold, the positioning points all belong to the same tracking target and there is no obstacle; when there is a positioning point and the error value of the reference movement of the moving target is higher than or equal to the error threshold, there is an obstacle, the specific category of the obstacle is determined, and the positioning points with error values lower than the error threshold are screened out and marked as the accuracy positioning points of the moving target.

[0010] Furthermore, the specific analysis for determining the specific category of the obstacle is as follows: obtaining a marine image frame in which an obstacle is marked, and retrieving the position of a positioning point in the marine image frame where the error value between the positioning point and the reference movement of the moving target is greater than or equal to the error threshold; extracting the contour features of the object at the positioning point position, and then performing a similarity detection between the contour features of the object and the contour features of the permanent obstacles at sea, and determining the specific category of the object at the positioning point position based on the similarity detection result.

[0011] Furthermore, the specific steps of S4 are as follows: using the hull's central axis as the rotation axis to simulate the ship's posture under various headings, setting different sea conditions and weather scene parameters, rendering the rotated model, generating images of various ships under different headings and different environmental conditions, and building a ship heading template library; performing contour feature extraction on the image of the ship being tracked, and performing the same contour extraction operation on each image in the ship heading template library at the same time, matching the contour of the image of the target ship being tracked with the contours of all images in the ship heading template library based on similarity, obtaining the fitting value of each template image and the image to be tracked, and selecting the fitting value. The highest template library image, the heading corresponding to this image is the heading of the tracked target; the center of mass of the tracked target is used as the motion positioning reference point, and a monitoring framework including a fixed optoelectronic observation station is constructed. Then, by tracking the true length of the target ship and the pixel space proportion of the bow and stern in the optical television window, combined with the field of view angle characteristics of the optoelectronic pan-tilt platform, a hull projection model is established, and the hull projection model is used to identify the relative distance of the tracked target relative to the optoelectronic observation station; based on the relative distance, the geographical location of the optoelectronic observation station and the heading information of the tracked target, the geographical coordinates of the center of mass of the tracked target are determined to achieve closed-loop measurement of the motion state of the tracked target.

[0012] Furthermore, the specific steps of S5 are: obtaining the coordinate position of the ship to be tracked in real time, calculating the relative distance and line of sight angle between the own ship and the target, and dynamically calculating the proportional coefficient according to the current relative distance: setting the basic proportional coefficient and sensitivity enhancement coefficient and introducing the distance attenuation factor; based on the velocity vectors of the own ship and the target ship, respectively calculating the projection components of their velocities in the line of sight direction and the vertical direction, and obtaining the relative distance change rate and the line of sight angle change rate; combining the line of sight angle change rate with the adaptive proportional coefficient to generate a proportional guidance constraint instruction, and then solving the required heading angle of the own ship according to the instruction, and adjusting the heading in real time through the servo system to make the velocity vector point to the predicted target position; continuously monitoring the tracking error and environmental changes, and dynamically adjusting the proportional coefficient parameters.

[0013] A maritime mobile target tracking system based on image vision guidance, applying the above-mentioned maritime mobile target tracking method based on image vision guidance, comprises: an image acquisition and target positioning module, configured to acquire an initial image of a maritime scene, outline the edge contours of objects in the initial image, and thereby locate the mobile target to be tracked; a positioning point selection module, configured to select multiple positioning points for the located mobile target, and simultaneously identify the relative positional relationship of the positioning points within the mobile target based on the initial position and posture information of the mobile target, thereby determining the initial relative position parameters of the positioning points; a positioning point verification module, configured to continuously acquire new marine image frames during the monitoring of the mobile target, identify the positions of the positioning points in the marine image frames, and thereby verify the positioning points to eliminate mispositioning caused by obstacles and ensure that the positioning points all belong to the same tracking target; a position determination module, configured to, after ensuring that the positioning points all belong to the same tracking target, fit the heading of the tracking target, determine the relative distance between the tracking target and the photoelectric observation value by establishing a hull projection model, and thereby determine the position coordinates of the tracking target; and a tracking prediction module, configured to track the tracking target in real time using a proportional guidance method based on the determined position coordinates of the tracking target.

[0014] The present invention has the following beneficial effects:

[0015] This method and system for tracking mobile targets at sea based on image vision guidance uses an edge detection algorithm to outline the edge contours of objects in the initial image to locate the mobile target. It can quickly and accurately identify the target object in complex marine scenes, laying the foundation for subsequent tracking. The edge detection algorithm can highlight the boundary features of the object, effectively reduce background interference, and improve the accuracy and efficiency of target positioning; multiple positioning points are selected and their relative position relationship and initial relative position parameters in the mobile target are determined, providing a key reference for monitoring the target status during subsequent tracking. These positioning points are like "coordinate points" on the target. Through them, the position and posture changes of the target can be accurately described, making the tracking process more accurate and reliable. During the monitoring process, new image frames are continuously acquired and the position of the positioning points is identified. The positioning points are verified to eliminate mispositioning caused by obstacles. The maritime environment is complex and changeable, with many obstacles, and misjudgment is easy to occur. Through this verification mechanism, it can be ensured that the positioning points all belong to the same tracking target, which greatly improves the stability and accuracy of the tracking process and avoids tracking failures caused by mispositioning. After ensuring that the positioning points all belong to the same tracking target, it can be guaranteed that the entire tracking process always revolves around the target and will not deviate from the tracking target due to external interference. This is especially important for long-term and long-distance maritime target tracking, and can ensure the consistency and reliability of the tracking results. The heading template library established using 3D models of multiple ships is matched to the tracking target, and then the heading of the tracking target is determined by fitting. This method, through a large amount of template data and advanced matching algorithms, can accurately match the target vessel's heading, providing an important basis for subsequent tracking and control. The use of a 3D model makes heading fitting more intuitive and accurate, taking into account the target vessel's actual shape and posture. By establishing a hull projection model to determine the relative distance between the tracking target and the optoelectronic observation value, it is possible to obtain real-time information on the distance between the target and the observation equipment. This is of great significance for assessing the target's threat level, formulating tracking strategies, and conducting subsequent guidance and control. It provides data support for decision-making during the tracking process and improves the effectiveness and safety of tracking. Real-time tracking of the target can promptly grasp the target's dynamic changes and provide real-time feedback for subsequent tracking and control. The speed and heading of a moving target at sea may change at any time. Real-time tracking of the heading ensures that the tracking system can adjust the tracking strategy in a timely manner to maintain stable tracking of the target. Taking into account the complexity and uncertainty of the maritime environment, through the combination of multiple technical means, it has strong adaptability and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of a method for tracking a mobile target at sea based on image vision guidance according to the present invention.

[0017] Figure 2This is a structural diagram of a maritime mobile target tracking system based on image vision guidance according to the present invention.

[0018] Figure 3 This figure shows the relationship between the observation point and the position of the ship to be tracked in a method for tracking a mobile target at sea based on image vision guidance.

[0019] Figure 4 The figure shows the relative motion relationship between the tracking ship and the ship to be tracked in a method for tracking a mobile target at sea based on image vision guidance. DETAILED DESCRIPTION

[0020] The embodiments of the present application achieve stable, accurate and real-time tracking of mobile targets at sea through a method and system for tracking mobile targets at sea based on image vision guidance.

[0021] The overall idea of the embodiment of the present application is: obtain an initial image at sea and locate the target using edge detection; select a positioning point for the target and determine its initial relative position parameters; continuously obtain new image frames during monitoring and verify the positioning point to eliminate mispositioning; use the 3D model of the ship to establish a heading template library and match the target image to determine the target heading, determine the relative distance by establishing a hull projection model, and complete real-time tracking of the target.

[0022] See also Figure 1 An embodiment of the present invention provides a technical solution: a method for tracking a mobile target at sea based on image vision guidance, comprising the following steps: S1, acquiring an initial image of a marine scene, outlining the edge contours of objects in the initial image, and then locating the mobile target to be tracked; S2, selecting multiple positioning points for the located mobile target, and identifying the relative position relationship of the positioning points in the mobile target based on the initial position and posture information of the mobile target, and then determining the initial relative position parameters of the positioning points; S3, during the monitoring process of the mobile target, continuously acquiring new marine image frames, identifying the positions of the positioning points in the marine image frames, and then verifying the positioning points to eliminate mispositioning caused by obstacles and ensure that all positioning points belong to the same tracking target; S4, after ensuring that all positioning points belong to the same tracking target, fitting to determine the heading of the tracking target, and determining the relative distance between the tracking target and the photoelectric observation value by establishing a hull projection model, and then determining the position coordinates of the tracking target; S5, based on the determined position coordinates of the tracking target, using a proportional guidance method to track the tracking target in real time.

[0023] Specifically, the specific steps of S2 are: based on the block feature score of the moving target image, select and mark no less than three positioning points; set the center of mass of the moving target as the origin of the target coordinate system, define the coordinate axes of the target coordinate system according to the structure and movement characteristics of the moving target, and then use the origin and coordinate axes of the target coordinate system as a reference, use geometric relationships and trigonometric functions to convert the coordinates of the positioning points in the geographic coordinate system into coordinates in the target coordinate system; identify the relative position vector of each positioning point relative to the origin of the target coordinate system based on the coordinate difference, and then record the identified relative position vector of the positioning point as the initial relative position parameter.

[0024] The specific method of selecting and marking no less than three positioning points based on the block feature score of the moving target image is as follows: extracting the moving target image based on the initial image, and then dividing the moving target image into multiple rectangular blocks using a uniform grid; identifying the feature information of each block, the feature information includes color features, texture features and shape features, and setting corresponding weights for the feature information according to the needs and characteristics of mobile target tracking, and then for each block, performing weighted summation based on its feature information and corresponding weights to obtain a comprehensive feature score; sorting all blocks in descending order according to the comprehensive feature score, screening out a preset number of blocks with the highest score as candidate blocks, and marking the center of mass point of each candidate block as a positioning point. The preset number is dynamically adjusted according to factors such as the size and complexity of the moving target.

[0025] In this embodiment, the specific logical steps of using the edge detection algorithm to outline the edge contour of the object in the initial image are as follows: converting the initial color image into a grayscale image to reduce the amount of data and facilitate subsequent processing; using a filtering algorithm to remove noise in the image, such as Gaussian filtering, which smoothes the image by performing weighted averaging on each pixel in the image and its neighborhood, thereby reducing the interference of noise on edge detection; using an operator (such as the Sobel operator, the Prewitt operator, etc.) to calculate the gradient of the image in the horizontal and vertical directions. Taking the Sobel operator as an example, it has convolution kernels in the horizontal and vertical directions, respectively, and obtains the image by performing convolution operation with the image. Gradients in the horizontal and vertical directions; in the gradient amplitude image, for each pixel, check the gradient amplitude of the adjacent pixels in the gradient direction. If the gradient amplitude of the pixel is not the local maximum in the gradient direction, set its amplitude to 0 to refine the edge; set two thresholds, namely the maximum threshold and the minimum threshold, and mark the pixels with gradient amplitude greater than the maximum threshold as strong edge points, the pixels with gradient amplitude less than the minimum threshold as non-edge points, and the pixels between the maximum threshold and the minimum threshold as weak edge points; check whether the weak edge points are connected to the strong edge points. If so, mark them as edge points, otherwise mark them as non-edge points.

[0026] The center of mass of a moving object is its mass center in two-dimensional or three-dimensional space. In image processing, the pixels of an object can be considered as points of mass with the same mass, and the center of mass is the average position of these points. The center of mass of each candidate block is the average position of the pixels within that block, reflecting the block's geometric center.

[0027] The specific logical steps for identifying the relative position vector of each positioning point relative to the origin of the target coordinate system based on the coordinate difference are as follows: determine the origin coordinates of the target coordinate system, that is, the center of mass coordinates of the mobile target, obtain the coordinates of each positioning point in the target coordinate system, calculate the coordinate difference of each positioning point relative to the origin of the target coordinate system, and obtain the relative position vector.

[0028] Color features describe the distribution of colors within a block and reflect the visual appearance of the target. By calculating the color histogram of the block and counting the frequency of occurrence of different colors within the block, the average color value of the block can also be calculated, that is, the color values of all pixels in the block are averaged. The color feature is represented by a vector, and each element of the vector corresponds to the frequency of a color interval. The average color value is represented by the values of the three RGB components.

[0029] Texture features reflect the spatial distribution of pixel grayscale values in an image and embody the texture characteristics of the target surface. Texture features are extracted using algorithms such as gray-level co-occurrence matrix (GLCM) and local binary pattern (LBP). GLCM describes texture by counting the frequency of grayscale combinations of two pixels in a specific spatial position relationship in an image. LBP characterizes texture by generating a binary pattern by comparing the grayscale values of the central pixel with those of the neighboring pixels. GLCM can extract statistics such as contrast, correlation, energy, and homogeneity as quantitative values of texture features. LBP can convert the generated binary pattern into a decimal number and count the frequency of occurrence of different decimal numbers as texture features.

[0030] Shape features describe the geometric characteristics of the block, such as size, aspect ratio, circularity, etc., and are identified by calculating parameters such as the area, perimeter, and aspect ratio of the enclosing rectangle of the block.

[0031] The specific method of setting corresponding weights for feature information is as follows: according to the needs and characteristics of mobile target tracking, domain experts directly set weights for color features, texture features, and shape features based on experience. For example, if the color feature of the target is relatively obvious and plays a key role in tracking, the color feature can be given a higher weight, such as 0.5, and the weights of texture features and shape features are set to 0.3 and 0.2 respectively; a large amount of training data can also be collected, each data contains the feature information of the block and the corresponding tracking effect evaluation indicators (such as tracking accuracy, stability, etc.), and machine learning algorithms (such as linear regression, decision trees, etc.) are used to learn the relationship between feature information and tracking effect, thereby determining the weight of each feature.

[0032] The edge detection algorithm is used to outline the edge of the object, which can clearly distinguish the target from the background, accurately find the moving target to be tracked, and provide a precise starting point for subsequent tracking; the positioning point is selected based on the block feature score, and the positions with significant features on the target can be selected as positioning points. These positioning points are easier to identify and track in the subsequent tracking process, which improves the accuracy and stability of tracking; the center of mass of the moving target is set as the origin of the target coordinate system, and the coordinate axis is defined according to the target structure and motion characteristics, so that the target's motion and posture changes can be conveniently described and analyzed in this coordinate system. Through coordinate transformation, the coordinates of the positioning points are unified to the target coordinate system, which facilitates the subsequent calculation of the relative position relationship of the positioning points, providing strong support for accurate tracking of the target.

[0033] Specifically, the specific steps of S3 are: using the multi-target tracking method to track the positioning points in real time and determine the real-time position parameters of the positioning points; combining the real-time position parameters of the positioning points with the initial relative position parameters to output the movement mode of the positioning points; obtaining the reference movement mode of the moving target based on the movement characteristics of the moving target, and then comparing the movement mode of each positioning point with the reference movement mode of the moving target, and outputting the error value of each positioning point and the reference movement of the moving target; comparing the error value of each positioning point and the reference movement of the moving target with the error threshold, when the error values of all positioning points and the reference movement of the moving target are lower than the error threshold, the positioning points all belong to the same tracking target and there is no obstacle; when there is a positioning point and the error value of the reference movement of the moving target is higher than or equal to the error threshold, there is an obstacle, the specific category of the obstacle is determined, and the positioning points with error values lower than the error threshold are screened out and marked as the accuracy positioning points of the moving target.

[0034] The specific analysis to determine the specific category of obstacles is as follows: obtain the marine image frame when the obstacle is marked, and retrieve the position of the positioning point in the marine image frame where the error value between the positioning point and the reference movement of the moving target is greater than or equal to the error threshold; extract the contour features of the object at the positioning point position, and then perform similarity detection on the contour features of the object and the contour features of the permanent obstacles at sea, and determine the specific category of the object at the positioning point position based on the similarity detection results.

[0035] In this embodiment, the multi-target tracking method is used to track the positioning point in real time. The specific steps for determining the real-time position parameters of the positioning point are as follows: first, in each frame of newly acquired marine image, the previously marked positioning point information is used as the initial search area to narrow the search range and improve the tracking efficiency; then, a suitable multi-target tracking algorithm is used, such as the Kalman filter algorithm or the particle filter algorithm. Taking the Kalman filter as an example, it estimates the state of the positioning point through two steps of prediction and update. In the prediction step, the position and speed of the positioning point at the next moment are predicted based on the motion model of the positioning point. In the update step, the prediction result is fused with the positioning point observation information obtained in the current image through feature matching and other methods to obtain more accurate real-time position parameters of the positioning point; finally, the above process is repeated to track the positioning point in each frame of the image in real time and update the position parameters to achieve accurate tracking of the positioning point in continuous image frames.

[0036] The specific steps of outputting the movement mode of the positioning point by combining the real-time position parameters and the initial relative position parameters of the positioning point are as follows: calculating the difference between the real-time position parameters and the initial relative position parameters of the positioning point in the target coordinate system; analyzing the movement trend of the positioning point based on the change of these differences over time, such as uniform linear motion, variable speed motion or curved motion; classifying and describing the movement trend of the positioning point, and outputting the movement mode of the positioning point.

[0037] The steps for obtaining the reference movement pattern of a mobile target are as follows: collecting the position information of multiple positioning points of the mobile target within a period of time and the corresponding time information; analyzing the motion trajectory and speed changes of these positioning points, and modeling and summarizing its movement pattern according to the type of mobile target (such as ships, buoys, etc.) and general movement characteristics. For example, for ships, they usually have a relatively stable sailing direction and speed, and their reference movement pattern can be determined by fitting their sailing trajectory, such as sailing in a straight line or sailing in a curve with a certain curvature.

[0038] The steps for obtaining the error value between each positioning point and the reference movement of the moving target are as follows: for each positioning point, obtain the real-time position parameters according to its movement mode, obtain the theoretical position that the positioning point should be at at the current moment according to the reference movement mode of the moving target, compare the actual real-time position of the positioning point with the theoretical position, and calculate the distance difference or coordinate difference between the two. Specifically, the Euclidean distance formula is usually used to calculate the error value.

[0039] The error threshold is set according to the specific tracking accuracy requirements, the characteristics of the moving target, and the complexity of the marine environment. The specific steps are: through a large number of experiments and data analysis, the error distribution between the positioning point and the reference movement of the moving target under normal tracking conditions is statistically analyzed, and then a suitable value is selected as the error threshold, so that under normal circumstances, errors caused by factors such as noise can be accepted, and when large errors occur, possible obstacles or other abnormal conditions can be detected in time. For example, for tracking tasks with high precision requirements, the error threshold may be set smaller. For some tasks with high real-time requirements but relatively low precision requirements, the error threshold can be appropriately relaxed. The error threshold can also be dynamically adjusted according to factors such as the speed and size of the moving target. For example, when the moving target is faster or larger in size, the error threshold can be increased accordingly.

[0040] The specific steps for extracting the contour features of the object at the positioning point are as follows: preprocessing the marine image frames when the obstacle is marked, such as grayscale and noise reduction, to improve image quality and reduce interference; using an edge detection algorithm to detect the edge of the object at the positioning point; using a chain code-based contour extraction method to connect the edge points into contour lines and perform contour extraction on the detected edges; and describing the contour features, such as calculating the contour perimeter, area, moment features (such as central moment, Hu moment, etc.), Fourier descriptors, etc. These features can reflect the shape and structural information of the object contour and serve as the contour features of the object at the positioning point.

[0041] The specific steps for performing similarity detection between the object contour features and the contour features of permanent obstacles at sea and determining the specific category of the object at the positioning point are as follows: establishing a contour feature library of permanent obstacles at sea, collecting contour feature data of various common permanent obstacles (such as lighthouses, reefs, etc.), and storing them in categories; for the extracted contour features of the object at the positioning point, similarity calculation is performed with each permanent obstacle contour feature in the feature library. Commonly used similarity calculation methods include Euclidean distance, cosine similarity, and Mahalanobis distance. Taking Euclidean distance as an example, the distance between the object contour feature vector and the permanent obstacle contour feature vector is calculated. The smaller the distance, the higher the similarity.

[0042] By using the multi-target tracking method to track positioning points in real time and combining multiple parameters and comparison methods, it can effectively eliminate interference from obstacles, accurately identify the positioning points of moving targets, and improve the accuracy and reliability of tracking; determine the specific categories of obstacles and screen out precise positioning points, so that the system can still maintain effective tracking of moving targets when facing various interference factors in complex marine environments, thereby enhancing the stability and adaptability of the system.

[0043] Specifically, the specific steps of S4 are as follows: using 3ds max software to build 3D models of various common ship types, using the hull's central axis as the rotation axis, rotating the 3D models at different angles to simulate the ship's posture under various headings, using the V-Ray rendering engine, setting different sea conditions and weather scene parameters, rendering the rotated model, generating images of various ships at different headings and different environmental conditions, and building a ship heading template library; extracting contour features from the image of the ship being tracked, and performing the same contour extraction operation on each image in the ship heading template library, matching the contour of the image of the target ship being tracked with the contours of all images in the ship heading template library based on similarity, and obtaining each template image. The template library image with the highest fitting value is selected based on the fitting value of the image to be tracked. The heading corresponding to this image is the heading of the tracked target. The center of mass of the tracked target is used as the motion positioning reference point, and a monitoring framework including a fixed optoelectronic observation station is constructed. Then, by tracking the true length of the target ship and the pixel space proportion of the bow and stern in the optical television window, combined with the field of view angle characteristics of the optoelectronic pan-tilt platform, a hull projection model is established. The hull projection model is used to identify the relative distance of the tracked target relative to the optoelectronic observation station. According to the relative distance, the geographical location of the optoelectronic observation station and the heading information of the tracked target, the geographical coordinates of the center of mass of the tracked target are determined, realizing closed-loop measurement of the motion state of the tracked target.

[0044] In this implementation, considering that the platform has no direct ranging equipment such as radar, and the current tracking methods (proportional guidance, line of sight guidance) all require the other party's position information, it is necessary to obtain the distance of the target to be tracked through optoelectronic equipment. The subsequent ranging method needs to know its heading first. Therefore, a variety of ship heading template libraries are constructed, and 3ds max software is used to build 3D models of various common ship types. After rotating different angles with the central axis as the rotation axis, V-Ray rendering is used (simulating different sea conditions and weather scenes) to obtain a template library of various ship headings. After extracting the contour of the input ship image, it is matched with the template library (the contour is also extracted), and the SSIM image similarity is used to calculate the fitting value. The heading corresponding to the template library image with the highest fitting value is the heading of the image to be matched, and tracking is performed after obtaining the heading.

[0045] For the position tracking of the target, photoelectric image acquisition is used. The specific analysis is as follows: Figure 3, set the target ship DB to be tracked with its center of mass C as the motion positioning reference point, and construct a monitoring framework including a fixed optoelectronic observation station A based on the north reference N of the geographic coordinate system. The optoelectronic observation station A forms a fan-shaped monitoring field of view through the line of sight boundaries AB and AD, and captures the real-time imaging data of the target ship's projection outline FE through the optical television window. Under the condition that the true heading parameters of the target ship are known, the geographic coordinates of the center of mass C need to be solved through the geometric model to realize the closed-loop measurement of the motion state. The specific calculation is based on the relationship between the true length of the target ship and the pixel space proportion of the bow and stern in the optical television window. Combined with the field of view angle characteristics of the optoelectronic pan-tilt platform, the hull projection model is established to calculate the relative distance to the target. The positioning solution result is directly connected to the subsequent guidance module. By improving the proportional guidance method or optimizing the line of sight guidance method, a complete dynamic target tracking control loop is constructed.

[0046] The ship's true heading DB has been calculated, so the angle ω is known. By aligning the center axis of the photoelectric imaging with the center C of the ship, the pixel coordinate distribution parameters of the bow and stern feature points in the optical video window are obtained. Based on the principle of the camera model, the total number of horizontal pixels of the photoelectric sensor is set to W. pixel , the corresponding physical size is L sensor , the actual length of the ship is L, the focal length of the imaging system is f, and the pixel offsets of the bow and stern relative to the center C on the imaging plane are Δd bow and Δd stern , then the horizontal pixel coverage η is: The physical size of a photoelectric sensor unit pixel is: The total span of the ship's projection is: D img =η·L sensor =η·W pixel ·δ pix ; According to the principle of geometric optics, the half angle of field of view satisfy: The full field of view angle is calculated as: Similarly, angles α and β can be solved separately. The angle between the photoelectric head and true north is known, so angle θ can be calculated. The true length DB of the ship is known, so the AC length can be calculated by the following formula: Therefore, the coordinates of point C (x C ,y C )for: Subsequently, an improved method can be used for tracking based on the coordinates of point C.

[0047] The ship type parameters in the 3D model construction parameters are obtained by measuring the actual dimensions of common ship types, consulting ship design drawings, etc. to obtain the ship type's length, width, height, hull structure and other parameters, which are used to build an accurate 3D model in 3ds max software.

[0048] The rotation angle parameters are obtained as follows: setting them according to actual needs, for example, rotating from 0 to 360 degrees at intervals of 10 or 15 degrees to simulate the posture of the ship under various possible headings.

[0049] The rendering scene parameters are obtained as follows: sea condition parameters can refer to marine environment monitoring data, such as wave height, current speed, etc. Weather scene parameters are obtained through meteorological data, including light intensity, cloud thickness, rain and fog concentration, etc. These parameters are input into the V-Ray rendering engine for scene setting.

[0050] The image contour extraction and matching parameter acquisition methods are as follows: for image contour extraction parameters, mature image contour extraction algorithms, such as the Canny algorithm, are adopted. Parameters such as the threshold in the algorithm are determined through experimental debugging to adapt to ship images with different clarity and contrast, ensuring accurate contour extraction. For matching parameters, when calculating image similarity based on SSIM (structural similarity index), parameters such as brightness contrast sensitivity, contrast contrast sensitivity, and structural contrast sensitivity in the SSIM algorithm are optimized and adjusted according to the characteristics of ship images to improve the accuracy of fitting value calculation.

[0051] The hull projection model parameters are obtained as follows: for the true length parameter of the ship, the real length data of the tracked target ship is obtained from the ship registration information, design data or actual measurements; for the pixel space ratio parameter, the projection outline of the target ship captured in the optical television window is processed, the number of pixels of the bow and stern in the image is calculated, and compared with the total number of pixels in the entire optical television window to obtain the pixel space ratio of the bow and stern.

[0052] To obtain the field of view angle parameters of the optoelectronic pan / tilt platform, refer to the product manual or technical parameter document of the optoelectronic pan / tilt platform to obtain its horizontal and vertical field of view angle data.

[0053] The geographic coordinate parameters are obtained as follows: for the geographic location parameters of the photoelectric observation station, the global positioning system (GPS) equipment is used to locate the photoelectric observation station to obtain its geographic coordinate information such as longitude, latitude and altitude; for the tracking target heading information parameters, the heading angle value of the tracking target is obtained through the ship heading template library matching method described above.

[0054] By constructing a ship heading template library containing different headings and environmental conditions, and performing contour extraction and matching on the images, it is possible to effectively cope with complex and changeable maritime environments, accurately determine the heading of the tracked target, and improve the accuracy and reliability of heading recognition; using the center of mass as the reference point, a hull projection model is established in combination with the ship size, pixel space ratio, and optoelectronic pan-tilt field of view to achieve accurate measurement of the relative distance to the tracked target, providing key data support for subsequent closed-loop measurement of the motion state; using relative distance, geographic location, and heading information to determine the geographic coordinates of the center of mass of the tracked target, achieving closed-loop measurement of the motion state of the tracked target, and being able to comprehensively and real-timely grasp the motion trajectory and status of the target ship, providing a strong basis for the formulation and adjustment of tracking strategies; this design can be adapted to common tracking methods such as improved proportional guidance or line-of-sight guidance, enhancing the versatility and flexibility of the tracking system, and meeting the needs of maritime target tracking in different scenarios.

[0055] In this implementation scheme, the specific steps of S5 are: calculate the relative distance and line of sight angle (the angle between the aiming line and the reference line) between the ship and the target to be tracked in real time through the optoelectronic equipment and the hull projection model in S4 to obtain the coordinate position of the target; dynamically calculate the proportional coefficient according to the current relative distance; set the basic proportional coefficient and the sensitivity enhancement coefficient to ensure that the coefficient value range meets the system stability conditions; introduce a distance attenuation factor to make the proportional coefficient decay exponentially with increasing distance, approaching the basic value at long distances and superimposing enhancement items to improve sensitivity at close distances. The reference distance and attenuation multiples are adjusted according to the actual scene needs. The system can preset the speed of the own ship and the target ship and support dynamic adjustment. Based on the velocity vectors of the own ship and the target ship, the projection components of their velocities in the line of sight and the vertical direction are calculated respectively to obtain the relative distance change rate and the line of sight angle change rate. The line of sight angle change rate is combined with the adaptive proportional coefficient to generate a proportional guidance constraint instruction. According to the instruction, the required heading angle (ballistic deviation angle) of the own ship is solved, and the heading is adjusted in real time through the steering gear system to make the velocity vector point to the predicted target position. The system continuously monitors the tracking error and environmental changes and dynamically adjusts the proportional coefficient parameters (such as base value, attenuation factor, etc.) to adapt to target maneuvers or sea condition fluctuations.

[0056] Specifically, see Figure 4 , obtain the real-time coordinates of the ship C to be tracked (x C ,y C ), the proportional guidance method is used for tracking, A is the position of the ship, C is the position of the ship to be tracked, AC is the line of sight, also known as the aiming line, r is the distance between the two, q is the angle between the line of sight and the reference line, λ A ,λ C are the angles between the velocity vectors of our ship and the ship to be tracked and the line of sight, called the advance angle or leading angle, σ A , σ Care the angles between the velocity vectors of our ship and the target to be tracked and the baseline, respectively, which is called the trajectory deviation angle. Based on the relative motion relationship between our ship and the target to be tracked, the rate of change of the relative distance can be obtained by projecting their velocities on the line of sight: The rate of change of the side angle can be obtained from the projection of the two velocities in the direction perpendicular to the line of sight: According to the constraint equation of the proportional guidance method, we have: Where N is the proportional constant. In the traditional proportional guidance method, it is a fixed parameter. However, considering that its value needs to take into account the contradiction between long-distance tracking stability and close-range maneuvering sensitivity, in view of the large range of target distance changes in the ship dynamic tracking scenario, in order to improve the guidance performance, this scheme proposes an adaptive proportional coefficient dynamic adjustment method based on the exponential decay law, which changes the fixed parameter N to a form that changes with the relative distance r: N(r) = N0 + ke -μr , where N0 is the basic proportional coefficient (1≤N0≤5), k is the sensitivity enhancement coefficient, and the value satisfies 0≤k≤2(N0-1) to avoid oscillation caused by excessive proportional coefficient, and μ is the distance attenuation factor, which is determined by the preset reference distance r ref Sure: Where ξ is r = r ref The attenuation multiple when (usually ξ=10), N0, k, ξ, r ref The specific value is dynamically adjusted based on the mission site environment, target relative speed and expected response speed of the tracking system. A +λ A =σ C +λ C Based on the above, a definite solution can be obtained. According to the obtained r(t) and q(t), the motion trajectory of our ship relative to the target can be obtained. The improved method takes into account the performance requirements of stable capture at medium and long distances and rapid response at close range while meeting the system stability. It has good adaptability and scalability and is suitable for complex dynamic tracking scenarios of marine targets.

[0057] The scaling factor dynamically adjusts with distance, balancing long-range stability with close-range agility. Parameters can be optimized in real time based on target speed, sea conditions, and other factors, adapting to complex scenarios. The method can be integrated into the automatic tracking systems of various vessels, including automated patrol vessels. For example, using a certain autonomous surface platform tracking a maneuvering target, the initial scaling factor is set to 3, and the sensitivity enhancement factor is 1.5. When the target's distance falls below the reference distance, the scaling factor automatically increases to 4.5 to enhance maneuverability, keeping the tracking error within ±3 meters.

[0058] See also Figure 2A maritime mobile target tracking system based on image vision guidance, applying the above-mentioned maritime mobile target tracking method based on image vision guidance, includes: an image acquisition and target positioning module, configured to acquire an initial image of a maritime scene, outline the edge contours of objects in the initial image, and thereby locate the mobile target to be tracked; a positioning point selection module, configured to select multiple positioning points for the mobile target to be located, and simultaneously identify the relative positional relationship of the positioning points within the mobile target based on the initial position and attitude information of the mobile target, thereby determining the initial relative position parameters of the positioning points; a positioning point verification module, configured to continuously acquire new marine image frames during the monitoring of the mobile target, identify the positions of the positioning points in the marine image frames, and thereby verify the positioning points to eliminate mispositioning caused by obstacles and ensure that the positioning points all belong to the same tracking target; a position determination module, configured to, after ensuring that the positioning points all belong to the same tracking target, fit the heading of the tracking target, determine the relative distance between the tracking target and the photoelectric observation value by establishing a hull projection model, and thereby determine the position coordinates of the tracking target; and a tracking prediction module, configured to track the tracking target in real time using a proportional guidance method based on the determined position coordinates of the tracking target.

[0059] In summary, this application has at least the following effects:

[0060] The edge detection algorithm can accurately locate moving targets, select multiple positioning points and verify them, effectively eliminate interference from obstacles, ensure positioning accuracy, and thus achieve precise tracking of the target; use the ship's 3D model to establish a heading template library to match the target image to determine its heading, and combine the hull projection model to determine the relative distance. It can better adapt to different ship types and sea conditions in complex maritime environments, and improve the reliability and stability of tracking; continuously acquire new image frames and process them in real time, which can timely track the dynamic changes of the target and adjust the tracking strategy in real time to meet the needs of real-time tracking of moving targets at sea.

[0061] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods or systems. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0062] The present invention is described with reference to the flowcharts and structure diagrams of the methods and systems according to the embodiments of the present invention. It should be understood that each process and combination of modules in the flowcharts and structure diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate the instructions for implementing the processes in the flowcharts. Figure 1 process or processes and structures Figure 1 A device that specifies functionality within a module or modules.

[0063] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 process or processes and structures Figure 1 Functionality specified in a module or modules.

[0064] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 process or processes and structures Figure 1 Steps for specifying functionality in a module or multiple modules.

[0065] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0066] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for tracking a mobile target at sea based on image vision guidance, characterized in that: The following steps are involved: S1, obtains the initial image of the marine scene, outlines the edge contours of the objects in the initial image, and then locates the moving target to be tracked; S2, for the mobile target to be located, select multiple positioning points, and identify the relative position relationship of the positioning points in the mobile target based on the initial position and posture information of the mobile target, and then determine the initial relative position parameters of the positioning points; S3, during the monitoring of the moving target, continuously acquire new marine image frames, identify the positions of the positioning points in the marine image frames, and then verify the positioning points to eliminate mispositioning caused by obstacles and ensure that the positioning points all belong to the same tracking target; S4, after ensuring that all the positioning points belong to the same tracking target, the heading of the tracking target is determined by fitting, and the relative distance between the tracking target and the photoelectric observation value is determined by establishing a hull projection model, thereby determining the position coordinates of the tracking target; S5, based on the determined position coordinates of the tracking target, the tracking target is tracked in real time using the proportional guidance method.

2. The method for tracking a mobile target at sea based on image vision guidance according to claim 1 is characterized in that ,The specific steps of S2 are as follows: select and mark no less than three ,locating points based on the block feature scores of the ,moving target image; The center of mass of the moving target is set as the origin of the target coordinate system. According to the structure and movement characteristics of the moving target, the coordinate axes of the target coordinate system are defined. Then, based on the origin and coordinate axes of the target coordinate system, the coordinates of the positioning point in the geographic coordinate system are converted to the coordinates in the target coordinate system. The relative position vector of each positioning point relative to the origin of the target coordinate system is identified based on the coordinate difference, and the identified relative position vector of the positioning point is recorded as the initial relative position parameter.

3. The method for tracking a mobile target at sea based on image vision guidance according to claim 2 is characterized in that ,The specific method of selecting and marking at least three ,locating points based on the block feature score of the ,moving target image based on the initial image, and ,then dividing the moving target image into a plurality of rectangular blocks using ,a uniform grid; Identify the feature information of each block, including color, texture, and shape features. According to the needs and characteristics of mobile target tracking, set corresponding weights for the feature information. Then, for each block, obtain a comprehensive feature score based on its feature information and corresponding weights. All blocks are sorted in descending order according to the comprehensive feature scores, and a preset number of blocks with the highest scores are selected as candidate blocks. For each candidate block, the centroid point is marked as the positioning point.

4. The method for tracking a mobile target at sea based on image vision guidance according to claim 1 is characterized in that ,The specific steps of S3 are: use the multi-target tracking method to track the ,locating points in real time and determine the real-time position parameters of the ,locating points; Combining the real-time position parameters of the positioning point with the initial relative position parameters, output the movement mode of the positioning point; Obtaining a reference movement pattern of the moving target based on the movement characteristics of the moving target, and then comparing the movement pattern of each positioning point with the reference movement pattern of the moving target, and outputting the error value between each positioning point and the reference movement of the moving target; The error value between each positioning point and the reference movement of the moving target is compared with the error threshold. When the error values between all positioning points and the reference movement of the moving target are lower than the error threshold, the positioning points belong to the same tracking target and there is no obstacle. When the error value between a positioning point and the reference movement of a moving target is higher than or equal to the error threshold, an obstacle exists. The specific category of the obstacle is determined, and the positioning points with error values lower than the error threshold are selected and marked as the accuracy positioning points of the moving target.

5. The method for tracking a mobile target at sea based on image vision guidance according to claim 4 is characterized in that ,The specific analysis for determining the specific category of the ,obstacle is as follows: ,acquiring the marine image frame when the obstacle is marked ,and retrieving the location of the positioning point in the marine image frame where ,the error value between the positioning point and the reference movement of the ,moving target is greater than or equal to the error threshold; The contour features of the object at the positioning point are extracted, and then the similarity between the contour features of the object and the contour features of the permanent obstacles at sea are detected. Based on the similarity detection results, the specific category of the object at the positioning point is determined.

6. The method for tracking a mobile target at sea based on image vision guidance according to claim 1 is characterized in that ,The specific steps of S4 are as follows: ,Simulate the attitude of the ship under various headings using the ,hull axis as the rotation axis, set different sea conditions and ,weather scene parameters, render the rotated model, generate ,images of various ships under different headings and different ,environmental conditions, and build a ship heading template library; The contour feature extraction is performed on the image of the ship being tracked. At the same time, the same contour extraction operation is performed on each image in the ship heading template library. Based on the similarity, the contour of the target ship image is matched with the contours of all images in the ship heading template library. The fitting value between each template image and the image to be tracked is obtained. The template library image with the highest fitting value is selected, and the heading corresponding to this image is the heading of the tracking target. The center of mass of the target is used as the reference point for motion positioning. A monitoring framework including a fixed optoelectronic observation station is constructed. Then, by tracking the true length of the target ship and the pixel space proportion of its bow and stern in the optical video window, combined with the field of view characteristics of the optoelectronic pan / tilt, a hull projection model is established. The hull projection model is used to identify the relative distance of the target to the optoelectronic observation station. According to the relative distance, the geographical location of the optoelectronic observation station and the heading information of the tracked target, the geographical coordinates of the center of mass of the tracked target are determined to achieve closed-loop measurement of the motion state of the tracked target.

7. The method for tracking a mobile target at sea based on image vision guidance according to claim 1 is characterized in that ,The specific steps of S5 are as follows: ,acquire the coordinate position of the ship to be tracked in real time, ,calculate the relative distance and the sight angle between the ship and the target, ,and dynamically calculate the proportional coefficient based on the current ,relative distance: Set the basic scale factor and sensitivity enhancement factor and introduce the distance attenuation factor; Based on the velocity vectors of the own ship and the target ship, the projection components of their velocities in the line of sight direction and the vertical direction are calculated respectively to obtain the relative distance change rate and the line of sight angle change rate; The rate of change of the sight angle is combined with the adaptive proportional coefficient to generate a proportional guidance constraint instruction. The required heading angle of the own ship is then calculated based on the instruction, and the heading is adjusted in real time through the steering gear system to make the velocity vector face the target predicted position. Continuously monitor tracking errors and environmental changes, and dynamically adjust the scale factor parameters.

8. A system for tracking a mobile target at sea based on image vision guidance, applying a method for tracking a mobile target at sea based on image vision guidance according to any one of claims 1 to 7, characterized in that: include: The image acquisition and target positioning module is used to obtain the initial image of the marine scene, outline the edge contours of the objects in the initial image, and then locate the moving target to be tracked; A positioning point selection module is used to select multiple positioning points for the mobile target to be positioned, and identify the relative position relationship of the positioning points in the mobile target based on the initial position and posture information of the mobile target, and then determine the initial relative position parameters of the positioning points; The positioning point verification module is used to continuously acquire new marine image frames during the monitoring of moving targets, identify the positions of positioning points in the marine image frames, and then verify the positioning points to eliminate mispositioning caused by obstacles and ensure that the positioning points belong to the same tracking target; The position determination module is used to ensure that all positioning points belong to the same tracking target, fit the heading of the tracking target, and determine the relative distance between the tracking target and the photoelectric observation value by establishing a hull projection model, and then determine the position coordinates of the tracking target; The tracking prediction module is used to track the target in real time using the proportional guidance method based on the determined position coordinates of the target.

Images