Water surface ship continuous identification method and system based on visual angle of unmanned aerial vehicle, and storage medium

Abstract

The invention discloses a water surface ship continuous identification method and system based on a visual angle of an unmanned aerial vehicle, and a storage medium. The method comprises the following steps: S1, identifying a ship in an initial image and calculating a motion parameter; s2, calculating a specific frame extraction interval; s3, extracting a new image according to a specific frame extraction interval, identifying ships in the image and calculating geographic coordinates of the ships; s4, predicting the position and the position interval of the ship by adopting a Kalman filtering model; s5, judging whether a plurality of ships exist in the position interval or not, if so, adjusting the range of the position interval, and otherwise, entering step S6; s6, matching the ship in the newly extracted image with the position interval; s7, judging whether ships which are not matched with the position interval exist or not, if not, entering the step S8, and otherwise, entering the step S9; s8, calculating motion parameters of the ship, and returning to the step S2; s9, extracting the next image according to the minimum frame extraction interval, identifying the ship, calculating motion parameters and returning to the step S2; the device can efficiently and accurately identify ships on the water surface.

Description

Technical Field

[0001] The present invention relates to unmanned aerial vehicle image recognition, and in particular to a method, system and storage medium for continuous recognition of surface vessels based on the perspective of an unmanned aerial vehicle. Background Art

[0002] Carrying out target recognition tasks in water environments is a long-standing challenge. Traditional human recognition cannot obtain a high field of view and is not easy to accurately locate problem areas. Although using fixed high-position cameras can obtain a high field of view, since surface targets are usually mobile, a large number of fixed cameras need to be deployed to fully cover them, which will incur high hardware costs. In addition, in specific locations in wider water areas, such as the center of the water body, fixed cameras have low recognition resolution and are difficult to install, which may result in blind spots in the field of view. The use of drones to perform recognition tasks can overcome these problems. It not only provides a high field of view, but also fully covers the mission water surface area. In addition, drones equipped with algorithm models can also track moving objects on the water surface in real time. Therefore, drones are increasingly used to perform target recognition and tracking tasks in complex water surface scenes such as ports.

[0003] Surface targets, especially those controlled by humans, usually have specific motion characteristics. In current target recognition algorithms, the intersection-over-union (IOU)-based method is usually used to determine the same object. However, for surface targets with specific motion trajectories, especially when the targets move quickly or overlap with each other, this method cannot clearly distinguish the targets due to the sensitivity of IOU to boundary overlap, resulting in reduced accuracy. At the same time, when multiple targets overlap, the complexity and time cost of calculating IOU will also reduce the detection efficiency.

[0004] Due to the motion characteristics of surface targets, the target area is constantly changing. In order to ensure the accuracy of recognition, it is necessary to reduce the frame extraction interval of the extracted image in the captured video as much as possible. However, this will increase the total amount of processed data and the processing efficiency is low. In addition, the overlapping of surface targets may be more complicated. When IOU processes targets in these dynamic environments, it is not only inefficient but also prone to misjudgment or target loss. Summary of the invention

[0005] Purpose of the invention: The purpose of the present invention is to provide a method, system and storage medium for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle, which can efficiently and accurately identify surface vessels.

[0006] Technical solution: The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle of the present invention comprises the following steps:

[0007] S1, extracting two initial adjacent images in the video taken by the drone according to the default frame extraction interval, and calculating the motion parameters of all ships according to the positions of the same ships in the two adjacent images;

[0008] S2, calculating the comprehensive grade of each ship according to the spatial grade of the position of all ships, their own grade and the over-limit mark of the motion parameter, and determining the specific frame extraction interval of the next image according to the comprehensive grade;

[0009] S3, extract the next image according to the specific frame extraction interval, use the target detection model to detect all ships in the image and calculate their geographic coordinates;

[0010] S4, using a Kalman filter model to predict the position and position interval of each ship in the image extracted in step S3 according to the motion parameters of each ship;

[0011] S5, judging the number of ships in all the position intervals according to the ship coordinates determined in step S3, if there are multiple ships in a position interval, returning to step S4 to adjust the relevant parameters to adjust the range of the position interval, otherwise proceeding to step S6;

[0012] S6. For a position interval in which a ship exists, the ship is used as the ship in the previous image corresponding to the position interval; for a position interval in which a ship does not exist, the ships outside all position intervals are matched with the predicted positions of the ships corresponding to all position intervals in which no ship exists, and the ship with the best probability match in each position interval is used as the ship in the previous image corresponding to the position interval;

[0013] S7, after confirming that all position intervals have been matched, whether there are any unmatched ships left, if not, proceed to step S8, otherwise proceed to step S9;

[0014] S8, recalculating the motion parameters of all ships according to the positions of the ships identified in step S6 and the positions of the corresponding ships in the previous image, and returning to step S2 until the drone stops shooting;

[0015] S9. The next image is extracted according to the minimum frame extraction interval, and the nearest neighbor algorithm or the target detection method based on intersection and union is used to complete the matching of the image with the ships in the previous image, and the motion parameters of all ships are calculated according to the matching results, and the process returns to step S2 until the drone stops shooting.

[0016] Based on the above technical solution, firstly, two initial adjacent frames are extracted through the default frame extraction interval, and the positions of the same ships in the two frames are identified by the traditional recognition method, and the motion parameters of all ships are calculated accordingly; then, the comprehensive level of each ship is calculated according to the spatial level of the position of the ship, its own level and the over-limit mark of the motion parameters. The comprehensive level reflects the complexity of the space where the ship is located, its own attributes and motion. The higher the comprehensive level, the more complex the overall situation is, and the shorter the frame extraction interval is required. The frame extraction interval can be determined more reasonably according to the comprehensive level. It will not be too long to cause inaccurate recognition, nor too short to increase the amount of calculation and reduce the processing efficiency. Compared with the transmission algorithm, in order to achieve the highest recognition accuracy, the minimum frame extraction interval will be used. The frame extraction interval determination method provided by the present invention can avoid directly using the minimum frame extraction interval while ensuring the accuracy, but selects the most suitable frame extraction interval. In this way, compared with the traditional method, the total number of images extracted is small, and the amount of data for subsequent recognition processing is also small, thereby improving the recognition efficiency.

[0017] Then, based on the ship motion parameters determined by the initial two frames of images, the Kalman filter model is used to predict the possible positions and position intervals of the ships. By adjusting the position interval to an appropriate range, it is only necessary to see whether there is a ship in the corresponding position interval. If there is a ship, the ship is directly identified as the predicted ship corresponding to the position interval; if there is no ship in the position interval, all ships outside the position interval are matched with the predicted positions corresponding to the position intervals where no ships exist, and the ship with the best probability match is selected as the predicted ship. This recognition method can ensure the accuracy of recognition without the need to match all ships with all position intervals. As long as the range of the position interval is selected appropriately, most ships will fall within the position interval. Compared with the nearest neighbor algorithm or the target detection method based on intersection and union, which requires all ships in the two images to be matched pairwise, it greatly saves the calculation amount of recognition and matching, and further improves the recognition efficiency.

[0018] Preferably, the step S3 first compresses the extracted image, then determines the area in the image where the ship exists through initial target detection, and then further detects only the area where the ship is detected to determine the type of the ship and the position in the image; and calculates the geographic coordinates of the ship in combination with the altitude, longitude and latitude, direction angle and camera angle of the drone.

[0019] The extracted image is first compressed to improve the efficiency of subsequent processing. The area where the ship exists is determined through the initial target detection line, and only the area where the ship exists is further detected subsequently. The accuracy of the detection is ensured through multiple target detections. In addition to the initial detection of all areas of the image, only the area where the ship exists is further detected subsequently, which also improves the processing efficiency.

[0020] The surface vessel continuous identification system based on the perspective of an unmanned aerial vehicle of the present invention comprises:

[0021] Initial processing module: used to extract the initial two adjacent images in the video shot by the drone according to the default frame extraction interval, and calculate the motion parameters of all ships according to the positions of the same ships in the two adjacent images;

[0022] Frame extraction interval determination module: used to calculate the comprehensive level of each ship according to the spatial level of the location of all ships, their own level and the over-limit mark of the motion parameters, and determine the specific frame extraction interval of the next image according to the comprehensive level;

[0023] The ship recognition module is used to extract the next image according to a specific frame extraction interval, detect all ships in the image using the target detection model and calculate their geographic coordinates;

[0024] Prediction module: used to predict the position and position interval of each ship in the image extracted by the ship recognition module based on the motion parameters of each ship in the initial processing module using the Kalman filter model;

[0025] Correction module: used to determine the number of ships in all position intervals according to the ship coordinates determined by the ship identification module. If there are multiple ships in a position interval, it returns to the prediction module to adjust the relevant parameters to adjust the range of the position interval, otherwise it enters the matching module;

[0026] Matching module: for a position interval where a ship exists, taking the ship as the ship in the previous image corresponding to the position interval; for a position interval where a ship does not exist, performing overall probability matching between the ships outside all position intervals and the predicted positions of the ships corresponding to all position intervals inside where no ship exists, and taking the ship with the best probability matching in each position interval as the ship in the previous image corresponding to the position interval;

[0027] Verification module: used to confirm whether there are any unmatched ships after all position intervals have been matched. If not, enter the first loop module, otherwise enter the second loop module;

[0028] The first loop module is used to recalculate the motion parameters of all ships according to the positions of the ships identified by the matching module in two adjacent images, and return to the frame extraction interval determination module to re-run each module in sequence until the drone stops shooting;

[0029] The second loop module is used to extract the next image according to the minimum frame extraction interval, use the nearest neighbor algorithm or the target detection method based on intersection and union to complete the matching of the image with the ships in the previous image, and calculate the motion parameters of all ships according to the matching results, return to the frame extraction interval determination module and re-run each module in sequence until the drone stops shooting.

[0030] The computer-readable storage medium storing one or more programs described in the present invention includes one or more programs including instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.

[0031] Beneficial effects: Compared with the prior art, the present invention has the following significant effects: the comprehensive level is calculated by the spatial level of the ship's position, its own level and the over-limit mark of the motion parameters, and the frame extraction interval is determined accordingly, so that the most appropriate frame extraction interval can be selected, which not only ensures the recognition accuracy but also avoids increasing the amount of data to be processed by using the minimum frame extraction interval, thereby improving the processing efficiency; the Kalman filter model is used according to the motion parameters of the ship to predict the position and position interval of the ship, and the ship is quickly identified according to the position interval, which not only ensures the accuracy of recognition but also greatly reduces the amount of calculation and improves the recognition efficiency. Overall, the recognition method of the present invention can accurately and efficiently identify ships. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0033] As shown in the figure, a method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to the present invention comprises the following steps:

[0034] S1. Extract the initial two adjacent images in the drone video according to the default frame extraction interval, and calculate the motion parameters of all ships according to the positions of the same ships in the two adjacent images.

[0035] In the actual processing process, when the video stream transmitted by the drone is received, images are continuously extracted according to the default frame extraction interval to identify and confirm whether there is a ship in the image. If there is no ship, images in the video stream are continuously extracted according to the default frame extraction interval until a ship appears in the image. The image in which the ship appears for the first time is taken as the first image to be identified, and then another image is extracted according to the default frame extraction interval as the second image to be identified. The nearest neighbor algorithm or the intersection-and-union-based target detection algorithm can be used to identify the same ship in the image. These are mature existing technologies and will not be described in detail in this article.

[0036] According to the recognition results of the first and second images, the motion characteristic parameters such as the speed, acceleration, and azimuth of the ship are calculated by comparing the geographic coordinate positions of the same ship in the two images. In order to reduce the influence of noise and accurately calculate the motion characteristic parameters, the Savitzky-Golay filter can be applied to smooth the original position data. The original geographic coordinates y i Get the smoothed geographic coordinates For the smoothed position data Calculate the speed, acceleration, and azimuth; then extract images according to the newly calculated frame extraction interval instead of the default frame extraction interval.

[0037] S2. Calculate the comprehensive grade of each ship according to the spatial grade of the location of all ships, their own grade and the over-limit mark of the motion parameters, and determine the specific frame extraction interval of the next image according to the comprehensive grade.

[0038] The process of setting the spatial level is as follows: according to the spatial characteristics, the target waters photographed by the drone are divided into multiple rectangular areas. For example, the dock and open waters are regarded as different spatial units. It should be noted that in addition to being divided into rectangular areas, areas of other shapes can also be used. On this basis, each spatial unit is graded according to the historical density of ship activities and the frequency of risk occurrence. The higher the historical density of ship activities and the frequency of risk occurrence, the higher the level of the corresponding spatial unit. The historical density of ship activities and the frequency of risk occurrence can be obtained based on past statistical data. The corresponding data can be obtained by dividing the corresponding intervals according to the set standards to achieve complete classification.

[0039] The ship's own level is set directly according to the ship type, because different types of ships involve different functions, operational complexity and operational risks, which can reflect the uncertainty of the ship during movement. The ship type itself can reflect the functions, operational complexity and operational risks involved as a whole. The higher the level of the ship, the higher the uncertainty in the ship's movement.

[0040] The process of determining the over-limit mark of the ship motion parameters is as follows:

[0041] The motion parameters of the ship include speed, acceleration and azimuth. The motion parameter thresholds are set for different types of ships to obtain the three-dimensional matrix M parameter ;M parameter (t,m) represents the threshold value of motion parameter m of t types of ships.

[0042] Compare the current motion parameters of the ship with the threshold matrix M parameter The corresponding value in is used to calculate the over-limit flag, which is calculated according to the following formula:

[0043]

[0044] Among them, V flag (t), A flag (t) and θ flag (t) are the speed, acceleration and azimuth overlimit marks of the t-th ship respectively; V current (t), A current (t) and θ current(t) are the current speed, acceleration and azimuth of the t-th ship; M parameter (t,speed),M parameter (t, acceleration) and M parameter (t, heading) are the thresholds of the velocity, acceleration and heading of the t-th ship respectively.

[0045] The present invention sets that any motion parameter exceeding the threshold value indicates an over-limit, and the over-limit flag calculation formula is:

[0046] ExceedLimit flag (t) = V flag (t)∨A flag (t)∨θ flag (t)

[0047] Among them, ExceedLimit flag (t) is the limit mark of the motion parameter of the t-th ship.

[0048] The comprehensive grade is calculated by weighted average of the spatial grade of the vessel's location, the vessel's own grade, and the over-limit mark of the vessel's motion parameters. The calculation formula is:

[0049] L composite (t) = ω block ·L block (t)+ω type ·L type (t)+ω exceed ·ExceedLimit flag (t)

[0050] Among them, L composite (t) is the comprehensive level of the t-th ship, Lblock(t) is the spatial level of the location of the t-th ship, L type (t) is the level of the t-th ship; ω block ,ω type and ω exceed They are the weights of space grade, own grade and over-limit mark, indicating the importance of each factor in the comprehensive score. The weight value can be set based on experience.

[0051] According to the comprehensive level L of all ships in the second image to be identified composite The highest comprehensive grade L high , use linear interpolation to calculate the specific frame extraction interval, the calculation formula is:

[0052]

[0053] Among them, highis the specific frame extraction interval, L max The maximum value set for the comprehensive level, L min The minimum value set for the comprehensive level, Τ max and Τ min are the maximum and minimum frame extraction intervals respectively; L high It is the maximum value of the comprehensive grades of all ships in the second image to be identified.

[0054] Since the frame decimation interval calculated by the linear interpolation method needs to be calculated when a moving ship is detected, when no ship can be identified in the video stream, a default frame decimation interval of Τ is set. base , Τ base Between max and Τ min The higher the recognition accuracy requirement, the closer it is to Τ min .

[0055] S3. Extract the next image according to a specific frame extraction interval, use the target detection model to detect all ships in the image and calculate their geographic coordinates.

[0056] The specific steps include:

[0057] S3.1 Image Preprocessing

[0058] The third image to be identified is extracted at a specific frame interval and processed using a compression algorithm, such as a JPEG compression algorithm, to ensure that processing time and storage requirements are reduced without significant loss of information.

[0059] S3.2 Preliminary target detection

[0060] Using a target detection model, such as a trained neural network model, the first round of target detection is performed on the compressed third image to be identified to obtain the category label, target coordinate information, and the rectangular space unit where the ship is located.

[0061] S3.3. Fine recognition processing

[0062] The target detection model is used to perform subsequent target detection only on the rectangular space unit where the ship is identified, and the category label and target coordinate information identified with high precision are used to determine the category of the ship and its position in the image, that is, the image coordinates.

[0063] S3.5. Calculate the geographic coordinates of the vessel

[0064] The standard geographic coordinates of the ship are calculated through the image coordinates of the ship in the drone screen, as well as the drone's own altitude, longitude and latitude, azimuth angle, and camera angle. By comparing the geographic coordinates of the ship, the drone can uniquely identify the ship even if there are differences in the coordinates of the ship images taken during the cruising process or from multiple drone perspectives.

[0065] S4. Using a Kalman filter model according to the motion parameters of each ship, predict the position and position interval of each ship in the image extracted in step S3.

[0066] According to the characteristic that surface objects are often in motion, the present invention uses the Kalman filter model in the nonlinear model to predict the position and position interval of the ship. The use of the Kalman filter model to predict the target position is a mature prior art, and its principle is not described in detail herein. Finally, the formula for calculating the position interval according to the predicted position of the Kalman filter model is as follows:

[0067]

[0068] in, and They represent the maximum and minimum values ​​of the horizontal coordinate of the predicted position interval of the t-th ship, and They represent the maximum and minimum values ​​of the ordinate of the predicted position interval of the t-th ship respectively; and are the horizontal and vertical coordinates of the t-th ship predicted by the Kalman filter; k is the confidence level constant, σ x (t) and σ y (t) are the preset standard deviation constants of the predicted position of the t-th ship in the x and y directions, σ x (t) and σ y (t) can be set based on empirical values ​​or can be adjusted to adjust the range of the position interval.

[0069] S5. Determine the number of ships in all position intervals according to the ship coordinates determined in step S3. If there are multiple ships in a position interval, return to step S4 to adjust the relevant parameters to adjust the range of the position interval, that is, adjust σ x (t) and σ y (t); otherwise go to step S6.

[0070] S6. For a position interval where a ship exists, the ship is taken as the ship in the previous image corresponding to the position interval; for a position interval where a ship does not exist, the ships outside all position intervals are matched with the predicted positions of the ships corresponding to all position intervals inside where no ships exist, and the ship with the best probability match in each position interval is taken as the ship in the previous image corresponding to the position interval.

[0071] The overall probability matching is the target probability matching based on the Euclidean distance, which specifically includes the following processes:

[0072] Calculating distance

[0073] Calculate the Euclidean distance d between the position of the i-th ship outside all position intervals and the predicted position of the ship corresponding to the j-th position interval where no ship exists ij :

[0074]

[0075] Among them, x i and i are the horizontal and vertical coordinates of the i-th ship outside all position intervals respectively; and are the horizontal and vertical coordinates of the predicted position of the ship corresponding to the j-th position interval respectively.

[0076] Calculate the matching probability

[0077] According to the Euclidean distance, the matching probability P is calculated by the following formula ij :

[0078]

[0079] Among them, ij The probability value of the i-th ship outside all position intervals matching the j-th position interval; Φ is the cumulative distribution function of the standard normal distribution; σ d is a preset standard deviation parameter used to control how much distance affects the probability.

[0080] Constructing the probability matrix

[0081] The matching probability P calculated above ij Construct the probability matrix P.

[0082] Select the best match

[0083] The Hungarian algorithm is used to select the best matching ship for each position interval in which no ship exists from the probability matrix, and this ship is used as the corresponding ship of the position interval in the previous image.

[0084] S7: After confirming that all position intervals have been matched, check whether there are any unmatched ships left. If not, proceed to step S8; otherwise, proceed to step S9.

[0085] After all the position intervals are actually matched, not all the position intervals can be matched to ships. There may also be a situation where the number of ships in the third image to be identified is less than that in the second image to be identified. However, this does not affect the subsequent recognition process, and step S8 can still be entered. Only when the number of ships in the third image to be identified is more than that in the second image to be identified, there will be a situation where some ships fail to match the position intervals, and then step S9 is entered. It should be noted that the second and third images to be identified here are only used as examples. As the recognition process is cycled, the corresponding image numbers should also increase accordingly, that is, the relationship between the image currently being identified and the previous identified image.

[0086] S8. Recalculate the motion parameters of all ships according to the positions of the ships identified in step S6 in two adjacent images, and return to step S2 until the drone stops shooting.

[0087] In this step, whether the number of ships in the current image is less than that in the previous image or not, it does not affect the calculation of the comprehensive level and the specific frame extraction interval, and the ship identification can be continued in a loop according to the above steps; in extremely special cases, due to sudden reasons, all ships drastically change their original motion state, or other reasons cause all ships to disappear collectively in the next image, then according to the method of step S1, the ships appearing in two adjacent frames are re-detected, their motion parameters are calculated, and then step S2 is entered to continue the loop identification, but this situation rarely occurs and the probability is too low, so it is not considered; the normal cruising speed of the drone used for shooting and identification is also relatively stable, because the probability of collective loss of ships due to a sudden change in the navigation state of the drone is also extremely low, and it is also not considered.

[0088] S9, extract the next image according to the minimum frame extraction interval, use the nearest neighbor algorithm or the target detection method based on intersection and union to complete the matching of the image with the ships in the previous image, and calculate the motion parameters of all ships according to the matching results, and return to step S2 until the drone stops shooting.

[0089] In this step, because a ship is added and the added ship does not appear in the previous image, it is impossible to calculate the motion parameters based on the different geographic coordinates of the same ship in the two images, and it is also impossible to calculate the comprehensive level and the specific frame extraction interval. Therefore, only the traditional target recognition method can be used to identify the ship. In order to ensure the accuracy of recognition, the minimum frame extraction interval is selected for image extraction. After all ships are identified and calculated using the traditional algorithm, including the type, position and motion parameters of the newly added ships, the ship identification can be continued in a loop according to steps S2-S7.

[0090] The surface vessel continuous identification system based on the perspective of an unmanned aerial vehicle of the present invention comprises:

[0091] Initial processing module: used to extract the initial two adjacent images in the video shot by the drone according to the default frame extraction interval, and calculate the motion parameters of all ships according to the positions of the same ships in the two adjacent images;

[0092] Frame extraction interval determination module: used to calculate the comprehensive level of each ship according to the spatial level of the location of all ships, their own level and the over-limit mark of the motion parameters, and determine the frame extraction interval of the next image according to the comprehensive level;

[0093] Vessel identification module: used to extract the next image, detect all the vessels in the image using the target detection model and calculate their geographic coordinates;

[0094] Prediction module: used to predict the position and position interval of each ship in the image extracted by the ship recognition module based on the motion parameters of each ship in the initial processing module using the Kalman filter model;

[0095] Correction module: used to determine the number of ships in all position intervals according to the ship coordinates determined by the ship identification module. If there are multiple ships in a position interval, it returns to the prediction module to adjust the relevant parameters to adjust the range of the position interval, otherwise it enters the matching module;

[0096] Matching module: used to take the ships existing in the position interval as the corresponding ships in the previous image; for the position intervals without ships, the ships outside all the position intervals are matched with all the position intervals without ships in the internal position intervals for overall probability, and the ship with the best probability matching in the position interval is taken as the corresponding ship in the previous image;

[0097] Verification module: used to confirm whether there are any unmatched ships after all position intervals have been matched. If not, enter the first loop module, otherwise enter the second loop module;

[0098] The first loop module is used to recalculate the motion parameters of all ships according to the positions of the ships identified by the matching module in two adjacent images, and return to the frame extraction interval determination module to re-run each module in sequence until the drone stops shooting;

[0099] The second loop module is used to extract the next image according to the minimum frame extraction interval, use the nearest neighbor algorithm or the target detection method based on intersection and union to complete the matching of the image with the ships in the previous image, and calculate the motion parameters of all ships according to the matching results, return to the frame extraction interval determination module and re-run each module in sequence until the drone stops shooting.

[0100] The computer-readable storage medium storing one or more programs described in the present invention includes one or more programs including instructions, and when the instructions are executed by a computing device, the computing device executes any one of the above methods.

Claims

1. A method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle, characterized in that: The following steps are involved: S1, extracting two initial adjacent images in the video taken by the drone according to the default frame extraction interval, and calculating the motion parameters of all ships according to the positions of the same ships in the two adjacent images; S2, calculating the comprehensive grade of each ship according to the spatial grade of the position of all ships, their own grade and the over-limit mark of the motion parameter, and determining the specific frame extraction interval of the next image according to the comprehensive grade; S3, extract the next image according to the specific frame extraction interval, use the target detection model to detect all ships in the image and calculate their geographic coordinates; S4, using a Kalman filter model to predict the position and position interval of each ship in the image extracted in step S3 according to the motion parameters of each ship; S5, judging the number of ships in all the position intervals according to the ship coordinates determined in step S3, if there are multiple ships in a position interval, returning to step S4 to adjust the relevant parameters to adjust the range of the position interval, otherwise proceeding to step S6; S6. For a position interval in which a ship exists, the ship is used as the corresponding ship of the position interval in the previous image; for a position interval in which a ship does not exist, the ships outside all position intervals are matched with all position intervals in which no ship exists by overall probability, and the ship with the best probability match of the position interval is used as the corresponding ship in the previous image; S7, after confirming that all position intervals have been matched, whether there are any unmatched ships left, if not, proceed to step S8, otherwise proceed to step S9; S8, recalculating the motion parameters of all ships according to the positions of the ships identified in step S6 in two adjacent images, and returning to step S2 until the drone stops shooting; S9, extract the next image according to the minimum frame extraction interval, use the nearest neighbor algorithm or the target detection method based on intersection and union to complete the matching of the image with the ships in the previous image, and calculate the motion parameters of all ships according to the matching results, and return to step S2 until the drone stops shooting.

2. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: In step S1, the same vessel in the two images is matched by a nearest neighbor algorithm or an intersection-over-union based target detection method.

3. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: In step S2, the shooting area is divided into multiple spaces, and the spatial level is determined according to the historical density of ship activities and the frequency of risk occurrence; the ship's own level and motion parameter threshold are determined according to the ship type, and the over-limit mark is determined according to whether the ship's motion parameters exceed the threshold.

4. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: In step S2, the comprehensive grade is calculated according to the following formula L composite (t)=ω block ·L block (t)+ω type ·L type (t)+ω exceed ·ExceedLimit flag (t) Among them, L composite (t) is the comprehensive level of the t-th ship; Lblock(t) is the spatial level of the location of the t-th ship, L type (t) is the level of the t-th ship, ExceedLimit flag (t) is the limit mark of the motion parameter of the t-th ship; ω block ,ω type and ω exceed They are the weights of space grade, own grade and over-limit mark respectively; ExceedLimit flag (t) is determined according to the following formula ExceedLimit flag (t)=V flag (t)∨A flag (t)∨θ flag (t) Among them, V flag (t), A flag (t) and θ flag (t) are the speed, acceleration and azimuth overlimit marks of the t-th ship, respectively, and are determined according to the following formulas: Among them, V current (t), A current (t) and θ current (t) are the current speed, acceleration and azimuth of the t-th ship; M parameter (t,speed),M parameter (t, acceleration) and M parameter (t, heading) are the thresholds of the velocity, acceleration and heading of the t-th ship respectively.

5. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The step S2 calculates the specific frame extraction interval according to the following formula Among them, high is a specific frame extraction interval, T max and Τ min are the maximum and minimum frame extraction intervals, L max and L min are the maximum and minimum values ​​set for the comprehensive level, L high is the maximum value of the comprehensive level of all ships in the extracted image.

6. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: In step S3, the extracted image is first compressed, and then the area where the ship exists in the image is determined through initial target detection, and then only the area where the ship is detected is further detected to determine the type of the ship and the position in the image; the geographic coordinates of the ship are calculated by combining the altitude, longitude and latitude, direction angle and camera angle of the drone.

7. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: The step S4 predicts the coordinates of the position interval by the following formula: in, and They represent the maximum and minimum values ​​of the horizontal coordinate of the predicted position interval of the t-th ship, and They represent the maximum and minimum values ​​of the ordinate of the predicted position interval of the t-th ship respectively; and are the horizontal and vertical coordinates of the t-th ship predicted by the Kalman filter; k is the confidence level constant, σ x (t) and σ y (t) are the preset standard deviation constants of the predicted position of the t-th ship in the x and y directions respectively.

8. The method for continuous identification of surface vessels based on the perspective of an unmanned aerial vehicle according to claim 1, characterized in that: In step S6, the overall probability matching determines the vessel with the best probability matching in each position interval where no vessel exists from the probability matrix by the Hungarian algorithm. The elements in the probability matrix are calculated by the following formula: Among them, ij represents the matching probability value of the i-th ship outside all position intervals and the j-th position interval; d ij represents the Euclidean distance between the position of the i-th ship that is not identified outside all position intervals and the predicted position of the ship corresponding to the j-th position interval; Φ is the cumulative distribution function of the standard normal distribution; σ d is a preset standard deviation parameter used to control how much distance affects the probability.

9. A continuous identification system for surface vessels based on the perspective of an unmanned aerial vehicle, characterized in that: The system comprises: Initial processing module: used to extract the initial two adjacent images in the video shot by the drone according to the default frame extraction interval, and calculate the motion parameters of all ships according to the positions of the same ships in the two adjacent images; Frame extraction interval determination module: used to calculate the comprehensive level of each ship according to the spatial level of the location of all ships, their own level and the over-limit mark of the motion parameters, and determine the specific frame extraction interval of the next image according to the comprehensive level; The ship recognition module is used to extract the next image according to a specific frame extraction interval, detect all ships in the image using the target detection model and calculate their geographic coordinates; Prediction module: used to predict the position and position interval of each ship in the image extracted by the ship recognition module based on the motion parameters of each ship in the initial processing module using the Kalman filter model; Correction module: used to determine the number of ships in all position intervals according to the ship coordinates determined by the ship identification module. If there are multiple ships in a position interval, it returns to the prediction module to adjust the relevant parameters to adjust the range of the position interval, otherwise it enters the matching module; Matching module: used to take the ships existing in the position interval as the corresponding ships in the previous image; for the position intervals without ships, the ships outside all the position intervals are matched with all the position intervals without ships in the internal position intervals by overall probability, and the ship with the best probability matching in the position interval is taken as the corresponding ship in the previous image; Verification module: used to confirm whether there are any unmatched ships after all position intervals have been matched. If not, enter the first loop module, otherwise enter the second loop module; The first loop module is used to recalculate the motion parameters of all ships according to the positions of the ships identified by the matching module in two adjacent images, and return to the frame extraction interval determination module to re-run each module in sequence until the drone stops shooting; The second loop module is used to extract the next image according to the minimum frame extraction interval, use the nearest neighbor algorithm or the target detection method based on intersection and union to complete the matching of the image with the ships in the previous image, and calculate the motion parameters of all ships according to the matching results, return to the frame extraction interval determination module and re-run each module in sequence until the drone stops shooting.

10. A computer-readable storage medium storing one or more programs, characterized in that : comprising one or more programs including instructions, which, when executed by a computing device, cause the computing device to perform any of the methods according to claims 1 to 8.

Images