A ship gear recognition method, device and medium based on target tracking and re-identification

Abstract

The present invention relates to a ship gear recognition method, device and medium based on target tracking and re-identification, the method comprising the following steps: obtaining ship information; performing target recognition based on YOLO5s that introduces an extended Kalman filter algorithm, tracking the ship's trajectory from the time the ship enters the lock, and superimposing a ship detection frame and basic ship information of the trajectory tracking result on the ship's real-time video; if occlusion occurs during the ship trajectory tracking process, resulting in the loss of the ship's trajectory, determining the lost target trajectory based on the Deep OC-SORT algorithm and Redis re-identification technology; during the ship trajectory tracking process, if it is identified that the ship has not moved for a preset time, determining that the ship's gear docking is complete, obtaining the ship's docking position and forming the ship's actual gear map; judging whether the ship is in gear according to the ship's scheduling map, the actual gear map and the ship's basic information, and issuing an alarm if the ship is not in gear according to the scheduling. Compared with the prior art, the present invention has the advantages of being able to accurately locate the ship's position.

Description

Technical Field

[0001] The present invention relates to the technical field of ship management, and in particular to a ship gear identification method, device and medium based on target tracking and re-identification. Background Art

[0002] Ship gearing refers to the orderly docking of ships in the lock chamber when passing through the lock. Generally, the lock will formulate a corresponding ship scheduling and gearing plan based on the ship's lock reporting situation. By comparing the scheduling and gearing plan with the actual gearing situation, the lock staff can know whether the ship has been geared according to the schedule. For ships that are not geared according to the gearing diagram, the staff can promptly notify the ship to adjust. At present, most patents related to ship gearing mainly focus on optimizing ship scheduling and gearing methods. For example, patent CN110110403B proposes a scheduling and gearing method suitable for one-way continuous passing ships, but does not propose a method for identifying the actual gearing of the ship. If the actual gearing of the ship is inconsistent with the scheduled gearing, manual comparison and judgment are required, which is time-consuming and labor-intensive. Summary of the Invention

[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a ship gear identification method, equipment and medium based on target tracking and re-identification, aiming to track the ship passing through the lock and identify the actual gear position of the ship, so as to realize rapid detection and early warning of whether the ship is in the scheduled gear.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] According to a first aspect of the present invention, a method for ship gear identification based on target tracking and re-identification is provided, the method comprising the following steps:

[0006] Obtaining ship information, including basic ship information, ship dispatch diagrams, ship pictures, and real-time ship videos;

[0007] The system uses the YOLO5s algorithm with an extended Kalman filter to identify targets. It tracks the ship's trajectory from the moment it enters the lock, matches basic ship information with images, and overlays the ship's detection frame and basic information from the trajectory tracking results on the real-time video.

[0008] If occlusion occurs during ship tracking, causing the ship's trajectory to be lost, the Deep OC-SORT algorithm predicts the position of the target lost in the real-time video in the next frame or frames, and searches for the lost target near the predicted position. After one or more targets are found near the predicted position, the Redis re-identification technology is used to calculate the Intersection over Union (IoU) with the content in the ship detection frame obtained by trajectory tracking. When the IoU value is greater than a preset threshold, the found target and the target in the corresponding ship detection frame are considered to be the same target.

[0009] During the ship trajectory tracking process, if it is identified that the ship has not moved for a preset time, the ship gear docking is determined to be completed, the ship's docking position is obtained, and the actual ship gear map is generated;

[0010] The system determines whether the ship is scheduled according to the ship scheduling diagram, actual gear diagram and basic ship information. If the ship is not scheduled according to the scheduling, an alarm will be issued.

[0011] As a preferred technical solution, the target recognition based on YOLO5s introduced with the extended Kalman filter algorithm specifically includes the following steps:

[0012] Initialize the state of the extended Kalman filter tracker: use the frame where YOL0v5s target detection fails as the third frame of the extended Kalman filter tracker, use the detection values ​​of the first two frames as the initial state of the tracker, and initialize the state vector and error covariance matrix of the second frame;

[0013] State prediction: predict the current state vector and state error covariance matrix based on the previous state;

[0014] Calculate the extended Kalman gain: define the observation vector, calculate the extended Kalman gain through the daily matrix and the prior results obtained in the prediction stage;

[0015] Update: Update the state vector and error covariance matrix according to the extended Kalman gain to achieve continuous tracking of the target;

[0016] Candidate region creation and matching: Based on the prior system state bounding box position of the extended Kalman filter, the region is expanded to obtain the candidate region. The difference value hash algorithm is used to match the candidate regions, and the candidate region with the smallest difference is selected as the observation value.

[0017] As a preferred technical solution, the state vector x and error covariance matrix P of the second frame are as follows:

[0018]

[0019] in and Represents the coordinates of the upper left corner and lower right corner of the bounding box on the x-axis and y-axis respectively. The subscript m represents the observation value, and the subscript 2 of m represents the second frame. and To calculate the result, ΔT is the time taken from the first frame to the second frame, cent(x) and cent(y) represent the centroid coordinates of the bounding box on the x-axis and y-axis, respectively.

[0020] As a preferred technical solution, the extended Kalman filter updates the state vector and error covariance matrix according to the extended Kalman gain:

[0021]

[0022] Among them, K k is the extended Kalman gain, is the prior state vector, is the prior error covariance matrix, H is the Jacobian matrix, Z k is the observation vector, Represents the nonlinear transformation from state vector to measurement vector:

[0023]

[0024] As a preferred technical solution, the candidate area is created by moving the position of the prior system state bounding box of the extended Kalman filter in eight directions, wherein the directions are 45° to the upper left, 45° to the upper right, 45° to the lower right, 45° to the lower left, and directly above, directly below, directly to the right, and directly to the left. The degree of movement is represented by the intersection over union (IoU), the bounding box of the prior system state is represented by E, and the bounding box after movement is represented by F. The intersection over union (IoU) is defined as:

[0025]

[0026] The degree of movement is determined by presetting the intersection-over-union ratio value, and the expansion areas in eight directions are obtained. Combined with the prior system state bounding box position, nine candidate areas are obtained.

[0027] As a preferred technical solution, the use of the difference value hash algorithm to match candidate regions is specifically as follows:

[0028] Reduce the expanded candidate area image to n×(n-1), where n is a preset value;

[0029] Convert the reduced image into a grayscale image, assign 1 to each point in each row where the previous pixel value is greater than the next pixel value, and 0 if the previous pixel value is greater than the next pixel value, and generate a hash value;

[0030] Calculate the Hamming distance between the candidate region and the image within the target bounding box of the previous frame. The smaller the distance, the smaller the image difference. Select the candidate region with the smallest difference as the observation value.

[0031] As a preferred technical solution, the Deep OC-SORT algorithm performs the following steps:

[0032] Get the scaling rotation matrix Mt=s t R t and the translation matrix T t ,in s t represents the scaling matrix, R t represents the rotation matrix;

[0033] Introducing the scaling rotation matrix and translation matrix into the three components of the OC-SORT algorithm: OOS, OCM and OCR:

[0034] For the OOS component, the Kalman filter is updated along a linear interpolation path starting from the last known measurement, which is given by [x c ,y c , a, s], the first two items are the center of the bounding box, the center of the bounding box is c<-M t c+T t Transformation is obtained, where a is the area and s is the aspect ratio;

[0035] For the OCM component, let p1 and p2 be the upper left and lower right corners of the bounding box. OCM uses the bounding box at the last Δt=3 to calculate the bounding box angular velocity. At each time step t, the transformation p is applied to the bounding box. i ←M t p i +T t ;

[0036] For the OCR component, for the last seen bounding box position in OCR, at each time step t, apply p i ←M t p i +T t to adjust its position;

[0037] Under the OC-SORT algorithm, the Kalman state is The Kalman state is adjusted as follows by camera motion compensation:

[0038]

[0039] As a preferred technical solution, low detector confidence is used to identify image degradation caused by occlusion or blur. Let e tis the tracklet appearance embedding at time t, the standard EMA is:

[0040] e t =α t e t-1 +(1-α t )e new

[0041] e new is the appearance of the matching detection added to the model, α t Defined as:

[0042]

[0043] Among them, s det is the detector confidence, σ is the detection confidence threshold used to filter noisy detections, and α f is a fixed value;

[0044] Adaptive weighting: Use the standard cosine similarity between track and box embeddings to get an M×N appearance cost matrix A c , where M and N are the number of trajectories and detections, respectively, and A c [m, n] represents the item at the intersection of the mth row and nth column, which is combined with the IoU cost matrix to obtain C=I c +a w A c , and the weighting factor w b (m, n) is added to the global a w In the example, let T m Indicates tracking, d n Indicates detection, when T m When only one box has a high similarity score, in A c [m,:] Increase the appearance weight if the detection d n has only one distinct association with a trajectory, then the same operation is applied to A c Columns, using z diff To measure the discriminability of the detection box-track pair, z diff Defined as the difference between the highest and second highest values ​​in a row or column:

[0045]

[0046] Among them, ∈ is a hyperparameter;

[0047] According to z diff The weighting factors are derived, which produce the final cost matrix C:

[0048]

[0049] C[m,n]=IoU[m,n]+[a w +w b (m, n)]A c [m,n].

[0050] The missing target search is achieved by minimizing the cost matrix C.

[0051] According to a second aspect of the present invention, an electronic device is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the method when executing the program.

[0052] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method described above is implemented.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] (1) The present invention provides a ship gear identification method based on target tracking and re-identification technology. The method tracks the ship's trajectory from the moment it enters the lock until the ship docks and locks the ship's position, thereby forming the ship's actual gear map. By comparing the actual gear map with the scheduled gear map, it is confirmed whether the ship is in the scheduled gear. For ships with mismatched gear coordinates, an alarm is automatically issued, saving manpower and time for manual gear identification and improving efficiency.

[0055] (2) During the ship tracking process, if a ship is obscured by a camera behind it and cannot be tracked continuously, the Deep OC-SORT algorithm of the present invention predicts the possible position of the obscured ship for several frames and uses target re-identification technology to retrieve the lost ship. Redis re-identification quickly and accurately compares the lost target with the re-identified target, assisting in the one-to-one recovery of the lost ship.

[0056] (3) The present invention adds an alarm mechanism, which issues an alarm for ships that fail to follow the scheduling schedule, reminding ship business personnel to deal with it in a timely manner, realizing the linkage of the entire process of ship passing through the lock, and thoroughly improving the scheduling efficiency and ship passing efficiency of the lock. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of the method of the present invention;

[0058] Figure 2 Flowchart of the ship trajectory tracking algorithm of the present invention;

[0059] Figure 3 Schematic diagram of the system structure of the present invention;

[0060] Figure 4 is the actual trajectory of the target in one embodiment;

[0061] Figure 5 is a change in the distance between the target and the observation origin in one embodiment;

[0062] Figure 6 The change in the orientation of the target and the observation origin in one embodiment;

[0063] Figure 7 A comparison between the actual and estimated x-coordinate values ​​of a target in one embodiment;

[0064] Figure 8 Comparison between the actual and estimated values ​​of the target y-coordinate in one embodiment;

[0065] Figure 9 A comparison between the actual value and the estimated value of the target x-direction velocity in one embodiment;

[0066] Figure 10 A comparison between the actual value and the estimated value of the target y-direction velocity in one embodiment;

[0067] Figure 11 Comparison between the actual trajectory and the estimated trajectory of the target in one embodiment;

[0068] Figure 12 is an extended Kalman filter position estimation bias in one embodiment;

[0069] Figure 13 An extended Kalman filter velocity estimation bias in one embodiment;

[0070] Figure 14 A comparison between the actual trajectory and the estimated trajectory after changing the initial state estimate in one embodiment;

[0071] Figure 15 The position and velocity deviation of the target tracking after changing the initial estimated state in one embodiment;

[0072] Figure 16 The actual trajectory and estimated trajectory of the target tracking after adding the time point in one embodiment;

[0073] Figure 17 In one embodiment, the position and velocity deviation of the target tracking after adding the time point are shown;

[0074] Figure 18 is the target true trajectory after increasing the process noise in one embodiment;

[0075] Figure 19 The position and velocity deviation of target tracking after increasing process noise in one embodiment;

[0076] Figure 20A comparison of target estimation and actual range and azimuth changes after increasing measurement noise in one embodiment;

[0077] Figure 21 Comparison between the actual and estimated target trajectories after increasing the measurement noise in one embodiment. DETAILED DESCRIPTION

[0078] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0079] Example 1

[0080] This embodiment provides a ship gear recognition method based on target tracking and re-identification, such as Figure 1 As shown, the method includes the following steps:

[0081] S1, obtaining ship information, which includes basic ship information, ship dispatch map, ship pictures and real-time ship video.

[0082] S2 performs target recognition based on YOLO5s with an extended Kalman filter algorithm. It tracks the ship's trajectory from the moment it enters the lock, uses AR video technology to match the ship's basic information with the ship's image, and overlays the ship detection frame and basic ship information from the trajectory tracking results on the ship's real-time video.

[0083] like Figure 2 As shown, the YOLO5s algorithm that introduces the extended Kalman filter algorithm performs the following steps:

[0084] S21, initialize the state of the extended Kalman filter tracker: use the frame where YOL0v5s target detection fails as the third frame of the extended Kalman filter tracker, use the detection values ​​of the first two frames as the initial state of the tracker, and initialize the state vector x and error covariance matrix P of the second frame:

[0085]

[0086] in and Represents the coordinates of the upper left corner and lower right corner of the bounding box on the x-axis and y-axis respectively. The subscript m represents the observation value, and the subscript 2 of m represents the second frame. and To calculate the result, ΔT is the time taken from the first frame to the second frame, cent(x) and cent(y) represent the centroid coordinates of the bounding box on the x-axis and y-axis, respectively.

[0087]

[0088] S22, state prediction: predict the current state vector and state error covariance matrix based on the previous state.

[0089]

[0090] in is the prior state vector, is the prior error covariance matrix.

[0091] S23, calculating the extended Kalman gain, which can determine how much the measurement quantity affects the system state prediction.

[0092] Define the observation vector:

[0093]

[0094] in and The calculation formula is as follows:

[0095]

[0096] For the extended Kalman filter, the relationship between the state vector and the measurement vector is nonlinear. Due to this nonlinear relationship, the H matrix is ​​different from the traditional Kalman filter. The H matrix is ​​a Jacobian matrix.

[0097]

[0098] Calculate the extended Kalman gain K using the daily matrix and the prior results obtained in the prediction phase k :

[0099]

[0100] S24, Update: Update the state vector and error covariance matrix according to the extended Kalman gain to achieve continuous tracking of the target:

[0101]

[0102] in Represents the nonlinear transformation from the state vector to the measurement vector.

[0103]

[0104] Appropriate Q and R values ​​are crucial for improving the performance of the Extended Kalman Filter. In Japanese target tracking, Q and R rely on empirical tuning. Therefore, we conducted three sets of experiments to optimize the performance of the Extended Kalman Filter tracker. In each set of experiments, Q was held constant while R was varied to 1, 0.1, or 0.01 across the diagonal elements. The final Q and R values ​​selected are as follows:

[0105]

[0106] S25, candidate region creation and matching: Based on the prior system state bounding box position of the extended Kalman filter, the region is expanded to obtain the candidate region, and the difference value hash algorithm is used to match the candidate region, and the candidate region with the smallest difference is selected as the observation value.

[0107] According to the position of the prior system state bounding box of the extended Kalman filter, it is moved in eight directions, namely 45° to the upper left, 45° to the upper right, 45° to the lower right, 45° to the lower left, and directly above, directly below, directly to the right, and directly to the left. The degree of movement is represented by the intersection over union (IoU). The bounding box of the prior system state is represented by E, and the bounding box after movement is represented by F. The intersection over union (IoU) is defined as:

[0108]

[0109] The degree of movement is determined by presetting the intersection-over-union ratio to 0.8, and the expansion areas in eight directions are obtained. Combined with the prior system state bounding box position, nine candidate areas are obtained.

[0110] Reduce the expanded candidate area image to n×(n-1), where n is a preset value;

[0111] Convert the reduced image into a grayscale image, assign 1 to each point in each row where the previous pixel value is greater than the next pixel value, and 0 if the previous pixel value is greater than the next pixel value, and generate a hash value;

[0112] Calculate the Hamming distance between the candidate region and the image within the target bounding box of the previous frame. The smaller the distance, the smaller the image difference. Select the candidate region with the smallest difference as the observation value.

[0113] In S3, if occlusion occurs during the ship trajectory tracking process, causing the ship trajectory to be lost, the Deep OC-SORT algorithm is used to predict the position of the lost target in the real-time video in the next frame or the next few frames, and the lost target is searched near the predicted position.

[0114] The Deep OC-SORT algorithm performs the following steps:

[0115] Get the scaling rotation matrix M t =s t Rt and the translation matrix T t ,in s t represents the scaling matrix, R t represents the rotation matrix;

[0116] Since the lost target prediction is prone to quadratic mixing errors over time, the scaling rotation matrix and translation matrix are introduced into the three components of the OC-SORT algorithm: OOS (online smoothing), OCM (momentum), and OCR (recovery):

[0117] For the OOS component, the Kalman filter is updated along a linear interpolation path starting from the last known measurement, which is given by [x c ,y c , a, s], the first two items are the center of the bounding box, and the center of the bounding box is c<-M t c+T t Transformation is obtained, where a is the area and s is the aspect ratio;

[0118] For the OCM component, let p1 and p2 be the upper left and lower right corners of the bounding box. OCM uses the bounding box at the last Δt=3 to calculate the bounding box angular velocity. At each time step t, the transformation p is applied to the bounding box. i ←M t p i +T t ;

[0119] For the OCR component, for the last seen bounding box position in OCR, at each time step t, apply p i ←M t p i +T t to adjust its position;

[0120] Under the OC-SORT algorithm, the Kalman state is The Kalman state is adjusted as follows by camera motion compensation:

[0121]

[0122] Using low detector confidence to identify image degradation due to occlusion or blur, let e t is the tracklet appearance embedding at time t, the standard EMA is:

[0123] e t =α t e t-1 +(1-α t )e new

[0124] enew is the appearance of the matching detection added to the model, α t Defined as:

[0125]

[0126] Among them, s det is the detector confidence, σ is the detection confidence threshold used to filter noisy detections, and α f It is a fixed value, which is set to 0.95 in this embodiment.

[0127] Adaptive weighting: Use the standard cosine similarity between track and box embeddings to get an M×N appearance cost matrix A c , where M and N are the number of trajectories and detections, respectively, and A c [m, n] represents the item at the intersection of the mth row and nth column, which is combined with the IoU cost matrix to obtain C=I c +a w A c , and the weighting factor w b (m, n) is added to the global a w In the example, let T m Indicates tracking, d n Indicates detection, when T m When only one box has a high similarity score, in A c [m,:] Increase the appearance weight if the detection d n has only one distinct association with a trajectory, then the same operation is applied to A c Columns, using z diff To measure the discriminability of the detection box-track pair, z diff Defined as the difference between the highest and second highest values ​​in a row or column:

[0128]

[0129] Among them, ∈ is a hyperparameter;

[0130] According to z diff The weighting factors are derived, which produce the final cost matrix C:

[0131]

[0132] C[m,n]=IoU[m,n]+[a w +w b (m, n)]A c [m,n].

[0133] The missing target search is achieved by minimizing the cost matrix C.

[0134] S4, after one or more targets are found near the predicted position, IoU calculation is performed on the contents in the ship detection frame obtained by Redis re-identification technology and trajectory tracking. When the IoU value is ≥ 0.8, the found target and the target in the corresponding ship detection frame are considered to be the same target.

[0135] S5, during the ship trajectory tracking process, if it is identified that the ship has not moved for 10 frames, it is determined that the ship gear shifting and docking is completed, the ship's docking position is obtained and the actual ship gear shifting map is formed.

[0136] S6, judging whether the ship is shifted according to the ship dispatch map, the actual shift map and the basic information of the ship, and if the ship is not shifted according to the dispatch, an alarm is issued.

[0137] Example 2

[0138] The above is an introduction to the method embodiment. The following further illustrates the solution of the present invention through a system embodiment.

[0139] like Figure 3 As shown, the system includes:

[0140] Data storage module: connected to the algorithm processing module, ship positioning module and lock control system, used to store relevant data and send data information to relevant modules;

[0141] Algorithm processing module: used to perform the following steps: obtain ship information, which includes basic ship information, ship scheduling diagram, ship picture and real-time ship video; perform target recognition based on YOLO5s with the introduction of extended Kalman filter algorithm, track the ship trajectory from the time the ship enters the lock, match the basic ship information and ship picture, and superimpose the ship detection frame and basic ship information of the trajectory tracking result on the real-time ship video; if occlusion occurs during the ship trajectory tracking process and the ship trajectory is lost, then based on the Deep OC-SORT algorithm, for the target lost in the real-time video, its position in the next frame or the next few frames is predicted, and the lost target is searched near the predicted position; after one or more targets are found near the predicted position, the IoU calculation is performed on the content of the ship detection frame obtained by Redis re-identification technology and trajectory tracking. When the IoU value is greater than the preset threshold, the found target and the target in the corresponding ship detection frame are regarded as the same target; during the ship trajectory tracking process, if it is identified that the ship has not moved for a preset time, it is judged that the ship gear docking is completed, and the actual gear map of the ship is formed;

[0142] Ship positioning module: used to obtain the ship's docking position when it is determined that the ship has completed docking;

[0143] Lock control system: used to compare the gear map with the scheduling gear map sent by the data storage module, the actual gear map, and the basic information of the ship to determine whether the ship is shifted according to the scheduling. If the ship does not shift according to the scheduling, an automatic alarm will be issued to remind the staff.

[0144] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0145] Example 3

[0146] This embodiment simulates the method described in Example 1 to verify the effectiveness of the present invention.

[0147] 1. Basic results display

[0148] Configure and use the Extended Kalman Filter to complete the filter estimation of the system model. When plotting the curve here, the time range n is determined to be 60. To make a better evaluation of the model at the beginning, the initial state is set to the same as the Extended Kalman Filter state. Figure 4 In the figure, the horizontal axis is r x [N], the vertical axis is r y [N], by Figure 4 It can be seen that the target's motion trajectory is roughly a straight line. The target state equation also shows that the target is performing uniform linear motion. Its x- and y-axis coordinate values ​​are proportional to the speed. The speed remains unchanged. The target is performing uniform linear motion and will be subject to random interference from noise, resulting in fluctuations. The reason is that a certain amount of process noise is added to its state equation.

[0149] Figure 5 is the distance change between the target and the observation origin, the horizontal axis is n, and the vertical axis is Figure 6 is the change in the orientation of the target and the observation origin, the horizontal axis is n, and the vertical axis is according to Figure 5 、 6 It can be seen that as the target moves, the azimuth angle continues to increase, indicating that the target is gradually moving upward. The EKF also has a good tracking effect on the target movement.

[0150] Figure 7 The horizontal axis is n, and the vertical axis is r determined by the state equation. y [N], vertical axis 2 is estimated by EKF Figure 8 Comparison of the true value and estimated value of the target y coordinate, the horizontal axis is n, the vertical axis 1 is ry[N] determined by the state equation, and the vertical axis 2 is estimated by EKF according to Figure 7、 8 Comparing the x and y coordinates of the target's true state with the state after EKF filtering, we can see that the EKF estimated curve can track the target's x and y coordinates well, regardless of the x or y coordinates. However, the tracking effect of EKF gradually deteriorates over time.

[0151] Figure 9 Comparison between the actual value and estimated value of the target x-direction velocity, the horizontal axis is n, and the vertical axis is v determined by the state equation k [N], vertical axis 2 is estimated by EKF Figure 10 Comparison between the actual and estimated values ​​of the target y-direction velocity, with n on the horizontal axis and v on the vertical axis determined by the state equation. r [N], vertical axis 2 is estimated by EKF according to Figure 9 、 10 It can be seen that the estimation errors of the velocities in the x and y directions are kept within 0.5 at all 60 time points, and the estimated values ​​at all 60 time points are relatively close to the true values.

[0152] 2. Error Analysis

[0153] From the tracking trajectory Figure 11 As can be seen above, the EKF algorithm initially performs well, but the estimated trajectory is generally poor. This is primarily because the state is four-dimensional information, while the observation is only two-dimensional angle information. Furthermore, the angle has a nonlinear relationship with the x and y values ​​in the state, making it difficult for the EKF algorithm to achieve good results. As time goes by, the Kalman estimate gradually deviates from the target's true trajectory.

[0154] from Figure 12 and 13 As can be seen, after several iterations of the algorithm, the error of the EKF estimation becomes larger and larger, and finally the EKF algorithm almost diverges. In a nonlinear system, it is very difficult to achieve continuous tracking of the target based on the initial state and observation information.

[0155] 3. Parameter Adjustment

[0156] In this embodiment, some parameters are changed and some typical curve changes are selected for analysis.

[0157] 1. Change the initial state

[0158] The initial state of the extended Kalman filter is s[0|0]=[15,5,-0.2,0.2] T Change to s[0|0]=[5,5,0,0] T, and the initial state is still s[0]=[15,5,-0.2,0.2] T .observe Figure 14 、 15 It can be seen that it is difficult to achieve continuous tracking of the target in a nonlinear system. For this target tracking model, the system is very dependent on the initial state, so the selection of the initial state is very important for target tracking.

[0159] 2. Increase the time range

[0160] The time point value is changed from 60 to 100. When the time range is increased, Figure 16 It can be seen that the estimated values ​​gradually disperse and the estimated value of the target y-axis speed is no longer so accurate. Figure 17 It can be seen that as the number of time points obtained increases, after several algorithm iterations, the error of EKF estimation becomes larger and larger, and finally the EKF algorithm is almost divergent.

[0161] 3. Increase process noise value

[0162] From 0.001 to 0.1, from Figure 18 The position and velocity estimation deviations are shown in Figure 2. The deviations reach more than 20 after 30 time points, and the Kalman filter has almost no effect. This is because the process noise is too large, and the increase in random disturbances leads to "no rules to follow", which ultimately makes the filter unable to reduce the noise. However, if the process noise is reduced to the original 0.000l, as shown in Figure 2. Figure 19 As shown in the figure, it can be clearly seen that Kalman filtering can greatly reduce noise and greatly reduce estimation deviation.

[0163] 4. Increase measurement noise

[0164] from Figure 20 Figure 21 It can be seen that increasing the measurement noise will cause the tracking trajectory of the Kalman filter to fluctuate, but overall it still fluctuates around the true trajectory of the target, and can also achieve a certain tracking effect.

[0165] Example 4

[0166] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM, and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0167] Many components in a device are connected to the I / O interface, including: input units, such as a keyboard and mouse; output units, such as various types of displays and speakers; storage units, such as magnetic disks and optical disks; and communication units, such as network cards, modems, and wireless communication transceivers. The communication unit allows the device to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0168] The processing unit performs the various methods and processes described above, such as methods S1 to S6. For example, in some embodiments, methods S1 to S6 may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S6 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S6 by any other appropriate means (e.g., by means of firmware).

[0169] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0170] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0171] In the context of the present invention, machine-readable medium can be a tangible medium that can contain or store a program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0172] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A ship gear recognition method based on target tracking and re-identification, characterized in that: The method comprises the following steps: Obtaining ship information, including basic ship information, ship dispatch diagrams, ship pictures, and real-time ship videos; The system uses the YOLO5s algorithm with an extended Kalman filter to identify targets. It tracks the ship's trajectory from the moment it enters the lock, matches basic ship information with images, and overlays the ship's detection frame and basic information from the trajectory tracking results on the real-time video. If occlusion occurs during ship tracking, causing the ship's trajectory to be lost, the Deep OC-SORT algorithm predicts the position of the target lost in the real-time video in the next frame or frames, and searches for the lost target near the predicted position. After one or more targets are found near the predicted position, the Redis re-identification technology is used to calculate the Intersection over Union (IoU) with the content in the ship detection frame obtained by trajectory tracking. When the IoU value is greater than a preset threshold, the found target and the target in the corresponding ship detection frame are considered to be the same target. During the ship trajectory tracking process, if it is identified that the ship has not moved for a preset time, the ship gear docking is determined to be completed, the ship's docking position is obtained, and the actual ship gear map is generated; Determine whether the ship is in accordance with the schedule according to the ship scheduling diagram, actual schedule diagram and basic ship information. If the ship is not in accordance with the schedule, an alarm will be issued. The Deep OC-SORT algorithm performs the following steps: Get the scaling rotation matrix M t =s t R t and the translation matrix T t ,in s t represents the scaling matrix, R t represents the rotation matrix; Introducing the scaling rotation matrix and translation matrix into the three components of the OC-SORT algorithm: OOS, OCM and OCR: For the OOS component, the Kalman filter is updated along a linear interpolation path starting from the last known measurement, which is given by [x c ,y c , a, s], the first two items are the center of the bounding box, and the center of the bounding box is c<-M t c+T t Transformation is obtained, where a is the area and s is the aspect ratio; For the OCM component, let p1 and p2 be the upper left and lower right corners of the bounding box. OCM uses the bounding box at the last Δt=3 to calculate the bounding box angular velocity. At each time step t, the transformation p is applied to the bounding box. i ←M t p i +T t ; For the OCR component, for the last seen bounding box position in OCR, at each time step t, apply p i ←M t p i +T t to adjust its position; Under the OC-SORT algorithm, the Kalman state is The Kalman state is adjusted as follows by camera motion compensation: Using low detector confidence to identify image degradation due to occlusion or blur, let e t is the tracklet appearance embedding at time t, the standard EMA is: e t =a t e t-1 +(1-a t )e new e new is the appearance of the matching detection added to the model, α t Defined as: Among them, s det is the detector confidence, σ is the detection confidence threshold used to filter noisy detections, and α f is a fixed value; Adaptive weighting: Use the standard cosine similarity between track and box embeddings to get an M×N appearance cost matrix A c , where M and N are the number of trajectories and detections, respectively, and A c [m, n] represents the item at the intersection of the mth row and nth column, which is combined with the IoU cost matrix to obtain C=I c +a w A c , and the weighting factor w b (m, n) is added to the global a w In the example, let T m Indicates tracking, d n Indicates detection, when T m When only one box has a high similarity score, in A c [m,:] increases the appearance weight, if the detection d n has only one distinct association with a trajectory, then the same operation is applied to A c Columns, using z diff To measure the discriminability of the detection box-track pair, z diff Defined as the difference between the highest and second highest values ​​in a row or column: Among them, ∈ is a hyperparameter; According to z diff The weighting factors are derived, which produce the final cost matrix C: C[m,n]=IoU[m,n]+[α w +w b (m,n)]A c [m,n]. The missing target search is achieved by minimizing the cost matrix C.

2. A ship gear recognition method based on target tracking and re-identification according to claim 1, characterized in that: The target recognition based on YOLO5s using the extended Kalman filter algorithm specifically includes the following steps: Initialize the state of the extended Kalman filter tracker: use the frame where YOL0v5s target detection fails as the third frame of the extended Kalman filter tracker, use the detection values ​​of the first two frames as the initial state of the tracker, and initialize the state vector and error covariance matrix of the second frame; State prediction: predict the current state vector and state error covariance matrix based on the previous state; Calculate the extended Kalman gain: define the observation vector, calculate the extended Kalman gain through the daily matrix and the prior results obtained in the prediction stage; Update: Update the state vector and error covariance matrix according to the extended Kalman gain to achieve continuous tracking of the target; Candidate region creation and matching: Based on the prior system state bounding box position of the extended Kalman filter, the region is expanded to obtain the candidate region. The difference value hash algorithm is used to match the candidate regions, and the candidate region with the smallest difference is selected as the observation value.

3. A ship gear recognition method based on target tracking and re-identification according to claim 2, characterized in that: The state vector x and error covariance matrix P of the second frame are shown as follows: in and Represents the coordinates of the upper left corner and lower right corner of the bounding box on the x-axis and y-axis respectively. The subscript m represents the observation value, and the subscript 2 of m represents the second frame. and To calculate the result, ΔT is the time taken from the first frame to the second frame, cent(x) and cent(y) represent the centroid coordinates of the bounding box on the x-axis and y-axis, respectively.

4. The method for ship gear recognition based on target tracking and re-identification according to claim 2, characterized in that: The extended Kalman filter updates the state vector and error covariance matrix according to the extended Kalman gain: Among them, K k is the extended Kalman gain, is the prior state vector, is the prior error covariance matrix, H is the Jacobian matrix, Z k is the observation vector, Represents the nonlinear transformation from state vector to measurement vector:

5. The ship gear recognition method based on target tracking and re-identification according to claim 2 is characterized in that: The candidate region is created by moving the position of the prior system state bounding box of the extended Kalman filter in eight directions, namely 45° to the upper left, 45° to the upper right, 45° to the lower right, 45° to the lower left, and directly above, directly below, directly to the right, and directly to the left. The degree of movement is represented by the intersection over union (IoU). The bounding box of the prior system state is represented by E, and the bounding box after movement is represented by F. The intersection over union (IoU) is defined as: The degree of movement is determined by presetting the intersection-over-union ratio value, and the expansion areas in eight directions are obtained. Combined with the prior system state bounding box position, nine candidate areas are obtained.

6. The method for ship gear recognition based on target tracking and re-identification according to claim 2, characterized in that: The specific method of using the difference value hash algorithm to match candidate regions is as follows: Reduce the expanded candidate area image to n×(n-1), where n is a preset value; Convert the reduced image into a grayscale image, assign 1 to each point in each row where the previous pixel value is greater than the next pixel value, and 0 if the previous pixel value is greater than the next pixel value, and generate a hash value; Calculate the Hamming distance between the candidate region and the image within the target bounding box of the previous frame. The smaller the distance, the smaller the image difference. Select the candidate region with the smallest difference as the observation value.

7. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images