An Online Pedestrian Tracking Method Based on an Improved Object Detection Network

Abstract

The present invention relates to an online pedestrian tracking method based on an improved object detection network, belonging to the field of object detection, and includes: inputting the current frame image into YOLOX object detection that fuses the CA attention mechanism, dividing the detection boxes according to the confidence of object detection into high-score detection boxes and low-score detection boxes, using the ReID network and GIoU to perform similarity matching between the high-score detection boxes and the Kalman filter prediction boxes, updating the successfully matched ones with the Kalman filter, for the detection boxes with higher scores but no matched tracking trajectories, creating a new tracking trajectory, and for the second matching, using the low-score detection boxes and the tracking trajectories of the high-score detection boxes that were not matched in the first time (objects with low scores due to severe occlusion in the current frame) for matching, and for the tracking trajectories without matched detection boxes, they will be retained for 30 frames and matched when they appear again, and deleted if not. The present invention can effectively reduce the influence of occlusion on recognition and improve the recognition rate and recognition speed.

Description

Technical Field

[0001] The present invention belongs to the field of object detection and relates to an online pedestrian tracking method based on an improved object detection network. Background Art

[0002] Multi-object tracking in videos is a basic and important task for many vision applications, such as video surveillance and autonomous driving. The purpose of this task is to locate multiple objects in each frame and obtain the trajectories of each identity. Currently, most methods are tracking methods based on detection boxes, including online tracking methods and offline tracking methods: The online tracking method constructs an association matrix according to the similarity between the target and the detection box, and uses a matching algorithm to match the positions of the target and the detection box; The offline tracking method constructs a graph according to the detection boxes within a period of time and the similarity between them, and uses subgraph partitioning to solve the object tracking problem.

[0003] The SORT algorithm proposed by Bewley et al. is a simple online real-time multi-object tracking algorithm, which mainly uses Kalman filtering to propagate the target object to future frames, and then uses IOU as a metric to establish relationships. The DeepSort method proposed by Wojke Nicolai et al. is the result of an improvement of the Sort method, which fuses a re-identification network and uses the Hungarian algorithm to perform similarity matching between the detection box and the predicted trajectory.

[0004] Existing object tracking systems are easily affected by factors such as occlusion, camera pixels, and background changes, and problems such as incorrect object recognition may occur. Most of the current tracking systems are based on the detection results, and the detection performance directly affects the tracking effect. However, if the detector is too precise, the overall speed will drop significantly. When the target is interfered by the background or occluded, the ID number of the target box may change, and it may also occur when a numbered target re-enters the camera range and its number changes. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a pedestrian tracking method with YOLOX integrating an attention mechanism as the object detection network, so as to suppress the influence of occlusion on recognition and improve the recognition rate and recognition speed. In this method, coordinate attention is integrated into the Neck part of YOLOX to enhance the feature extraction ability, and then the results of object detection are input into the tracker. Then, the ReID re-identification module and GIoU are used to perform similarity matching between the detection box and the predicted box, and finally the trajectories are matched according to the matching distance cost.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] An online pedestrian tracking method based on an improved object detection network, comprising the following steps:

[0008] S1: Obtain the pedestrian image in the pedestrian video frame and preprocess the pedestrian image;

[0009] S2: Input the picture into the YOLOX object detection network, and a CA attention module is fused at the output position of the Backbone of the YOLOX object detection network;

[0010] S3: Input the output of the object detection into the tracker to obtain a high-score detection box and a low-score detection box, and the confidence of the high-score detection box is higher than that of the low-score detection box;

[0011] S4: Perform similarity matching between the high-score detection box and the Kalman filter prediction box; for the detection box that has no matched trajectory but has a high score, start a new trajectory and use the Kalman filter to update the trajectory. For the successfully matched detection box, update the trajectory set with the Kalman filter, and the trajectory that fails to match waits for the second similarity matching;

[0012] S5: Perform similarity matching between the low-score detection box and the unmatched trajectory. The matching metric uses IoU. For the successfully matched one, update the trajectory with the Kalman filter. Delete the unmatched detection box. If the unmatched trajectory fails to match the detection box again after a certain period of time, delete it.

[0013] Further, for step S1, the obtaining the pedestrian image in the pedestrian video frame and preprocessing the pedestrian image specifically includes:

[0014] S11: Obtain the pedestrian image in the pedestrian video frame, sample it into a set of several frame images {I1, I2, ···, I n}, and use YOLOX-s to detect each frame image, and output the detection set {D1, D2, ···, D n} of the first to nth frames, and the coordinate position information {P1, P2, ···, P m} of pedestrians 1 to m, including the center coordinates, aspect ratio, and acceleration in each direction;

[0015] S12: Preprocess the pedestrian image by using Mosaic data augmentation and MixUp data augmentation.

[0016] Further, in step S2, the input-output process of the CA attention module includes:

[0017] S21: The data is input from the output position of the Backboone, and each channel is encoded along the horizontal and vertical directions by using pooling kernels of (H, 1) and (1, W), and an orientation-aware attention feature map z of size C×H×1 is output Hand a direction-aware attention feature map z of size C×1×W W ;

[0018] S22: Concatenate z H and z W using Concat, and then generate a process feature map f∈R C / r×1×(H+W) , where r represents the reduction ratio of channel downsampling;

[0019] S23: Then divide f into f h ∈R C / r×H and f w ∈R C / r×W along the horizontal and vertical directions, and use another two 1×1 convolutions to adjust f h and f w into tensors with the same number of channels as the input X;

[0020] S24: Then use the Sigmiod activation function to obtain two attention weights g h and g w for two independent spatial directions, and then expand g h and g w . Finally, obtain an output feature map with stronger representation information, output it to the Neck part of YOLOX-s, and finally pass it through the detection head.

[0021] Furthermore, step S3 specifically includes: inputting the output of object detection into a tracker, where there are two confidence thresholds in the tracker, including a high-score threshold (high-shresh) and a low-score threshold (low-shresh); those above the high-score threshold are high-score detection boxes, and those between the high-score threshold and the low-score threshold are low-score detection boxes. Delete all the pedestrian boxes with confidence less than the low-score threshold among all the pedestrian boxes to obtain the set of low-score detection boxes, and finally obtain the set of high-score detection boxes and the set of low-score detection boxes.

[0022] Furthermore, the Kalman filter prediction boxes in step S4 are used to predict a trajectory set through Kalman filtering, and its state update equation is as follows:

[0023]

[0024] where represents the posterior state estimate value at time k, represents the prior estimate value, that is, according to the optimal prediction estimate value of the previous moment, z k represents the observation value. Form a prediction box set D t .

[0025] Further, in step S4, the similarity matching between the high-score detection box and the Kalman filter prediction box is specifically performed by obtaining the final similarity c through the ReID module and GIoU, and using the Hungarian algorithm to match the trajectories;

[0026] The ReId module is used to detect pedestrians in the pedestrian trajectory library and under the high-score detection box, extract the appearance feature distances of pedestrians under the high-score detection box to determine whether they are the same person, and is also used to update the pedestrian information in the pedestrian trajectory library; the pedestrian trajectory library includes pedestrian appearance features and pedestrian positions;

[0027] The GIoU considers the non-overlapping parts of the detected box and the prediction box, reflecting the overlapping manner and coincidence degree of the detected box and the prediction box.

[0028] Further, the ReID module uses the ReID network model to extract feature vectors from the prediction box and the detection box respectively, uses the coordinates in P j to crop the image, inputs the set of pedestrian images under the cropped high-score detection box into the pedestrian re-identification network model to obtain the appearance features of pedestrians under the high-score detection box, compares the appearance features with the pedestrians reappearing in the picture, and calculates the similarity d (1) (i, j) of the feature vectors.

[0029] Further, the formula for the GIoU to calculate the similarity between the prediction box and the detection box is as follows:

[0030]

[0031] where IoU is the intersection over union, and the similarity d (2) (i, j) is obtained. According to the formula c i,j =μd (1) (i, j)+(1 - μ)d (2) (i, j), a hyperparameter μ is set to obtain the final similarity c.

[0032] Further, the use of the Hungarian algorithm to match the trajectories specifically includes:

[0033] Initialize the bipartite graph, confirm the previous frame target and the possible matching current frame target according to the input cost matrix. Let U be the previous frame set and V be the current frame set, and perform the matching in order according to the ID:

[0034] First, match the target 1 in the current frame that may match the target 1 in the previous frame, then match the target 2, and then match the target 3. If the target that the previous frame's target 3 can match has already been matched by targets 1 and 2, then match the target in U that previously matched target 1 with another target. If the target 1 in U can match the target 2 in V, and the target 2 has also been matched by the target 2 in U, then replace the matching target of the target 2 in U, so that the targets 1, 2, and 3 in U are all matched to the targets, and then match the subsequent targets until the targets in V that are not matched are regarded as newly emerging targets. Generally speaking, it is a recursive process.

[0035] The beneficial effects of the present invention are as follows:

[0036] (1) Due to problems such as too small targets and frequent occlusions in the actual situation of multi-target tracking, the present invention adds a coordinate attention module to the Neck part of YOLOX, enabling the system to better focus on the detailed features of the characters in the video stream, reducing information loss during the feature extraction process, making the feature fusion part have richer information, and having a very small computational overhead, thereby improving the detection effect, reducing false detection results, and achieving better tracking effects.

[0037] (2) The present invention uses an object detection network to perform pedestrian detection on the current frame image to obtain high-score detection frames and low-score detection frames. The high-score detection frames and prediction frames use ReID and GIoU as metrics for the first similarity matching, and the low-score detection frames and unmatched trajectories use IoU for the second similarity matching. The use of two pedestrian matchings enhances the accuracy of pedestrian tracking, and the fusion of the ReID module makes the tracking effect better in the case of occlusion, and can avoid problems such as pedestrian tracking loss and identity ID exchange.

[0038] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be learned from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail preferably with reference to the accompanying drawings, where:

[0040] Figure 1 is the flow chart of the online pedestrian tracking method based on the improved object detection network described in the present invention;

[0041] Figure 2 is the position schematic diagram of adding the CA attention module;

[0042] Figure 3 This is a comparison chart of the experimental results of the present invention and the detection frame results with the deepSORT algorithm in a verification video, where (a) and (b) are screenshots of the deepSORT algorithm, and (c) and (d) are the experimental results of the present invention. Specific Embodiments

[0043] The following specific examples illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0044] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as limiting the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0045] In the drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0046] The present invention is a pedestrian tracking algorithm that improves the object detection network, and its flowchart is as Figure 1 shown. Specifically, it includes the following steps:

[0047] Step 1: Obtain pedestrian images in the pedestrian video frames, sample them into a set of several frame images {I1, I2, ···, I n}, and use YOLOX-s to detect each frame image and output the detection results {D1, D2, ···, D n}. Among them, D i represents the detection set of the i-th frame, {P1, P2, ···, P m}, Pj Represents the coordinate position information of each pedestrian, including the center coordinates, aspect ratio, and accelerations in each direction. Preprocess the pedestrian images using Mosaic data augmentation and MixUp data augmentation.

[0048] Step 2: Input the image into the YOLOX-s object detection network. The network integrates the CA attention mechanism. The CA attention is located at the output position of the Backbone of YOLOX-s, and its position is as Figure 2 , and the input-output process of the CA attention module is as follows: Input from the output position of the Backboone, use pooling kernels of (H,1) and (1,W) to encode each channel along the horizontal and vertical directions, and output a direction-aware attention feature map z of size C×H×1 H and a direction-aware attention feature map z of size C×1×W W . Concatenate z H and z W through Concat, and then use a 1×1 convolution module to generate the process feature map f∈R C / r×1×(H+W) , where r represents the reduction ratio of channel downsampling. Then divide f along the horizontal and vertical directions into f h ∈R C / r×H and f w ∈R C / r×W , and then use another two 1×1 convolutions to adjust f h and f w into tensors with the same number of channels as the input X. Then use the Sigmiod activation function to obtain the attention weights g h and g w in two independent spatial directions. Then expand g h and g w . Finally, obtain the output feature map with stronger representation information and output it to the Neck part of YOLOX-s, and finally through the detection head.

[0049] Step 3: Input the output of the object detection into the tracker. Set two confidence thresholds, a high score threshold (high-shresh) and a low score threshold (low-shresh). Those above the high score threshold are high score detection boxes, and those between the high score threshold and the low score threshold are low score detection boxes. Delete all the pedestrian boxes with a confidence less than the low score threshold among all the pedestrian boxes to obtain the set of low score detection boxes. Finally, obtain the set of high score detection boxes and the set of low score detection boxes, where the confidence of the high score detection boxes is higher than that of the low score detection boxes.

[0050] Step 4: Predict the trajectory set using Kalman filtering. Its state update equation is as follows:

[0051]

[0052] wherein represents the posterior state estimate value at time k, represents the prior estimate value, that is, according to the optimal prediction estimate value of the previous moment, z k represents the observation value. Form a set of prediction boxes D t .

[0053] Use the ReID network model to extract feature vectors from the prediction boxes and detection boxes respectively. Use the coordinates in P j to crop the image, and input the set of cropped pedestrian images (i.e., the pedestrian images under the high-score detection boxes) into the pedestrian re-identification network model to obtain the pedestrian appearance features (the pedestrian appearance features under the high-score detection boxes), compare the appearance features with the pedestrians who appear in the picture again, and calculate the similarity d (1) (i, j); then use the GIoU formula to calculate the similarity between the prediction box and the detection box. The GIoU formula is as follows:

[0054]

[0055] IoU is the intersection over union, and the similarity d (2) (i, j) is obtained. According to the formula c i,j =μd (1) (i, j)+(1 - μ)d (2) (i, j), and a hyperparameter μ is set to obtain the final similarity c.

[0056] Perform similarity matching between the high-score detection boxes and the Kalman filter prediction boxes. The matching metric uses the final similarity c obtained from the ReID feature metric and GIoU. The ReId module can detect the pedestrian trajectory library and the pedestrians under the high-score detection boxes, and can extract the appearance feature distance of the pedestrians under the high-score detection boxes to determine whether they are the same person, and can also update the pedestrian information in the pedestrian trajectory library. The pedestrian trajectory library includes pedestrian appearance features and pedestrian positions. GIoU takes into account the non-overlapping parts of the detected box and the prediction box that IoU does not consider, and can reflect the overlapping method and coincidence degree of the detected box and the prediction box. According to the similarity c, use the Hungarian algorithm to match the trajectories. For the detection boxes that do not match the trajectories but have high scores, a new trajectory is opened, and the Kalman filter is used for trajectory update. The detected boxes that are successfully matched are updated with the Kalman filter for the trajectory set, and the trajectories that fail to match wait for the second similarity matching.

[0057] Step Five: Perform similarity matching between the low-score detection boxes and the trajectories that have not been matched. The matching metric uses IoU. The successfully matched ones are updated with the Kalman filter for the trajectories, the detection boxes that fail to match are deleted, and the trajectories that fail to match are retained for 30 frames. If the detection boxes cannot be matched again, they are deleted.

[0058] Example: Multi-object Tracking Experiment

[0059] The datasets used are MOT17 and CrowdHuman. Training is carried out on the MOT17 training set and CrowdHuman, and validation is performed on half of the MOT17 test set. The online pedestrian tracking algorithm based on the improved object detection network uses Mosaic and MixUp for data augmentation, adopts the cosine annealing strategy to dynamically update the learning rate, and applies the FP16 mixed precision technology to accelerate convergence. The experimental data is shown in Table 1.

[0060] Table 1

[0061]

[0062] It can be seen from the table that the accuracy has been greatly improved compared with other methods, the switching frequency of identity ID is low (high IDF1), and the real-time performance is good (high FPS), which fully proves that the present invention can not only improve the accuracy of multi-object tracking, but also effectively control the influence of missed detection targets on the experimental results.

[0063] Figure 3 This is a comparison diagram of the experimental results of the present invention and the detection box results in a validation video. Among them, (a) and (b) are screenshots of the deepSORT algorithm. It can be found that the target will be misdetected due to background problems and the position information is missing; (c) and (d) are the experimental results of the present invention. It can be seen that there is no situation of misdetecting a dummy as a real person. It can be seen from (b) and (d) that the present invention can effectively track the target when the target pedestrian is occluded or the target is small. Even if the target is occluded, the present invention will match the same target in the next step, and has good robustness in the case of occlusion.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An online pedestrian tracking method based on an improved object detection network, characterized in that: Including the following steps: S1: Obtain the pedestrian image in the pedestrian video frame and preprocess the pedestrian image; S2: Input the picture into the YOLOX object detection network, and a CA attention module is fused at the output position of the Backbone of the YOLOX object detection network; S3: Input the output of the object detection into the tracker to obtain high-score detection boxes and low-score detection boxes, and the confidence of the high-score detection boxes is higher than that of the low-score detection boxes; S4: Perform similarity matching between the high-score detection boxes and the Kalman filter prediction boxes; for the detection boxes that do not match the trajectory but have a high score, a new trajectory is opened, and the Kalman filter is used for trajectory update. The detection boxes that are successfully matched are used to update the trajectory set with the Kalman filter, and the trajectories that fail to match wait for the second similarity matching; in step S4, the similarity matching between the high-score detection boxes and the Kalman filter prediction boxes is specifically to obtain the final similarity c through the ReID module and GIoU, and the Hungarian algorithm is used to match the trajectories; The ReId module is used to detect pedestrians in the pedestrian trajectory library and under the high-score detection boxes, extract the appearance feature distance of pedestrians under the high-score detection boxes to determine whether they are the same person, and is also used to update the pedestrian information in the pedestrian trajectory library; the pedestrian trajectory library includes pedestrian appearance features and pedestrian positions; The GIoU considers the non-overlapping parts of the detected box and the predicted box, reflecting the overlapping method and coincidence degree of the detected box and the predicted box; S5: Perform similarity matching between the low-score detection boxes and the trajectories that do not match. The matching metric is IoU. The successfully matched ones are updated with the Kalman filter for the trajectory. The detection boxes that fail to match are deleted. After the trajectories that fail to match are retained for a certain time, if they fail to match the detection boxes again, they will be deleted.

2. The online pedestrian tracking method based on an improved object detection network according to claim 1, characterized in that: In step S1, the obtaining of the pedestrian image in the pedestrian video frame and the preprocessing of the pedestrian image specifically include: S11: Obtain the pedestrian image in the pedestrian video frame, sample it into a set of several frame images {I1, I2, ···, I n}, and use YOLOX-s to detect each frame image, and output the detection sets {D1, D2, ···, D n} for the 1st to nth frames, as well as the coordinate position information {P1, P2, ···, P m} of pedestrians 1 to m, including the center coordinates, aspect ratio, and accelerations in each direction; S12: Preprocess the pedestrian image using Mosaic data augmentation and MixUp data augmentation.

3. The online pedestrian tracking method based on an improved object detection network according to claim 1, characterized in that: In step S2, the input-output process of the CA attention module includes: S21: The data is input from the output position of Backboone. The pooling kernels of (H, 1) and (1, W) are used to encode each channel along the horizontal and vertical directions, and a direction-aware attention feature map z of size C×H×1 is output. H and a direction-aware attention feature map z of size C×1×W W ; W and C represent the height, width, and number of channels of the feature map, respectively. S22: Concatenate z H and z W through Concat, and then use a 1×1 convolution module to generate the process feature map f ∈ R C / r×1×(H+W) , where r represents the reduction ratio of channel downsampling, and R represents the scale of the feature map; S23: Then divide f into f h h ∈R C / r×H and f w w ∈R C / r×W along the horizontal and vertical directions, and then use another two 1×1 convolutions to adjust f h h and f w w into tensors with the same number of channels as the input X; S24: Then use the Sigmoid activation function to obtain the attention weights g in two independent spatial directions h and g w , and then expand g h and g w . Finally, obtain the output feature map with stronger representation information and output it to the Neck part of YOLOX-s, and finally pass it through the detection head.

4. The online pedestrian tracking method based on the improved object detection network according to claim 1, characterized in that: Step S3 specifically includes: Input the output of the object detection into the tracker. There are two confidence thresholds in the tracker, including a high-score threshold and a low-score threshold; those higher than the high-score threshold are high-score detection boxes, and those between the high-score threshold and the low-score threshold are low-score detection boxes. Delete all the pedestrian boxes with a confidence less than the low-score threshold among all the pedestrian boxes to obtain the set of low-score detection boxes, and finally obtain the set of high-score detection boxes and the set of low-score detection boxes.

5. The online pedestrian tracking method based on the improved object detection network according to claim 1, characterized in that: In step S4, the Kalman filter prediction box is used to predict the trajectory set through the Kalman filter, and its state update equation is as follows: Among them represents the posterior state estimate value at time k, represents the prior estimate value, that is, based on the optimal prediction estimate value of the previous moment, z k represents the observation value, forming the prediction box set D t ; K k represents the Kalman gain, and H represents the linear relationship matrix between z k and...

6. The online pedestrian tracking method based on an improved object detection network according to claim 1, characterized in that: The ReID module uses the ReID network model to extract feature vectors from the prediction box and the detection box respectively, and uses the coordinates in P j to crop the image, and inputs the set of pedestrian images under the high-score detection box obtained by cropping into the pedestrian re-identification network model to obtain the appearance features of the pedestrians under the high-score detection box, and compares the appearance features with the pedestrians who appear in the picture again. Calculate the similarity d of the eigenvectors (1) (i, j), where i and j represent the i-th predicted bounding box and the j-th detected bounding box respectively, and P represents the coordinate position information.

7. The online pedestrian tracking method based on an improved object detection network according to claim 6, characterized in that: The formula for the GIoU to calculate the similarity between the predicted box and the detected box is as follows: Among them, A c represents the area of the minimum closed region of the prediction box and the detection box, u represents the area of the union of the prediction box and the detection box, IoU is the intersection over union, and the similarity d is obtained (2) (i, j), according to the formula c i,j = μd (1) (i, j) + (1 - μ)d (2) (i, j), and a hyperparameter μ is set to obtain the final similarity c.

8. The online pedestrian tracking method based on an improved object detection network according to claim 7, wherein: The specific steps of using the Hungarian algorithm to match the trajectories include: Initialize the bipartite graph, confirm the previous frame objects and the possible matching current frame objects according to the input cost matrix. Let U be the previous frame set and V be the current frame set, and perform matching in order according to the ID: First, match the target 1 in the current frame that may match the target 1 in the previous frame, then match the target 2, and then match the target 3. If the target that the previous frame's target 3 can match has already been matched by the targets 1 and 2, then match the target in U that previously matched the target 1 with another target. If the target 1 in U can match the target 2 in V, and the target 2 has also been matched by the target 2 in U, then replace the matching target of the target 2 in U, so that the targets 1, 2, and 3 in U are all matched to the targets. Then match the subsequent targets until the targets in V that are not matched are regarded as newly emerging targets.

Images