Multi-target tracking method and apparatus

Abstract

The present disclosure relates to the technical field of computer, and provides a multi-target tracking method and device. The method comprises: performing target detection on a first scene picture and a second scene picture respectively to obtain a tracking box set, a first feature map, a detection box set and a second feature map; determining a first search area in the second scene picture according to the position of a first tracking box in the first scene picture to obtain a first search area feature map; inputting the first tracking box, a first detection box in the detection box set and the first search area feature map into a preset adaptive search model to obtain a second search area and obtain a second search area feature map; modeling target motion according to the second search area and the second search area feature map to obtain a prediction box; and matching the prediction box and the detection box in the detection box set to obtain the trajectory of the target corresponding to the first tracking box.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to a multi-target tracking method and device. BACKGROUND

[0002] Compared with the detection problem, the tracking problem needs to accurately predict the target of the current frame by using the detection of the past frame in addition to the target detection of the current frame.

[0003] In the tracking process of the target, determining the search region with appropriate size is a key step for realizing efficient and effective object association and tracking. In the related art, the determination process of the search region is relatively rough, so that the accurate object association cannot be realized, and the tracking result with high precision cannot be obtained. SUMMARY

[0004] Therefore, the embodiments of the present disclosure provide a multi-target tracking method, device, electronic equipment and computer readable storage medium to solve the technical problem of low multi-target tracking precision in the prior art.

[0005] In a first aspect, the embodiments of the present disclosure provide a multi-target tracking method, which comprises: performing target detection on a first scene picture collected at a first time and a second scene picture collected at a second time respectively to obtain a tracking box set at the first time, a first feature map of the first scene picture, a detection box set at the second time and a second feature map of the second scene picture; determining a first search region corresponding to a first tracking box in the tracking box set in the second scene picture according to the position of the first tracking box in the first scene picture, to obtain a first search region feature map of the first search region in the second feature map; inputting the first tracking box, a first detection box in the detection box set and the first search region feature map into a preset adaptive search model to obtain a second search region, and mapping the position information of the second search region to the first feature map to obtain a second search region feature map; modeling the target motion according to the second search region and the second search region feature map to obtain a prediction box; matching the prediction box and the detection box in the detection box set to obtain the trajectory of the target corresponding to the first tracking box.

[0006] In a second aspect, the present disclosure provides a multi-target tracking device, comprising: a target detection unit, configured to perform target detection on a first scene picture collected at a first time and a second scene picture collected at a second time, to obtain a tracking box set at the first time, a first feature map of the first scene picture, a detection box set at the second time, and a second feature map of the second scene picture; a preliminary search unit, configured to determine a first search area corresponding to a first tracking box in the tracking box set in the second scene picture according to a position of the first tracking box in the first scene picture, to obtain a first search area feature map of the first search area in the second feature map; an adaptive search unit, configured to input the first tracking box, a first detection box in the detection box set, and the first search area feature map into a preset adaptive search model, to obtain a second search area, and map position information of the second search area to the first feature map, to obtain a second search area feature map; a motion modeling unit, configured to model target motion according to the second search area and the second search area feature map, to obtain a prediction box; and a matching unit, configured to match the prediction box and the detection box in the detection box set, to obtain a trajectory of a target corresponding to the first tracking box.

[0007] In a third aspect, the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0008] In a fourth aspect, the present disclosure provides a computer readable storage medium, which stores a computer program, wherein the computer program is executable by a processor to implement the steps of the above method.

[0009] Compared with the prior art, the present disclosure has the beneficial effects that: the corresponding first search area is determined according to the position of the first tracking box in the first scene picture, the second search area is determined according to the first search area and the preset adaptive search model, the target motion is modeled according to the second search area, and the trajectory of the target is obtained according to the modeling result and the detection box in the detection box set. This scheme can dynamically adjust the search area according to the detection and tracking prior, thereby improving the accuracy of object association and obtaining a target tracking result with higher precision. BRIEF DESCRIPTION OF DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0011] Figure 1is a flowchart of a multi-target tracking method provided by an embodiment of the present disclosure.

[0012] Figure 2 is a structural diagram of a multi-target tracking system provided by an embodiment of the present disclosure.

[0013] Figure 3 is a structural diagram of an adaptive search model provided by an embodiment of the present disclosure.

[0014] Figure 4 is a structural diagram of an adjustment factor generator provided by an embodiment of the present disclosure.

[0015] Figure 5 is a structural diagram of a motion modeling layer provided by an embodiment of the present disclosure.

[0016] Figure 6 is a structural diagram of a multi-target tracking device provided by an embodiment of the present disclosure.

[0017] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0018] In the following description, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the present embodiments of the present disclosure. However, persons skilled in the art should understand that the present disclosure can be implemented in other embodiments without these specific details. In other cases, well-known systems, devices, circuits, and methods have not been described in detail in order not to obscure the description of the present disclosure.

[0019] The multi-target tracking method and device according to the embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0020] Figure 1 is a flowchart of a multi-target tracking method provided by an embodiment of the present disclosure. The method provided by the embodiment of the present disclosure can be executed by any electronic device with computer processing capability, such as a terminal or a server. As shown in the figure, the multi-target tracking method includes: Figure 1

[0021] In step S101, target detection is performed on a first scene picture collected at a first time and a second scene picture collected at a second time, respectively, to obtain a tracking box set at the first time, a first feature map of the first scene picture, a detection box set at the second time, and a second feature map of the second scene picture.

[0022] Specifically, a first scene picture collected at a first time t can be input to a target detector to obtain a tracking box set for tracking targets at the first time t​ input a second scene picture collected at a second time t+δ to the target detector to obtain a detection box set at the second time t+δ . The target detector can be a human body detector that takes a human body as a detection target. The target detector can adopt a yolox model, that is, the first scene picture and the second scene picture can be input to the target detector yolox model for target detection. The yolox model can provide high-performance target detection. The backbone of the yolox model or other target detector can respectively extract features from the first scene picture and the second scene picture to obtain corresponding first feature map and second feature map .

[0023] In the embodiments of the present disclosure, the first time t is a previous time, and the second time t+δ is a later time. The first scene picture and the second scene picture can be two pictures taken in sequence in the same environment, or two pictures in the same video taken in the environment.

[0024] In step S102, a first search region corresponding to the first tracking box in the second scene picture is determined according to the position of the first tracking box in the tracking box set in the first scene picture, to obtain a first search region feature map of the first search region in the second feature map.

[0025] Specifically, step S102 is a process of preliminary search to obtain the first search region. According to the position of the first tracking box in the tracking box set in the first scene picture, a corresponding first search region can be set in the second scene picture, and a first search region feature map of the first search region is obtained , wherein i is the index of the target in the tracking box set, i=1, 2, 3…n, and n is the number of tracking boxes in the tracking box set.

[0026] In step S103, the first tracking box, the first detection box in the detection box set, and the first search region feature map are input to a preset adaptive search model to obtain a second search region, and position information of the second search region is mapped to the first feature map to obtain a second search region feature map.

[0027] Specifically, the adaptive search model can be an adaptive search region module (Adaptive Search Region Module). The first tracking box , the detection box set , and the first detection box feature map are input to the adaptive search region module, and a refined second search region can be obtained. The position information of the refined second search region is mapped to the first feature map The second search region feature map can be obtained .

[0028] In step S104, the target motion is modeled according to the second search region and the second search region feature map, and a prediction box is obtained.

[0029] Specifically, the second search region and the second search region feature map are input to a motion modeling layer to model the target motion, and a prediction box .

[0030] In step S105, the prediction box is matched with the detection boxes in the detection box set, and the trajectory of the target corresponding to the first tracking box is obtained.

[0031] Specifically, the prediction box set is matched with the detection boxes in the detection box set at the second time t+delta , an association is formed, and the trajectory of the target corresponding to the first tracking box is obtained , where i is the index of the target in the tracking box set, i=1, 2, 3……n, and n is the number of tracking boxes in the tracking box set. Step S105 can be implemented by inputting the prediction box and the detection boxes in the detection box set to a matching layer.

[0032] According to the multi-target tracking scheme provided by the embodiments of the present disclosure, an adaptive search region module can be provided, which generates a scaling factor with adaptation by simultaneously considering detection and tracking priors to dynamically adjust the second search region, and further provides a motion modeling method to improve tracking accuracy.

[0033] As shown in Figure 2 , the processing of the first scene picture 211 and the processing of the second scene picture 221 are like two processing processes of twins, and in combination with the use of the adaptive search model 231, the embodiments of the present disclosure provide an adaptive search region twin multi-target tracking network, which uses detection priors to improve tracking accuracy and keeps the target on the trajectory in the case of tracking prior compensation for missed detection.

[0034] Specifically, the first scene picture 211 is input to the target detector 212, and a first feature map 213 can be obtained. The second scene picture 221 is input to the target detector 222, and a second feature map 223 can be obtained. According to the position of the first tracking box 214 in the first scene picture, a first search region 224 corresponding to the first tracking box in the second scene picture is determined, and a first search region feature map 225 of the first search region in the second feature map is obtained.

[0035] The first tracking box, the first detection box in the detection box set, and the first search region feature map are input to a preset adaptive search model 231 to obtain a second search region, and position information of the second search region is mapped to the first feature map through a mapping layer 232 , to obtain a second search region feature map . The second search region and the second search region feature map are input to a motion modeling layer 233 to model the motion of the target, and a predicted box can be obtained. The predicted box is input to a matching layer 234, and a target trajectory can be obtained.

[0036] In step S102, the corresponding position in the second scene picture can be determined according to the position of the first tracking box in the first scene picture, and a coordinate region in which the corresponding position is located is expanded outward by 2 times the size to obtain the first search region.

[0037] In the embodiments of the present disclosure, expanding the coordinate region outward by 2 times the size is only an exemplary description of expanding the coordinate region outward by a certain size, and in actual applications, it is not limited to 2 times the size. Expanding the coordinate region outward can improve the accuracy of obtaining the predicted box set and the target tracking accuracy.

[0038] In step S103, the first tracking box and the first search region feature map are input into a first box filter of the adaptive search model to obtain a first filtered feature map; the first detection box and the first search region feature map are input into a second box filter of the adaptive search model to obtain a second filtered feature map; the first tracking box, the set of detection boxes and the first search region feature map are input into a mask generator of the adaptive search model to obtain a mask feature map; the mask feature map and the first search region feature map are input into a scaling factor generator of the adaptive search model to obtain a scaling factor; an adaptive filter of the adaptive search model using the scaling factor and a filtered feature value as input data is used to perform adaptive filtering on the first search region feature map to obtain the second search region, wherein the filtered feature value is a maximum value selected from the first filtered feature map and the second filtered feature map.

[0039] As shown in Figure 3 , the adaptive search model can adaptively adjust and refine the first search region based on tracking information of past frames and detection results of a current frame to obtain an adjusted second search region.

[0040] Specifically, the first tracking box and the first search region feature map are input into a first box filter 311 to obtain a first filtered feature map , and the first box filter 311 processes the first tracking box and the first search region feature map according to the following formula (1):

[0041] (1)

[0042] wherein, is a center point coordinate of the first tracking box , and is a width and height of the first tracking box .

[0043] The set of detection boxes and the first search region feature map are input into a second box filter 312 to obtain a second filtered feature map , and the second box filter 312 processes the set of detection boxes and the first search region feature map according to the following formula (2):

[0044] (2)

[0045] wherein, center point coordinates of the bounding box set width and height of the bounding box set

[0046] Further, the first filtered feature map and the second filtered feature map are input into the maximum selector 313 to retain the most significant value, so that different objects can be better distinguished, which is particularly advantageous for crowded scenes.

[0047] After that, the first tracking box , the bounding box set and the first search area feature map are input into the mask generator (Mask Filter) 321, and the mask feature map can be obtained according to the following formula (3):

[0048] (3)

[0049] The mask feature map and the first search area feature map are sent to the adjustment factor generator (Adjustment factor Generator) 331, and the scaling factor λ can be obtained, which can represent the detection confidence and scene complexity. The scaling factor is also called the scaling factor.

[0050] Further, the scaling factor λ can be used to determine the scale of the adaptive filter 341, and the following formula (4) is obtained to represent the adaptive filter:

[0051] (4)

[0052] wherein, is the input data before filtering, is the output data after filtering.

[0053] After obtaining the above adaptive dynamic filter, the adaptive filter can be applied to the first search area feature map , and the specific form is to obtain the second search area feature map by the following element multiplication formula (5) to highlight the foreground features.

[0054] (5)

[0055] ​​Further, when the mask feature map and the first search region feature map are input to the adjustment factor generator of the adaptive search model, the mask feature map and the first search region feature map can be spliced to obtain first spliced data; the first spliced data is respectively averaged-pooled from the x direction and the y direction to correspondingly obtain first pooled data and second pooled data; the first pooled data and the second pooled data are spliced to obtain second spliced data; the second spliced data is subjected to convolution processing and exponential operation to obtain the scaling factor.

[0056] Specifically, when the second spliced data is subjected to convolution processing, the second spliced data can sequentially pass through convolution, batch normalization, activation and global pooling operations to obtain first convolution data, and the first convolution data is input to a fully connected layer to obtain second convolution data.

[0057] As shown in Figure 4 , the adjustment factor generator can splice the mask feature map and the first search region feature map . Then, the spliced data is respectively averaged-pooled from the x direction and the y direction to encode features and prior spatial structure information from different directions, and then the averaged-pooled data is spliced again. Then, the data spliced again is subjected to convolution, BN (Batch Normalization), Relu (Rectified Linear Unit) and global pooling operations, so that the information between the features and the prior is interacted. Finally, the data obtained by global pooling is input to a fully connected layer, and the data output by the fully connected layer is subjected to exponential operation to obtain the scaling factor λ.

[0058] In step S104, the second search region and the second search region feature map can be spliced by region channel feature number using a motion model preset in the motion modeling layer to obtain third spliced data; the third spliced data is input to a multi-layer perceptron to obtain the predicted frame. Specifically, as shown in Figure 5 , in the motion modeling layer, is input to a third splicing layer 501 by feature channel number for splicing, and then a target frame prediction value is obtained through a multi-layer perceptron 502. The multi-layer perceptron is MLP (Multilayer Perceptron).

[0059] In step S105, it can be determined that the prediction frame matches the current detection frame in the detection frame set when the intersection over union of the prediction frame and the current detection frame is greater than or equal to a set first value. Specifically, the first value can be set to 0.5, and is not limited thereto.

[0060] In the embodiments of the present application, the existing yolox model can be directly used. The adaptive search model needs to be trained after selecting the initial model framework. In the training process of the adaptive search model, the training data set is used to train the whole composed of each part of the convolutional neural network of the adaptive search model, and the adaptive search model when the training result converges is obtained.

[0061] According to the multi-target tracking method of the embodiments of the present disclosure, the corresponding first search region is determined by the position of the first tracking frame in the first scene picture, and the second search region is determined according to the first search region and the preset adaptive search model, and the target motion is modeled according to the second search region, so as to obtain the trajectory of the target according to the modeling result and the detection frame in the detection frame set. The scheme can dynamically adjust the search region according to the detection and tracking priori, thereby improving the accuracy of object association and obtaining a higher-precision target tracking result.

[0062] The following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiments of the present disclosure. The multi-target tracking device described below can be referred to in correspondence with the multi-target tracking method described above. For details not disclosed in the device embodiments of the present disclosure, please refer to the method embodiments of the present disclosure.

[0063] Figure 6 is a schematic diagram of a multi-target tracking device provided by an embodiment of the present disclosure. As shown in Figure 6 , the multi-target tracking device comprises:

[0064] The target detection unit 601 is configured to perform target detection on a first scene picture collected at a first time and a second scene picture collected at a second time, respectively, to obtain a tracking frame set at the first time, a first feature map of the first scene picture, a detection frame set at the second time, and a second feature map of the second scene picture.

[0065] Specifically, a first scene picture collected at a first time t can be input into a target detector to obtain a tracking frame set for tracking a target at the first time t , and a second scene picture collected at a second time t+δ can be input into the target detector to obtain a detection frame set at the second time t+δ . The backbone network of the yolox model or other target detector can perform feature extraction on the first scene picture and the second scene picture, respectively, to obtain the corresponding first feature map and the second feature map .

[0066] The preliminary search unit 602 is configured to determine a first search region corresponding to a first tracking box in the second scene picture according to a position of the first tracking box in the first scene picture in the set of tracking boxes, to obtain a first search region feature map of the first search region in the second feature map.

[0067] Specifically, the preliminary search unit 602 is configured to preliminarily search, to obtain the first search region, a process of setting a corresponding first search region in the second scene picture according to a position of the first tracking box in the first scene picture in the set of tracking boxes, and obtaining a first search region feature map of the first search region. wherein i is an index of a target in the set of tracking boxes, i = 1, 2, 3, …, n, and n is a number of tracking boxes in the set of tracking boxes.

[0068] The adaptive search unit 603 is configured to input the first tracking box, a first detection box in the set of detection boxes, and the first search region feature map into a preset adaptive search model to obtain a second search region, and map position information of the second search region to the first feature map to obtain a second search region feature map.

[0069] Specifically, the adaptive search model can be an adaptive search region module. The adaptive search unit 603 inputs the first tracking box at the t time, the set of detection boxes, and the first detection box feature map into the adaptive search region module to obtain a refined second search region. The position information of the refined second search region is mapped to the first feature map to obtain the second search region feature map. .

[0070] The motion modeling unit 604 is configured to model a target motion according to the second search region and the second search region feature map to obtain a prediction box.

[0071] Specifically, the motion modeling unit 604 inputs the second search region and the second search region feature map into a motion modeling layer to model the target motion, and obtains the prediction box. .

[0072] ​​​​​​​The matching unit 605 is configured to match the prediction box and the detection box in the detection box set to obtain a track of a target corresponding to the first tracking box.

[0073] Specifically, the matching unit 605 can input the prediction box and the detection box in the detection box set into a matching layer, match the prediction box set and the detection box in the detection box set and the detection box in the detection box set at the second time t+delta to form an association, and obtain the track of the target corresponding to the first tracking box wherein i is an index of the target in the tracking box set, i=1, 2, 3,..., n, and n is the number of tracking boxes in the tracking box set.

[0074] According to the multi-target tracking scheme provided in the embodiments of the present disclosure, an adaptive search area module can be provided, a second search area is dynamically adjusted by simultaneously considering detection and tracking priors to generate a scaling factor with adaptivity, and a motion modeling method is further provided to improve tracking accuracy.

[0075] In the embodiments of the present disclosure, the preliminary searching unit 602 can determine a corresponding position in the second scene picture according to the position of the first tracking box in the first scene picture, and expand the coordinate region where the corresponding position is located to the outside by 2 times the size to obtain the first search area.

[0076] In the embodiments of the present disclosure, expanding the coordinate region to the outside by 2 times the size is only an exemplary description of expanding the coordinate region to the outside by a certain size, and in actual applications, it is not limited to 2 times the size. Expanding the coordinate region to the outside can improve the accuracy of obtaining the prediction box set and the target tracking accuracy.

[0077] In the embodiments of the present disclosure, the adaptive searching unit 603 can input the first tracking box and the first search area feature map into a first box filter of the adaptive search model to obtain a first filtered feature map; input the first detection box and the first search area feature map into a second box filter of the adaptive search model to obtain a second filtered feature map; input the first tracking box, the detection box set and the first search area feature map into a mask generator of the adaptive search model to obtain a mask feature map; input the mask feature map and the first search area feature map into an adjustment factor generator of the adaptive search model to obtain a scaling factor; and use the adaptive filter of the adaptive search model with the scaling factor and a filtered feature value as input data to perform adaptive filtering on the first search area feature map to obtain the second search area, wherein the filtered feature value is a maximum value selected from the first filtered feature map and the second filtered feature map.

[0078] Further, when the mask feature map and the first search region feature map are input to the adjustment factor generator of the adaptive search model, the mask feature map and the first search region feature map can be spliced to obtain first spliced data; the first spliced data is respectively averaged-pooled from the x direction and the y direction to correspondingly obtain first pooled data and second pooled data; the first pooled data and the second pooled data are spliced to obtain second spliced data; the second spliced data is subjected to convolution processing and exponential operation to obtain the scaling factor.

[0079] Specifically, when the second spliced data is subjected to convolution processing, the second spliced data can sequentially undergo convolution, batch normalization, activation, and global pooling operations to obtain first convolution data, and the first convolution data is input to a fully connected layer to obtain second convolution data.

[0080] In the embodiment of the present disclosure, the motion modeling unit 604 can use a motion model preset in the motion modeling layer to splice the second search region and the second search region feature map by region channel feature numbers to obtain third spliced data; and input the third spliced data to a multi-layer perception to obtain the predicted box. Specifically, as shown in Figure 5 The third spliced data is input to a third splicing layer 501 by feature channel numbers, and then passes through a multi-layer perception 502 to obtain a predicted value of the target box. The multi-layer perception is MLP (Multilayer Perceptron).

[0081] In the embodiment of the present disclosure, the matching unit 605 can determine that the predicted box matches a current detection box in the detection box set when the intersection over union of the predicted box and the current detection box is greater than or equal to a set first value. Specifically, the first value can be set to 0.5, and is not limited thereto.

[0082] Since the various functional modules of the multi-target tracking device of the example embodiment of the present disclosure correspond to the steps of the example embodiments of the multi-target tracking method described above, for details not disclosed in the device embodiments of the present disclosure, please refer to the above-mentioned embodiments of the multi-target tracking method of the present disclosure.

[0083] According to the multi-target tracking device of the embodiment of the present disclosure, the corresponding first search region is determined through the position of the first tracking box in the first scene picture, the second search region is determined according to the first search region and the preset adaptive search model, the target motion is modeled according to the second search region, and the trajectory of the target is obtained according to the modeling result and the detection box in the detection box set. The scheme can dynamically adjust the search region according to the detection and tracking prior, thereby improving the accuracy of object association and obtaining a target tracking result with higher precision.​

[0084] Figure 7 is a schematic diagram of an electronic device 7. As shown in the figure, the electronic device 7 of this embodiment includes a processor 701, a memory 702, and a computer program 703 stored in the memory 702 and executable on the processor 701. The processor 701 implements the steps in each of the above method embodiments when executing the computer program 703. Alternatively, the processor 701 implements the functions of each module in each of the above device embodiments when executing the computer program 703. Figure 7

[0085] The electronic device 7 can be a desktop computer, a notebook computer, a palm computer, a cloud server, or the like. The electronic device 7 can include but is not limited to the processor 701 and the memory 702. Those skilled in the art can understand that the electronic device 7 can include more or fewer components than those shown in the figure, or different components. Figure 7 The electronic device 7 shown in the figure is merely an example and does not constitute a limitation on the electronic device 7, which can include more or fewer components than those shown in the figure, or different components.

[0086] The processor 701 can be a central processing unit (CPU), or other general purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc.

[0087] The memory 702 can be an internal storage unit of the electronic device 7, such as a hard disk or a memory of the electronic device 7. The memory 702 can also be an external storage device of the electronic device 7, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. The memory 702 can also include both the internal storage unit and the external storage device of the electronic device 7. The memory 702 is used to store computer programs and other programs and data required by the electronic device.

[0088] ​Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0089] The integrated module, if realized in the form of software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be instructed by a computer program to related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can be executed by a processor to realize the steps of each method embodiment described above. The computer program can include computer program code, which can be in the form of source code, object code, executable file or some intermediate form. The computer readable medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction, for example, in some jurisdictions, according to legislation and patent practice, the computer readable medium does not include electric carrier wave signal and telecommunication signal.

[0090] The above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure.

Claims

1. A multi-target tracking method, characterized in that, The method includes: Target detection is performed on the first scene image acquired at the first moment and the second scene image acquired at the second moment, respectively, to obtain the tracking box set at the first moment, the first feature map of the first scene image, the detection box set at the second moment, and the second feature map of the second scene image; Based on the position of the first tracking box in the tracking box set in the first scene image, the first search region corresponding to the first tracking box is determined in the second scene image, and the first search region feature map of the first search region in the second feature map is obtained. The first tracking box, the first detection box in the detection box set, and the first search region feature map are input into a preset adaptive search model to obtain a second search region, and the position information of the second search region is mapped to the first feature map to obtain a second search region feature map. The target motion is modeled based on the second search region and the feature map of the second search region to obtain the prediction box; The predicted bounding box and the detection boxes in the set of detection boxes are matched to obtain the trajectory of the target corresponding to the first tracking box; The first tracking box, the first detection box in the detection box set, and the feature map of the first search region are input into a preset adaptive search model, including: The first tracking box and the feature map of the first search region are input into the first box filter of the adaptive search model to obtain the first filtered feature map; The first detection box and the feature map of the first search region are input into the second box filter of the adaptive search model to obtain the second filtered feature map; The first tracking box, the first detection box in the detection box set, and the feature map of the first search region are input into the mask generator of the adaptive search model to obtain the mask feature map. The mask feature map and the first search region feature map are input into the adjustment factor generator of the adaptive search model to obtain the scaling factor; An adaptive filter of the adaptive search model, which uses the scaling factor and the filter feature value as input data, is used to adaptively filter the feature map of the first search region to obtain the second search region, wherein the filter feature value is the maximum value selected in the first filter feature map and the second filter feature map.

2. The method according to claim 1, characterized in that, The mask feature map and the first search region feature map are input into the adjustment factor generator of the adaptive search model, including: The mask feature map and the first search region feature map are concatenated to obtain the first concatenated data; The first concatenated data is averaged in both the x and y directions to obtain the first pooled data and the second pooled data. The first pooled data and the second pooled data are concatenated to obtain the second concatenated data; The scaling factor is obtained by performing convolution and exponential operations on the second concatenated data.

3. The method according to claim 2, characterized in that, The second concatenated data is subjected to convolution processing, including: The second concatenated data is sequentially processed through convolution, batch normalization, activation, and global pooling to obtain the first convolutional data; The first convolutional data is input into a fully connected layer to obtain the second convolutional data.

4. The method according to claim 1, characterized in that, Target detection is performed on the first scene image captured at the first moment and the second scene image captured at the second moment, including: The first scene image and the second scene image are input into the YOLOX object detector model for object detection.

5. The method according to claim 1, characterized in that, Modeling the target motion based on the second search region and its feature map includes: The second search region and its feature map are concatenated according to the number of region channel features to obtain the third concatenated data; The third stitched data is input into a multilayer perceptron to obtain the prediction box.

6. The method according to any one of claims 1 to 5, characterized in that, Matching the predicted bounding box with the detection bounding boxes in the set of detection boxes includes: When the intersection-union ratio (IUU) between the predicted bounding box and the current detection box in the set of detection boxes is greater than or equal to a set first value, it is determined that the predicted bounding box matches the current detection box.

7. A multi-target tracking device, characterized in that, The device includes: The target detection unit is used to perform target detection on the first scene image acquired at the first time and the second scene image acquired at the second time, respectively, to obtain the tracking box set at the first time, the first feature map of the first scene image, the detection box set at the second time, and the second feature map of the second scene image. The preliminary search unit is used to determine the first search region corresponding to the first tracking box in the second scene image based on the position of the first tracking box in the first scene image, and to obtain the first search region feature map of the first search region in the second feature map. An adaptive search unit is used to input the first tracking box, the first detection box in the detection box set, and the first search region feature map into a preset adaptive search model to obtain a second search region, and to map the position information of the second search region onto the first feature map to obtain a second search region feature map. The motion modeling unit is used to model the motion of the target based on the second search region and the feature map of the second search region to obtain a prediction box; A matching unit is used to match the predicted box with the detection boxes in the detection box set to obtain the trajectory of the target corresponding to the first tracking box; The adaptive search unit is specifically used to: input the first tracking box and the feature map of the first search region into the first box filter of the adaptive search model to obtain the first filtered feature map; The first detection box and the first search region feature map are input into the second box filter of the adaptive search model to obtain a second filtered feature map; the first tracking box, the first detection box in the detection box set, and the first search region feature map are input into the mask generator of the adaptive search model to obtain a mask feature map; the mask feature map and the first search region feature map are input into the adjustment factor generator of the adaptive search model to obtain a scaling factor; the first search region feature map is adaptively filtered using the adaptive filter of the adaptive search model, which uses the scaling factor and the filtering feature value as input data, to obtain the second search region, wherein the filtering feature value is the maximum value selected from the first filtered feature map and the second filtered feature map.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images