Method and electronic device for tracking multiple objects
A neural network model with a transformer, ReID, and motion estimation modules addresses the challenge of tracking multiple objects with varying motions and appearances, improving tracking accuracy in scenarios with limited frame overlap, particularly in autonomous driving systems.
Patent Information
- Application Number
- JP2024205046
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-02-15
- Filing Date
- 2024-11-25
- Publication Date
- 2025-08-27
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing object tracking technologies face challenges in accurately tracking multiple objects with different motions and/or similar appearances, particularly in scenarios with limited overlapping areas between frames.
A neural network model comprising a transformer, ReID embedding module, motion estimation module, and matching module is employed to track multiple objects by utilizing both object appearance and motion, with the transformer deriving feature matrices and the ReID module extracting external features, while the motion estimation module estimates object positions, and the matching module calculates similarities to associate detected and tracked objects.
The method enables accurate tracking of multiple objects with diverse motions and appearances, even in scenarios with minimal frame overlap, enhancing tracking performance in applications like autonomous driving systems.
Smart Images
Figure 2025125502000001_ABST
Abstract
Description
[Technical Field]
[0001] The following disclosure relates to a method for tracking multiple objects and an electronic device for doing so. [Background technology]
[0002] Computer vision object tracking lies at the intersection of image processing, pattern recognition, and machine learning. Object tracking has the goal of automatically tracking and monitoring the movement of objects in video, and is used in a variety of fields, including security, robotics, autonomous vehicles, and augmented reality.
[0003] The main aspects of object tracking technology include object detection, motion analysis, feature extraction, combination algorithms, multiple object tracking, deep learning, and sensor fusion. Object tracking plays an important role in various industrial fields as it continues to evolve with technological advancements such as deep learning and sensor fusion.
[0004] The preceding information may be provided as related art to aid in understanding the present disclosure. No assertion or determination is being made as to whether any of the preceding content is prior art with respect to the present disclosure. Summary of the Invention [Problem to be solved by the invention]
[0005] SUMMARY OF THE INVENTION It is an object of the present invention to provide a method for tracking multiple objects and an electronic device for doing so. [Means for solving the problem]
[0006] According to one embodiment, a method for operating an electronic device includes detecting at least one object in a current frame, utilizing a neural network model to obtain a matching result between the detected object and the tracked object based on information about the detected object, information about at least one object tracked in a previous frame, frame features of the previous frame, and object recognition frame features of the previous frame, and obtaining trajectory information of the detected object up to the current frame based on the matching result.
[0007] According to one embodiment, the neural network model may include a transformer that outputs the current object recognition frame features based on the frame features and the previous object recognition frame features. The neural network model may include a ReID embedding module that outputs Re-identification (ReID) features for the at least one detected object based on the current object recognition frame features and the previous object recognition frame features. The neural network model may include a motion estimation module that outputs a heat map containing information about the estimated position of each of the at least one tracked object at the current time based on the current object recognition frame features and the previous object recognition frame features. The neural network model may include a matching module that matches the at least one detected object with the at least one tracked object based on information about the at least one tracked object, information about the at least one detected object, the ReID features, and the heat map.
[0008] According to one embodiment, the Transformer may obtain a first feature matrix based on the previous object recognition frame features, obtain a second feature matrix based on the frame features, and output the current object recognition frame features based on the first feature matrix, the second feature matrix, and the frame features.
[0009] According to one embodiment, the Transformer may derive first object features by performing a region of interest (ROI) alignment for the at least one tracked object on the previous object recognition frame features, and the Transformer may derive the first feature matrix based on the first object features.
[0010] According to one embodiment, the Transformer may obtain fused object features from the first feature matrix and the second feature matrix based on a cross-attention layer, and may output the current object recognition frame features based on the fused object features and the frame features.
[0011] According to one embodiment, the ReID embedding module may obtain a first feature matrix based on the previous object recognition frame features, obtain a third feature matrix based on the current object recognition frame features, and output the ReID features of the at least one detected object based on the first and third feature matrices.
[0012] According to one embodiment, the ReID embedding module may derive second object features by performing a region of interest alignment (ROI Align) on the detected at least one object on the current object recognition frame features, and the ReID embedding module may derive the third feature matrix based on the second object features.
[0013] According to an embodiment, the motion estimation module may acquire third object features based on the previous object recognition frame features, acquire fourth object features based on the current object recognition frame features, and output the heat map based on the third object features and the fourth object features.
[0014] According to one embodiment, the motion estimation module may obtain the fourth object feature by performing a region of interest alignment (ROI Align) on a search region on the current object recognition frame feature, the search region being a scaled bounding box of the at least one detected object.
[0015] According to one embodiment, the matching module may calculate a first similarity between the at least one tracked object and the at least one detected object based on the ReID features. The matching module may calculate a second similarity between the at least one tracked object and the at least one detected object based on information about the current estimated location of each of the at least one tracked object included in the heat map. The matching module may output the matching result based on a weighted sum of the first similarity and the second similarity.
[0016] According to one embodiment, the first similarity is based on a two-way softmax similarity and a cosine similarity between the current ReID feature and the previous ReID feature.
[0017] According to an embodiment, an electronic device may include a processor and a memory storing instructions. When the instructions are executed by the processor, the electronic device may detect at least one object in a current frame. When the instructions are executed by the processor, the electronic device may utilize a neural network model to obtain a matching result between the detected at least one object and the tracked at least one object based on information about the detected at least one object, information about at least one object tracked in a frame prior to the current frame, frame features of the frame, and object-recognition frame features of the previous frame. When the instructions are executed by the processor, the electronic device may obtain trajectory information of the detected at least one object up to the current frame based on the matching result. [Effects of the Invention]
[0018] According to the present invention, a method for tracking multiple objects and an electronic device for doing so can be provided. [Brief explanation of the drawings]
[0019] [Figure 1] 1 shows a schematic block diagram of an electronic device according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a neural network according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating the operation of a transformer according to an embodiment. [Figure 4]FIG. 2 is a diagram for explaining the operation of a ReID (Re-identification) embedding module according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating the operation of a motion estimation module according to an embodiment. [Figure 6] FIG. 1 is a diagram illustrating multiple object tracking in an autonomous driving system according to an embodiment. [Figure 7] 1 shows a flowchart of a method of operating an electronic device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0020] Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only and may be modified in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of the present specification includes modifications, equivalents, or alternatives within the technical spirit.
[0021] Although terms such as "first" or "second" may be used to describe multiple components, such terms should be construed only to distinguish one component from the other components. For example, a first component may be designated as a second component, and similarly, a second component may be designated as a first component.
[0022] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that although it is directly coupled or connected to the other component, there may be other components in between.
[0023] The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, phrases such as "at least one of A or B" and "at least one of A, B, or C" may each include any one or all possible combinations of the items listed with the phrase. In this specification, the terms "comprise" or "have" and the like indicate the presence of a feature, numeral, step, operation, element, part, or combination thereof described in the specification, and should be understood as not precluding the possibility of the presence or addition of one or more other features, numerals, steps, operations, elements, parts, or combinations thereof.
[0024] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the present invention belongs. Commonly used predefined terms should be interpreted as having a meaning that matches the meaning they have in the context of the relevant art, and should not be interpreted as having an ideal or overly formal meaning unless expressly defined in this specification.
[0025] The term "module" as used herein may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrated component, or the smallest unit or portion of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).
[0026] The term "module" as used in this document refers to software or hardware components such as FPGAs or ASICs, and a "module" may perform either function. However, a "module" is not limited to software or hardware. A "module" may be configured to reside on an addressable storage medium or to implement one or more processors. For example, a "module" may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within a component and a "module" may be combined into fewer components and "modules" or further separated into additional components and "modules." Furthermore, a component and a "module" may be implemented to implement one or more CPUs within a device or a security multimedia card. A "module" may also include one or more processors.
[0027] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals will be used to designate the same elements regardless of the reference numerals, and redundant description thereof will be omitted.
[0028] FIG. 1 shows a schematic block diagram of an electronic device according to one embodiment.
[0029] According to one embodiment, the electronic device 100 can track multiple objects. The electronic device 100 may utilize a neural network for multiple object tracking. The neural network model utilized by the electronic device 100 can utilize both object appearance and object motion. The electronic device 100 can accurately track multiple objects with different motions and / or similar appearances.
[0030] The neural network utilized by the electronic device 100 refers to a model in which artificial neurons (nodes) that form a network through synaptic connections change the strength of synaptic connections through learning, thereby having problem-solving capabilities.
[0031] Neurons in a neural network may contain a combination of weights or biases. A neural network may contain one or more layers, each made up of one or more neurons or nodes. By changing the weights of the neurons through learning, a neural network can infer the outcome it is trying to predict from any input.
[0032] The neural network may include a deep neural network. Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Automatic Encoder), VAE (Variational Automatic Encoder), DAE (Denoising Automatic Encoder), SAE (Sparse Automatic Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), and GAN (Generative Adversarial Network). The neural network may include a neural network (NNN), a liquid state machine (LSM), an extreme learning machine (ELM), an echo state network (ESN), a deep residual network (DRN), a differentiable neural computer (DNC), a neural turning machine (NTM), a capsule network (CN), a Kohonen network (KN), and an attention network (AN). The structure of the neural network utilized by the electronic device 100 will be described in detail with reference to FIGS. 2 to 5.
[0033] The electronic device 100 may be implemented in a personal computer (PC), a data server, or a handheld device.
[0034] The portable device may be embodied in a laptop computer, a mobile phone, a smartphone, a tablet PC, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital still camera, a digital video camera, a portable multimedia player (PMP), a personal or portable navigation device (PND), a handheld game console, an e-book, or a smart device, which may be embodied in a smart watch, a smart band, or a smart ring.
[0035] Referring to FIG. 1, an electronic device 100 includes a processor 110 and a memory 120 .
[0036] The processor 110 can process data stored in the memory 120. The processor 110 can execute computer-readable code (e.g., software) and instructions stored in the memory 120.
[0037] The processor 110 may be a data processing device implemented in hardware having circuits with physical structures for performing desired operations, and may perform the desired operations by executing code or instructions included in a program.
[0038] For example, a data processing device implemented in hardware may include a microprocessor, a central processing unit, a processor core, a multi-core processor, a multiprocessor, an ASIC (Application-Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).
[0039] The memory 120 may be implemented in a volatile memory device or a non-volatile memory device.
[0040] The volatile memory device may be implemented as dynamic random access memory (DRAM), static random access memory (SRAM), thyristor RAM (T-RAM), zero capacitor RAM (Z-RAM), or twin transistor RAM (TTRAM).
[0041] The nonvolatile memory device may be realized as an EEPROM (Electrically Erasable Programmable Read-Only Memory), a flash memory, an MRAM (Magnetic RAM), a Spin-Transfer Torque (STT)-MRAM, a Conductive Bridging RAM (CBRAM), an FeRAM (Ferroelectric RAM), a PRAM (Phase change RAM), a Resistive RAM (RRAM), a Nanotube RRAM, a Polymer RAM (PoRAM), a Nano Floating Gate Memory (NFGM), a holographic memory, a Molecular Electronic Memory Device, or an Insulator Resistance Change Memory.
[0042] The processor 110 may detect an object in the current frame (eg, image frame).
[0043] The processor 110 can utilize a neural network model to obtain a matching result between the detected object and the tracked object based on information about the detected object, information about the object tracked in a frame prior to the current time point, frame features of the frame, and object recognition frame features of the previous time point.
[0044] The processor 110 obtains trajectory information of the detected object up to the current time point based on the matching result.
[0045] FIG. 2 is a diagram illustrating a neural network according to an embodiment.
[0046] Referring to FIG. 2, according to one embodiment, a neural network model 10 utilized by an electronic device (eg, electronic device 100 of FIG. 1) can be seen. JPEG2025125502000002.jpg28115
[0047] JPEG2025125502000003.jpg31115
[0048] JPEG2025125502000004.jpg15115
[0049] JPEG2025125502000005.jpg35115
[0050] JPEG2025125502000006.jpg32115
[0051] JPEG2025125502000007.jpg47115
[0052] JPEG2025125502000008.jpg31115
[0053] JPEG2025125502000009.jpg40115
[0054] The neural network model 10 includes a transformer 210, a ReID (Re-identification) embedding module 220, a motion estimation module 230, and a matching module 240.
[0055] The transformer 210 may be a temporal transformer. The structure and operation of JPEG2025125502000010.jpg27115 transformer 210 will be described in more detail with reference to FIG.
[0056] The ReID embedding module 220 extracts the external features of the object. The structure and operation of the JPEG2025125502000011.jpg27115 ReID embedding module 220 will be described in detail with reference to FIG.
[0057] The motion estimation module 230 estimates the motion of the objects and can estimate the current position for each object tracked in the previous frame 202. The structure and operation of the JPEG2025125502000012.jpg28115 motion estimation module 230 will be described in more detail with reference to FIG.
[0058] The matching module 240 associates the external features of the object with the estimated motion of the object. The matching module 240 matches the detected object with the tracked object. JPEG2025125502000013.jpg20115
[0059] JPEG2025125502000014.jpg24115
[0060] FIG. 3 is a diagram illustrating the operation of a transformer according to an embodiment.
[0061] Referring to FIG. 3, the structure of a transformer (eg, transformer 210 of FIG. 2) can be seen according to one embodiment. JPEG2025125502000015.jpg21115
[0062] JPEG2025125502000016.jpg15115
[0063] JPEG2025125502000017.jpg34115
[0064]
number
[0065] JPEG2025125502000019.jpg16115
[0066] JPEG2025125502000020.jpg24115
[0067] JPEG2025125502000021.jpg23115
[0068] JPEG2025125502000022.jpg32115
[0069]
number
[0070] JPEG2025125502000024.jpg27115
[0071]
number
[0072] JPEG2025125502000026.jpg34115
[0073] FIG. 4 is a diagram for explaining the operation of a ReID (Re-identification) embedding module according to an embodiment.
[0074] Referring to FIG. 4, the structure of a ReID (Re-identification) embedding module (eg, ReID embedding module 220 of FIG. 2) can be seen according to one embodiment. JPEG2025125502000027.jpg30115
[0075] JPEG2025125502000028.jpg57115
[0076] JPEG2025125502000029.jpg15115
[0077] JPEG2025125502000030.jpg42115
[0078]
number
[0079] JPEG2025125502000032.jpg19115
[0080] JPEG2025125502000033.jpg35115
[0081] JPEG2025125502000034.jpg42115
[0082] JPEG2025125502000035.jpg20115
[0083] FIG. 5 is a diagram illustrating the operation of the motion estimation module according to one embodiment.
[0084] Referring to FIG. 5, the structure of a motion estimation module (eg, motion estimation module 230 of FIG. 2) can be seen according to one embodiment.
[0085] JPEG2025125502000036.jpg27115
[0086] JPEG2025125502000037.jpg27115
[0087] JPEG2025125502000038.jpg27115The search area is a scaled version of the bounding box of the detected object. For example, the search area may be an expanded version of the bounding box of the detected object based on a scale factor of 3.
[0088] JPEG2025125502000039.jpg20115
[0089]
number
[0090] In equation (5), the subscript k is an index for the object, and the superscript t is an index for the time point. JPEG2025125502000041.jpg37115
[0091] JPEG2025125502000042.jpg26115
[0092] As described above, the motion estimation module 230 can estimate the current position for each object tracked in the previous frame. The motion estimation module 230 can estimate the motion of the object.
[0093] A matching module (e.g., matching module 240 in FIG. 2) associates the external features of the object (e.g., the output of ReID embedding module 220 in FIG. 2) with the estimated motion of the object (e.g., the output of motion estimation module 230 in FIG. 2). The matching module 240 matches the detected object with the tracked object. JPEG2025125502000043.jpg32115
[0094] JPEG2025125502000044.jpg19115
[0095] JPEG2025125502000045.jpg16115
[0096]
number
[0097] JPEG2025125502000047.jpg22115
[0098] JPEG2025125502000048.jpg42115
[0099]
number
[0100] JPEG2025125502000050.jpg23115
[0101] JPEG2025125502000051.jpg21115
[0102]
number
[0103] In equation (8), β is a weight parameter.
[0104] The matching module 240 obtains a matching result between the detected object and the tracked object based on the joint similarity (e.g., S), and matches the detected object and the tracked object with the largest joint similarity as one pair. JPEG2025125502000053.jpg38115If there is no tracked object that matches the detected object (e.g., there is no tracked object whose combined similarity is greater than a threshold), the detected object may be a newly appeared object, in which case it can be managed as trajectory information of the newly appeared object.
[0105] The electronic device 100 can update the trajectory information based on the matching results between the detected object and the tracked object. JPEG2025125502000054.jpg15115
[0106] FIG. 6 is a diagram illustrating multiple object tracking in an autonomous driving system according to an embodiment.
[0107] Referring to FIG. 6, according to one embodiment, exemplary image frames (e.g., 601-603) captured by an autonomous driving system are shown. An electronic device (e.g., electronic device 100 of FIG. 1) may process the image frames 601-603. The electronic device 100 may be embodied as a part of the autonomous driving system or may be attached to (e.g., mounted on) the autonomous driving system. Each of the image frames 601-603 may be data captured by a camera of the autonomous driving system.
[0108] In image frame 601, objects 611 and 612 may be identified, in image frame 602, objects 612 and 613 may be identified, and in image frame 603, objects 613 and 614 may be identified. When referring to the identified objects, the overlapping area between image frames may not be large due to the arrangement of the cameras of the autonomous vehicle. When the overlapping area between image frames is not large, utilizing a ReID embedding module (e.g., 220 in FIG. 2 ) may improve the tracking performance of multiple objects. By utilizing the ReID embedding module, electronic device 100 can perform appropriate multiple object tracking in an autonomous driving system where the overlapping area between image frames is not large. Furthermore, the neural network model utilized by electronic device 100 for multiple object tracking takes into account both object appearance and object motion. Electronic device 100 can accurately track multiple objects with different motions and / or similar appearances.
[0109] FIG. 7 shows a flowchart of a method for operating an electronic device according to one embodiment.
[0110] 7, according to one embodiment, operations 710-730 are performed sequentially, but are not limited to this. For example, two or more operations may be performed in parallel.
[0111] In operation 710, an electronic device (eg, electronic device 100 of FIG. 1) detects at least one object in a current frame.
[0112] In operation 720, the electronic device 100 utilizes a neural network model to obtain a matching result between the at least one detected object and the at least one tracked object based on information about the at least one detected object, information about the at least one tracked object in a frame prior to the current time point, frame features of the frame, and object recognition frame features of the previous time point.
[0113] In operation 730, the electronic device 100 obtains trajectory information up to the current time point of the at least one object detected based on the matching result.
[0114] The above-described embodiments may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or other adaptive supersampling device capable of executing and responding to commands. The processing device may execute an operating system (OS) and software applications that run on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used, but those skilled in the art will recognize that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.
[0115] Software may include computer programs, code, instructions, or any combination thereof, capable of configuring a processing device or instructing the processing device, either individually or collectively, as desired. The software and / or data may be permanently embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device, or transmitted signal wave, to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed over networked computer systems and stored or executed in a distributed manner. The software and data may be stored on a computer-readable recording medium.
[0116] The method according to the present invention may be embodied in the form of program instructions that can be executed by various computer means and recorded on a computer-readable recording medium. The recording medium may include program instructions, data files, data structures, and the like, alone or in combination. The recording medium and program instructions may be specially designed and constructed for the purposes of the present invention, or may be well-known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code that is executed by a computer using an interpreter, for example.
[0117] The hardware adaptive supersampling devices described above may be configured to operate as one or more software models to perform the operations shown in this invention, and vice versa.
[0118] Although the embodiments have been described above with reference to limited drawings, those skilled in the art may apply various technical modifications and variations based on the above description. For example, the described techniques may be performed in a different order than described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than described, and may be replaced or substituted with other components or equivalents, while still achieving suitable results.
[0119] Accordingly, other implementations, other embodiments, and equivalents of the claims are intended to fall within the scope of the following claims.
Claims
1. 1. A method of operating an electronic device, comprising: detecting at least one object in a current frame; obtaining a matching result between the at least one detected object and the at least one tracked object based on information about the at least one detected object, information about at least one tracked object in a frame prior to the current time point, frame features of the frame, and object recognition frame features of the previous time point using a neural network model; an operation of acquiring trajectory information of the at least one detected object up to the current time point based on the matching result; A method of operating an electronic device, comprising:
2. The neural network model is a transformer that outputs the object recognition frame feature of the current time point based on the frame feature and the object recognition frame feature of the previous time point; a Re-identification (ReID) embedding module that outputs a ReID feature of the at least one detected object based on the current object recognition frame feature and the previous object recognition frame feature; a motion estimation module that outputs a heat map containing information regarding an estimated position of each of the at least one tracked object based on the current object-recognition frame features and the previous object-recognition frame features; a matching module that matches the at least one detected object with the at least one tracked object based on information about the at least one tracked object, information about the at least one detected object, the ReID features, and the heat map; 10. The method of claim 1, comprising:
3. The transformer is Obtain a first feature matrix based on the object recognition frame features of the previous time point; Obtaining a second feature matrix based on the frame features; The method of claim 2 , further comprising: outputting the current object recognition frame feature based on the first feature matrix, the second feature matrix, and the frame feature.
4. The transformer obtains first object features by performing a region of interest alignment (ROI Align) for the tracked at least one object on the previous object recognition frame features; The method of claim 3 , further comprising obtaining the first feature matrix based on the first object feature.
5. The transformer is Obtaining fused object features from the first feature matrix and the second feature matrix based on a cross-attention layer; The method of claim 3 , further comprising: outputting the current object recognition frame feature based on the fused object feature and the frame feature.
6. The ReID embedding module Obtain a first feature matrix based on the object recognition frame features of the previous time point; Obtaining a third feature matrix based on the current object recognition frame features; The method of claim 2 , further comprising: outputting the ReID feature of the detected at least one object based on the first feature matrix and the third feature matrix.
7. The ReID embedding module obtaining second object features by performing a region of interest alignment (ROI Align) on the detected at least one object on the current object recognition frame features; The method of claim 6 , further comprising obtaining the third feature matrix based on the second object features.
8. The motion estimation module: obtaining a third object feature based on the object recognition frame feature of the previous time point; obtaining a fourth object feature based on the current object recognition frame feature; The method of claim 2 , further comprising outputting the heat map based on the third object feature and the fourth object feature.
9. the motion estimation module obtains the fourth object feature by performing a region of interest alignment (ROI Align) on a search region on the current object recognition frame feature; The method of claim 8 , wherein the search area is a scaled bounding box of the at least one detected object.
10. The matching module: calculating a first similarity between the at least one tracked object and the at least one detected object based on the ReID features; calculating a second similarity between the at least one tracked object and the at least one detected object based on information about the current estimated location of each of the at least one tracked object included in the heat map; The method of claim 2 , further comprising outputting the matching result based on a weighted sum of the first similarity and the second similarity.
11. The method of claim 10 , wherein the first similarity is based on a two-way softmax similarity and a cosine similarity between the ReID feature at the current time point and the ReID feature at the previous time point.
12. A computer program stored on a computer-readable recording medium for performing the method according to any one of claims 1 to 11 in combination with hardware.
13. 1. An electronic device comprising: a processor; a memory for storing instructions; Including, The instructions, when executed by the processor, cause the electronic device to: Detecting at least one object in the current frame; Utilizing a neural network model, obtain a matching result between the at least one detected object and the at least one tracked object based on information about the at least one detected object, information about at least one tracked object in a frame prior to the current time point, frame features of the frame, and object recognition frame features of the previous time point; The electronic device is configured to obtain trajectory information of the detected at least one object up to the current time point based on the matching result.