Method, computing apparatus and computer readable recording medium for tracking same object in video

Abstract

A method for tracking identical objects within an image performed on a computing device of the present invention may include: detecting an object within an image and generating object detection data corresponding to the detected object; tracking each object within the image based on an object tracking algorithm and generating tracklet data for the movement path of the tracked object; and grouping the tracklet data of objects determined to be identical objects to map a unique identification code. The present invention identifies each identical object by assigning a unique identification code, thereby allowing the user to quickly search for tracked images using the unique identification code and verify the movement path of a specific object.

Description

Technology Field

[0001] The present invention relates to a technology for tracking connections of identical objects within an image. Background Technology

[0002] Recent CCTV technology is evolving into intelligent CCTV systems based on artificial intelligence learning.

[0003] Intelligent CCTV systems analyze specific behaviors in video footage and detect abnormal behavior, providing the ability to automatically identify and process specific situations such as loitering and intrusion.

[0004] In such intelligent CCTV systems, it is important to accurately recognize humans or objects and accurately identify the situation.

[0005] While current intelligent CCTV systems are expanding into various video processing areas such as crowd density analysis and fire detection, the technology to accurately identify and track the same subject within video footage remains at the laboratory level.

[0006] In particular, conventional identical object tracking technology has a problem in that it cannot properly perform identical object tracking when the object moves frequently, when the object's appearance within the video is deformed (e.g., when wearing a hat or taking off a coat), or when only a part of the object is visible because it is obscured by other objects.

[0007] As a result, there are limitations in completing the overall movement path by connecting the same subject within the video. The problem to be solved

[0008] The present invention aims to solve the above-mentioned problems, and the objective of the present invention is to provide the user with information and movement paths of specific subjects by more accurately implementing the connection of the same subject within the image. means of solving the problem

[0009] A method for tracking identical objects within an image performed on a computing device according to the present invention to solve the above technical problem may include the steps of detecting an object within an image and generating object detection data corresponding to the detected object; tracking each object within the image based on an object tracking algorithm and generating tracklet data for the movement path of the tracked object; and grouping the tracklet data of objects determined to be identical objects to map a unique identification code.

[0010] In addition, the object detection data is generated on a frame-by-frame basis within the video, and the object detection data may include physical feature information and characteristic text of the object.

[0011] In addition, the object tracking algorithm tracks an object based on the bounding box of the object within the image, and when a new object is detected within the image, it can determine whether the newly detected object is the same object by comparing the feature information of the previously detected object with that of the previously detected object.

[0012] In addition, the step of generating the tracklet data can map object detection data corresponding to the tracked object to the tracklet data.

[0013] Additionally, the step of generating the tracklet data may further include the step of generating a 3D face model by 3D modeling the face of the object and the step of calculating face angle information by comparing the generated 3D face model with the face of the object.

[0014] In addition, the step of generating the tracklet data can map the face angle information of the object to the tracklet data.

[0015] In addition, the step of calculating the face angle information may rotate the 3D face model to a specific angle and compare the 3D face model rotated to a reference angle with the face of the object.

[0016] In addition, the specific angle mentioned above may be an angle at which the 3D face model looks in the reference direction.

[0017] Additionally, the step of mapping the unique identification code may further include the step of classifying the tracklet data based on the characteristic text and the step of measuring the similarity between each tracklet data using face angles within the tracklet data classified by the characteristic text.

[0018] In addition, the step of matching the unique identification code may determine that the same object is the same and assign the same unique identification number if the similarity is greater than or equal to a reference value.

[0019] In addition, the step of measuring the similarity can be performed by modeling an object within each tracklet data to create a 3D face model, rotating it using each face angle information, generating a face image of each object through 3D projection, and comparing the face feature information of the generated face images with each other to measure the similarity.

[0020] Additionally, when a user's object tracking request is received, the method may further include a step of providing the user with a tracking result for the same object connection using tracklet data within a unique identification number corresponding to the object to be tracked.

[0021] Meanwhile, the computing device includes a processor, a memory for loading a computer program executed by the processor, and a storage for storing the computer program, wherein the computer program may include an operation of detecting an object in an image and generating object detection data corresponding to the detected object, an operation of tracking each object in the image based on an object tracking algorithm and generating tracklet data for the movement path of the tracked object, and an operation of grouping tracklet data of objects determined to be the same object and mapping a unique identification code.

[0022] In addition, the object detection data is generated on a frame-by-frame basis within the video, and the object detection data may include physical feature information and characteristic text of the object.

[0023] In addition, the object tracking algorithm tracks an object based on the bounding box of the object within the image, and when a new object is detected within the image, it can determine whether the newly detected object is the same object by comparing the feature information of the previously detected object with that of the previously detected object.

[0024] And, the operation of generating the tracklet data further includes the operation of generating a 3D face model by 3D modeling the face of the object and the operation of calculating face angle information by comparing the generated 3D face model with the face of the object, and the operation of generating the tracklet data can map the face angle information of the object to the tracklet data.

[0025] Meanwhile, a computer-readable recording medium according to one embodiment of the present invention for achieving the above-described purpose may store a program that performs the above-described identical object tracking method.

[0026] In addition, a computer program stored on a computer-readable recording medium according to one embodiment of the present invention for achieving the above-described purpose may include program code for executing the above-described identical object tracking method. Effects of the invention

[0027] The present invention can perform accurate object identification and tracking by using a deep learning model to detect objects in each image and generating object detection data corresponding to the objects.

[0028] In addition, the present invention can provide connection tracking results by performing object search using only the unique identification codes after mapping unique identification codes to tracklet data.

[0029] In addition, the present invention can identify and track the same object more accurately by considering the face angle of the object.

[0030] According to the present invention, accuracy in tracking specific individuals such as missing children, elderly people with dementia, and suspects can be increased, and the time and human resources required for such tracking can be effectively reduced. Brief explanation of the drawing

[0031] FIG. 1 is a conceptual diagram illustrating a method for tracking identical objects according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a method for tracking identical objects according to an embodiment of the present invention. FIGS. 3 and 4 are exemplary diagrams for explaining object detection data for tracking the same object according to an embodiment of the present invention. FIG. 5 is a diagram showing the format of object detection data according to one embodiment of the present invention. FIG. 6 is a drawing showing an improved SORT according to one embodiment of the present invention. FIG. 7 is a diagram showing the format of tracklet data according to one embodiment of the present invention. FIG. 8 is a diagram showing the format of the final metadata according to one embodiment of the present invention. FIG. 9 is a timing diagram illustrating a process for tracking the same object according to another embodiment of the present invention. FIG. 10 is a diagram illustrating the hardware implementation of a server as a computing device that performs identical object tracking according to an embodiment of the present invention. Specific details for implementing the invention

[0032] The following description merely illustrates the principles of the invention. Therefore, those skilled in the art may invent various devices that embody the principles of the invention and are included within the concept and scope of the invention, even if they are not explicitly described or illustrated in this specification. Furthermore, all conditional terms and embodiments listed in this specification are, in principle, explicitly intended only for the purpose of understanding the concept of the invention and should be understood not as being limited to the embodiments and conditions specifically listed elsewhere.

[0033] The aforementioned objectives, features, and advantages will become clearer through the following detailed description in conjunction with the attached drawings, and accordingly, a person skilled in the art to which the invention pertains will be able to easily implement the technical concept of the invention.

[0034] In addition, in describing the invention, if it is determined that a detailed description of known technology related to the invention may unnecessarily obscure the essence of the invention, such detailed description will be omitted. Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings.

[0035] FIG. 1 is a conceptual diagram illustrating a method for tracking identical objects according to an embodiment of the present invention.

[0036] Referring to FIG. 1, at least one camera (200) can acquire an image containing an object. Here, the cameras (200) can each capture an object at adjacent locations or at related locations where the object can be tracked, and the camera (200) may refer to an image capturing device such as an intelligent CCTV.

[0037] Additionally, the computing device (300) can receive captured images containing an object from a plurality of cameras (200). Here, the object may preferably be a person, but is not limited thereto, and the object may be any object that is subject to tracking (e.g., an animal, a car, a motorcycle, etc.).

[0038] And, the computing device (300) can recognize and track objects within the received image to generate object-specific tracklet data, and map the same unique identification number to the tracklet data of the same object to generate a single metadata.

[0039] Additionally, the computing device (300) can provide information about the target object to be tracked using the metadata in response to the object tracking request of the user (100).

[0040] Such a computing device (300) may be composed of hardware including a computing device and software including a module capable of generating and storing tracklet data and detecting the same object, and may also exist in the form of an external web server.

[0041] Here, the software configuration within the computing device (300) may consist of an object detection data generation module (320), a tracklet data generation module (340), a unique identification code matching module (360), and an identical object connection tracking module (380), and each module may operate in sequence to track identical objects.

[0042] Hereinafter, the method of tracking the same object performed in the computing device (300) will be explained in more detail with reference to FIG. 2.

[0044] FIG. 2 is a flowchart illustrating a method for tracking identical objects according to an embodiment of the present invention.

[0045] Referring to FIG. 2, a computing device (300) can receive an image from a camera (200) (S100). Here, the image that the computing device (300) can receive may be configured in any one or more of the following forms: AVI (Audio Video Interleave), MP4 (MPEG-4 Part 14), MKV (MatrosKa multimedia container for Video), WMV (Windows Media Video), MOV, FLV (Flash Video), GIF (Graphics Interchange Format), and WEBM, but is not limited thereto.

[0046] Additionally, the type of image received by the computing device (300) may be any one of a real-time captured image, a recorded image, and a non-identified image.

[0047] Additionally, the computing device (300) can extract consecutive frames from an image received from a camera (200), and the extracted frames can be extracted and features analyzed through a distributed processing technique rather than being processed sequentially to reduce the time required for image processing. Here, the distributed processing technique can be determined according to the type of image received.

[0048] For example, if you want to analyze real-time video footage, the computing device (300) can extract images of the video in sequence alternately for distributed processing.

[0049] As another example, if you want to analyze a stored video, the computing device (300) can extract the video in blocks of time.

[0050] Next, the object detection data generation module (320) of the computing device (300) can detect objects within the image and generate object detection data corresponding to each object (S200). Here, object detection data can be generated for each frame.

[0051] Specifically, the object detection data generation module (320) can recognize the location and size of an object using a deep learning model in the extracted frame, and can recognize the location and size of the face of the recognized object. Here, the deep learning model can generate bounding boxes corresponding to the whole or part (e.g., face) of the recognized object, and each bounding box can have a respective detection confidence score. At this time, the deep learning model used for object detection may, for example, utilize a 1-Stage FPN (Feature Pyramid Network) based detection algorithm.

[0052] And, the object detection data generation module (320) can generate recognition data including physical feature information and characteristic text of the object by extracting features of the object from the detected object in a plurality of ways.

[0053] Here, body feature information refers to feature information related to the object's body, and the body feature information may include feature information regarding the object's whole body, upper body, lower body, and face.

[0054] In addition, feature text refers to a text representation of an object's characteristics, and feature text can refer to information expressing various characteristics related to the object's behavior, state, and face in text form.

[0056] A plurality of methods for implementing the above-described S200 step will be explained in more detail below.

[0057] In the first method, the object detection data generation module (320) selects a bounding box based on confidence and can extract feature information about the entire body of the recognized object within the bounding box using a feature extraction model. At this time, the object detection data generation module (320) can use a feature extraction model such as a Convolutional Neural Network (CNN) as an example.

[0058] In the second method, the object detection data generation module (320) detects key points of the recognized object using a pose estimation model as shown in FIG. 3, extracts N*N images at each point based on the reliability of the key points, and arranges them into a single image form considering the extracted positions. Here, the extracted images can be arranged by distinguishing between the upper body or the lower body of the object. Next, the object detection data generation module (320) can extract feature information about the upper body or the lower body of the object from the arranged images using a feature extraction model.

[0059] As a third method, the object detection data generation module (320) may extract coordinate points of specific locations (face landmarks) of parts such as eyes, nose, mouth, and forehead through a face recognition model as shown in FIG. 4, and then encode based on the coordinate points to extract feature information of the face.

[0060] As a fourth method, the object detection data generation module (320) can classify the category of the recognized object using a feature extraction model and extract feature text that expresses the features of the object in text form.

[0061] For example, when there is an image of a man wearing glasses and black shoes standing in front of a chair with his hands in his pockets and smiling, the characteristic text can be extracted in the form of keywords such as “glasses,” “black shoes,” “man,” “chair,” “hands in pockets,” and “smiling,” or it can be extracted as characteristic text in the form of sentence structures such as “man wearing glasses,” “man wearing black shoes,” “standing in front of the chair,” “hands in pockets,” and “a man wearing glasses and black shoes is standing in front of a chair with his hands in his pockets and smiling.”

[0062] These characteristic texts can improve search speed when searching for data to assign unique identification numbers or navigate specific objects.

[0063] As described above, the object detection data generation module (320) can generate object detection data for each frame and object by extracting features from objects detected in frames within the image using the four feature extraction methods described above.

[0064] Here, the format of the generated object detection data may be as shown in Fig. 5.

[0065] FIG. 5 is a diagram illustrating the format of object detection data according to an embodiment of the present invention. Referring to FIG. 5, the object detection data may be composed of a first data field (11) and a second data field (12), and the first data field (11) may include a camera number, a frame number, a frame time, and arbitrary IDs (e.g., object IDs). The second data field (12) may be classified into multiple categories (12-1, 12-2, 12-3) by object ID, and each object ID may be assigned a “coordinate (x1, y1, x2, y2)” corresponding to the coordinate values ​​of a bounding box, as well as “object detection reliability,” “body feature reliability,” “face detection reliability,” and “feature text.” At this time, “object detection reliability” may be assigned full-body embedding data, upper body data, and lower body data generated according to the extraction of feature information about the entire body of the object. Here, upper body data and lower body data may each refer to a classification of at least one of the types of clothing (e.g., jacket, T-shirt, shorts, long pants, shorts, skirt, dress, etc.) generated by detecting upper body clothing and lower body clothing, the color of the clothing, and the pattern of the clothing, and the information thereof.

[0066] Additionally, "body feature reliability" may be assigned patch and key point embedding data generated by extracting image regions from key points of a recognized object using a pose estimation model. Here, the key point embedding data is data created by extracting N*N images of key point portions from an image, stacking them into an array, and embedding them, which can be used for the purpose of comparison with other objects.

[0067] Face detection confidence can be assigned face coordinate points and face embedding data calculated through a face recognition model, and feature text expressing the characteristics of the object in text form can be assigned to “feature text.”

[0068] In this way, the present invention can assign embedding data to a lower level of reliability in order to facilitate the updating of embedding data through frame-by-frame comparison when reliability is low in a frame constituting an image.

[0069] Next, the tracklet data generation module (340) of the computing device (300) can generate tracklet data by tracking an object in the image based on an object tracking algorithm (S300).

[0070] Specifically, the tracklet data generation module (340) can generate tracklet data using an improved SORT (Simple Online and Realtime Tracking) algorithm according to the present invention.

[0071] Conventional methods for generating tracklets are limited to a single camera area and use Kalman filters and Hungarian algorithms to generate an object's movement path by finding the correlation between the previous frame and the current frame for tracking. However, while existing SORT algorithms offer fast tracking speeds, they suffer from issues such as occlusion and fragmentation (ID switching). To address this, deep learning-based SORT algorithms that compare features in every frame have been researched to improve accuracy, but there is a problem with significantly reduced frame processing speed.

[0073] FIG. 6 is a diagram illustrating an improved SORT according to an embodiment of the present invention. Referring to FIG. 6, the improved SORT according to an embodiment of the present invention detects an object based on the bounding box of the object in each frame of the image (S11), performs an Intersection Over Union (IOU) Match (S12), and can delete objects that do not match (S13) (S16). In particular, when a new object is detected in addition to existing objects (S14), the improved SORT according to an embodiment of the present invention can extract feature information of the newly detected object to generate new tracklet data (S17) and compare it with the feature information of the tracklet data (S15) of existing detected objects (S18). Here, if it corresponds to an existing object, it is included in the tracklet data of that object, and if it does not correspond to an existing object, new tracklet data can be generated (S20). At this time, a Kalman filter is updated, and the location of the object can be predicted and tracked using the updated Kalman filter (S21).

[0074] That is, the object tracking algorithm of the present invention can generate tracklet data for a new object by extracting and comparing feature information of the object (here, the newly detected object) only when a new object is detected in addition to an existing object.

[0075] In this way, by comparing and matching feature information between previously detected objects and newly detected objects, tracklet data can be generated more accurately and quickly.

[0076] Additionally, the computing device (300) can assign a tracklet number to each tracklet data to distinguish and identify tracklet data.

[0077] Meanwhile, the tracklet data generation module (340) can calculate the face angle of an object per frame and map it to the tracklet data. More specifically, the tracklet data generation module (340) can generate a 3D face model by 3D modeling based on the face of an object within a frame (hereinafter, 2D face model). At this time, the tracklet data generation module (340) can use a deep learning model optimized for face 3D modeling, and the face feature information of the object can be used.

[0078] And, the tracklet data generation module (340) can extract keypoints for important feature points of the face, such as eyes, nose, and mouth, of the object's 2D face model and 3D face model, calculate face angle information of the object through comparison between the extracted keypoints, and map the calculated face angle information to the tracklet data. At this time, the face angle information may include the direction and angle of rotation of the object's face relative to the front.

[0079] Additionally, the tracklet data generation module (340) can rotate the 3D face model to a specific angle and compare the 3D face model rotated to a reference angle with the face of the object. At this time, the specific angle may refer to the angle at which the 3D face model faces the reference direction (in this example, the front). Additionally, the specific angle may be set by reflecting the statistical values ​​of the tracklet data.

[0080] That is, the computing device (300) can calculate the face angle of each object for each object within each frame of each image.

[0081] Here, the format of the generated tracklet data may be as shown in Fig. 7.

[0083] FIG. 7 is a diagram showing the format of tracklet data according to an embodiment of the present invention. Referring to FIG. 7, the tracklet data may be composed of a first data field (21) and a second data field (22), and the first data field (21) may include a camera number, frame numbers, frame times, and arbitrary IDs (e.g., object IDs). The second data field (22) may be classified into multiple categories (22-1, 22-2, 22-3) by frame number, and each frame number may be assigned a “coordinate (x1, y1, x2, y2)” corresponding to the coordinate value of a bounding box, and “object detection reliability,” “body feature reliability,” “face detection reliability,” and “feature text.” At this time, full-body embedding data, upper body data, and lower body data generated based on the extraction of feature information about the object's entire body may be assigned to “object detection reliability,” key point embedding data of the object recognized using a pose estimation model may be assigned to “body feature reliability,” face coordinate points and face embedding data calculated through a face recognition model may be assigned to “face detection reliability,” and feature text expressing the characteristics of the object in text form may be assigned to “feature text.”

[0084] In particular, the “face angle” calculated according to the process described above may be further assigned to the face detection reliability.

[0085] In addition, the aforementioned tracklet data may include information regarding the movement speed and acceleration of the object in addition to the data described above.

[0086] Next, the unique identification code matching module (360) of the computing device (300) can match unique identification codes by grouping the tracklet data of objects determined to be the same object when the same object tracking is completed (S400).

[0087] Specifically, the unique identification code matching module (360) can classify tracklet data based on characteristic text. For example, tracklet data can be classified by characteristic text regarding gender (e.g., male, female), and tracklet data can be classified by subdividing through characteristic text regarding clothing, age, etc.

[0088] In addition, the unique identification code matching module (360) can measure the similarity between each tracklet data using face angles mapped to tracklet data classified as feature text. Specifically, the unique identification code matching module (360) can 3D model an object within each tracklet data to create a 3D face model, rotate it using each face angle information, generate a 2D face image of each object through 3D projection, and measure the similarity between tracklet data by comparing the face feature information of the generated face images. Here, the face feature information may be information representing facial features such as the position of eyes, nose, mouth, chin, ears, or the angle of eyebrows, the area of ​​the face, the shape of the nose, or the width of the forehead.

[0089] At this time, at least one of the Cosine, Euclidean, Jaccard, Manhattan, Hamming, Minkowski, Haversine, Chebyshev, and Sørensen-Dice methods may be used to measure similarity.

[0090] Additionally, the unique identification code matching module (360) can determine that the tracklet data is of the same object if the similarity between the tracklet data is greater than or equal to a reference value. Here, the reference value can be determined variably depending on environmental conditions (e.g., camera resolution, illumination value, etc.) and characteristics of the object (e.g., small size, complex shape, etc.).

[0091] In this way, the unique identification code matching module (360) can determine a lower threshold value as the condition becomes more difficult to detect an object.

[0092] In addition, if it is determined that the tracklet data belongs to the same object, the unique identification code matching module (360) can group the corresponding tracklet data as shown in FIG. 8 and map a unique identification number to generate a single metadata. The metadata generated in this way can be stored and utilized in a search database.

[0094] FIG. 8 is a diagram showing the format of final metadata according to an embodiment of the present invention. Referring to FIG. 8, the final metadata may be classified according to a unique identification number assigned to the same object and may include tracklet data.

[0095] In this way, by performing a linkage analysis on the tracklet data of multiple objects within an image and assigning a unique identification code to the same object, the analyzed images can be searched and tracked using only the unique identification code.

[0096] In addition, the computing device (300) can track and connect objects more accurately by first classifying tracklet data based on feature text and then secondarily classifying it using face angles.

[0098] Next, the computing device (300) can provide the same object connection tracking result to the user (100) through the same object connection tracking module (380) (S500). Here, the same object connection tracking result may include an image of the expected movement path within a single image and an image of the expected movement path tracked in multiple images.

[0099] Specifically, when a user's object tracking request is received, the computing device (300) can provide the same object connection tracking result using tracklet data matched to the unique identification number of the tracking target object that corresponds to the user's request.

[0100] For example, when a user requests object tracking using the unique identification number of the object to be tracked, the computing device (300) can use the final metadata of FIG. 8 to search for tracklet data (metadata) of the object corresponding to the received unique identification number, and provide a tracking result for the same object based on the searched tracklet data.

[0101] As another example, when a user requests object tracking with characteristic text of the object to be tracked (e.g., “Request to search for a man wearing a hat and pants as an appearance characteristic of the object”), the computing device (300) can provide the same object connection tracking result by using the tracklet data within the unique identification number corresponding to the object characteristic data using the final metadata of FIG. 8.

[0102] Additionally, when a user requests object tracking through an image of an object to be tracked, the computing device (300) can extract feature information of the object from the image, compare the extracted feature information with the feature information included in the final metadata of FIG. 8 to search for a unique identification number corresponding to the object to be tracked, and then provide a tracking result for the same object.

[0103] Here, the same object connection tracking result may include the movement path of the object in multiple frames and the video corresponding to the movement path (for example, a video formed by connecting multiple frames in which the object was captured in chronological order) through tracklet data.

[0104] This can help the user (100) improve their understanding of the object tracking results.

[0105] Meanwhile, if the computing device (300) receives information about the object to be tracked from the user (100) before performing the unique identification number assignment step (S400), the method of tracking the same object may change. This will be explained with further reference to FIG. 9.

[0106] FIG. 9 is a timing diagram showing a process for tracking the same object according to an embodiment of the present invention.

[0107] Referring to FIG. 9, the computing device (300) receives an image from the camera (200) (S1001), generates object detection data within the image using a deep learning model for the received image (S1002), and can generate tracklet data by tracking objects within the image based on an object tracking algorithm (S1003).

[0108] And, the computing device (300) can provide the user (100) with an image based on the generated tracklet data (S1004). At this time, the image provided to the user (100) may be an image that has been tracked by classifying it by object based on the tracklet data.

[0109] Next, the user (100) can select a specific object that is a tracking target within the received video and transmit it to a computing device (300) (S1005).

[0110] And, the computing device (300) can select a specific video and a specific time zone containing the object to be tracked through the ID of the object selected by the user (100), and check the similarity of the object using only the tracklet data of the selected specific video and specific time zone and group them to match a unique identification number (S1006).

[0111] Next, the computing device (300) can provide the user (100) with an image of the tracked object using metadata corresponding to a unique identification number corresponding to the object selected by the user (100) (S1007).

[0112] In this way, the computing device (300) can save time and system resources by proceeding only with the selected camera and time zone of the object selected by the user (100).

[0113] A specific hardware implementation of a computing device (300) that performs identical object tracking according to the present embodiment will be described below with reference to FIG. 10.

[0114] Referring to FIG. 10, in some embodiments of the present invention, the computing device (300) may be implemented in the form of a computing device. For example, one or more of the modules constituting the computing device (300) may be implemented on a general-purpose computing processor and thus may include a processor (308), an input / output I / O (302), a memory device (304), an interface (306), storage (312), and a bus (314). The processor (308), input / output I / O (302), the memory device (304), the storage (312), and / or the interface (306) may be coupled to each other through the bus (314). The bus (314) corresponds to a path through which data is moved.

[0115] Specifically, the processor (308) may include at least one of a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), microprocessor, digital signal processor, microcontroller, application processor (AP), and logic elements capable of performing similar functions.

[0116] The input / output I / O (302) may include at least one of a keypad, a keyboard, a touchscreen, and a display device. The memory device (304) may store data and / or programs, etc.

[0117] The interface (306) can perform the function of transmitting data to a communication network or receiving data from a communication network. The interface (306) may be wired or wireless. For example, the interface (306) may include an antenna or a wired / wireless transceiver, etc. The memory device (304) enhances the operation of the processor (308) and, as a volatile operating memory for the protection of personal information, may further include high-speed DRAM and / or SRAM, etc.

[0118] The internal storage (312) stores programming and data configurations that provide the functions of some or all of the modules described herein. For example, it may include logic that enables the execution of selected modes of the same object tracking method described above.

[0119] Additionally, a computer program or application is loaded, which includes a set of instructions containing each step of performing the aforementioned identical object tracking method stored in a memory device (304), the operation of recognizing an object in each of the multiple images using a deep learning model and generating object recognition information, the operation of tracking an object in an image based on an object tracking algorithm and generating tracklet data, and the operation of grouping the tracklet data of objects determined to be identical objects and matching a unique identification code, and the processor (308) is enabled to perform each step.

[0120] According to the present invention, in tracking the same object in images captured by a CCTV, the present invention can more accurately detect and track the movement path of an object in multiple images by using not only a deep learning model but also an algorithm for tracking.

[0121] In addition, for images where tracklet data analysis is complete, object search and tracking can be performed using only unique identification codes, thereby effectively reducing the time and human resources required for tracking.

[0122] In addition, once connection tracking is complete, video of the same object can be provided to the user in the form of playback based on a unique identification code, which can help the user improve their understanding of the object tracking results.

[0123] Furthermore, the various embodiments described herein may be implemented, for example, in a recording medium readable by a computer or similar device using software, hardware, or a combination thereof.

[0124] According to hardware implementation, the embodiments described herein may be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, and other electrical units for performing functions. In some cases, the embodiments described herein may be implemented as the control module itself.

[0125] According to software implementation, embodiments such as the procedures and functions described herein may be implemented in separate software modules. Each of the software modules may perform one or more functions and operations described herein. Software code may be implemented as a software application written in a suitable programming language. The software code may be stored in a memory module and executed by a control module.

[0126] The above description is merely an illustrative explanation of the technical concept of the present invention, and those skilled in the art to which the present invention pertains will be able to make various modifications, changes, and substitutions within the scope of the essential characteristics of the present invention without departing from its nature.

[0127] Accordingly, the embodiments disclosed in this invention and the accompanying drawings are intended to illustrate, not limit, the technical concept of the invention, and the scope of the technical concept of the invention is not limited by such embodiments and accompanying drawings. The scope of protection of this invention shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of this invention.

Claims

Claim 1 A method for tracking identical objects within an image performed on a computing device comprises: detecting an object within the image and generating object detection data corresponding to the detected object; tracking each object within the image based on an object tracking algorithm and generating tracklet data for the movement path of the tracked object; and grouping the tracklet data of objects determined to be identical objects to map a unique identification code.The step of generating the object detection data includes, based on the reliability of key points detected from the object, extracting images of a predetermined size at the location of each key point, arranging the extracted images into a single image form, and extracting body feature information of the object from the arranged images to generate the object detection data, wherein the upper and lower body of the object are distinguished and arranged into a single image form considering the extracted locations of the extracted images, and the object detection data includes key point embedding data in which the arranged images are embedded, and the key point embedding data is utilized for comparison with other objects, and the step of generating the object detection data generates the object detection data by configuring it into a first data field and a second data field, wherein the second data field is composed of a hierarchical data format that includes object detection reliability, body feature reliability, face detection reliability, and characteristic text as independent upper levels for each object ID, and so as to facilitate the processing of embedding data update operations at the corresponding lower level through frame-by-frame comparison based on the object detection reliability, body feature reliability, and face detection reliability in the frames constituting the image, the lower level of the object detection reliability The object detection data is generated such that the full-body embedding data of the object is assigned, the face embedding data of the object is assigned dependently to a lower level of the face detection confidence level, and the key point embedding data of the object, generated by embedding the arranged image, is assigned dependently to a lower level of the body feature confidence level; and the step of mapping the unique identification code comprises: a step of classifying the tracklet data based on feature text within the tracklet data; a step of measuring the similarity between the tracklet data classified by feature text; and a step of determining them as the same object and assigning the same unique identification number if the similarity is greater than or equal to a threshold value.A method for tracking identical objects, further comprising, wherein the reference value is variably determined according to the resolution and illumination values ​​of the camera that captured the image and the size and shape characteristics of the object detected in the image.; Claim 2 A method for tracking identical objects according to claim 1, wherein the object detection data is generated on a frame-by-frame basis within an image, and the object detection data includes physical feature information and characteristic text of the object. Claim 3 A method for tracking identical objects according to claim 1, wherein the object tracking algorithm tracks an object based on the bounding box of the object in the image, and when a new object is detected in the image, determines whether the newly detected object is the same object by comparing the feature information of the previously detected object with that of the previously detected object. Claim 4 A method for tracking the same object according to claim 1, wherein the step of generating the tracklet data is characterized by mapping object detection data corresponding to the tracked object to the tracklet data. Claim 5 The method for tracking the same object according to claim 1, wherein the step of generating tracklet data further comprises: a step of generating a 3D face model by 3D modeling the face of the object; and a step of calculating face angle information by comparing the generated 3D face model with the face of the object; and wherein the step of generating tracklet data is characterized by mapping the face angle information of the object to the tracklet data. Claim 6 A method for tracking the same object according to claim 5, wherein the step of calculating the face angle information is characterized by rotating the 3D face model by a specific angle and comparing the 3D face model rotated by a reference angle with the face of the object. Claim 7 A method for tracking identical objects according to claim 6, characterized in that the specific angle is the angle at which the 3D face model looks in the reference direction. Claim 8 A method for tracking identical objects according to claim 1, wherein the step of measuring similarity is characterized by measuring similarity between each tracklet data using face angles within tracklet data classified as feature text. Claim 9 A method for tracking identical objects according to claim 8, wherein the step of measuring similarity comprises modeling an object within each tracklet data to generate a 3D face model, rotating it using each face angle information, generating a face image of each object through 3D projection, and measuring the similarity by comparing the face feature information of the generated face images with each other. Claim 10 A method for tracking identical objects according to claim 1, further comprising the step of, when a user’s object tracking request is received, providing the user with an identical object connection tracking result using tracklet data within a unique identification number corresponding to the object to be tracked. Claim 11 The system comprises a processor, a memory for loading a computer program executed by the processor; and a storage for storing the computer program, wherein the computer program performs the operations of detecting an object in an image and generating object detection data corresponding to the detected object; and tracking each object in the image based on an object tracking algorithm and generating tracklet data for the movement path of the tracked object.The operation includes grouping tracklet data of objects determined to be the same object and mapping a unique identification code, and the operation of generating the object detection data includes extracting an image of a predetermined size at the location of each key point based on the reliability of key points detected from the object, arranging the extracted images into a single image form, and extracting body feature information of the object from the arranged image to generate the object detection data, wherein the upper and lower body of the object are distinguished and arranged into a single image form considering the extracted locations of the extracted images, and the object detection data includes key point embedding data and feature data in which the arranged image is embedded, and the key point embedding data is utilized for comparison with other objects, and the operation of generating the object detection data generates the object detection data by configuring it into a first data field and a second data field, wherein the second data field is composed of a hierarchical data format that includes object detection reliability, body feature reliability, face detection reliability, and feature text as independent upper levels for each object ID, and wherein the corresponding lower To facilitate the processing of embedding data update operations of the level, the object detection data is generated such that the full-body embedding data of the object is assigned to a lower level of the object detection reliability, the face embedding data of the object is assigned dependently to a lower level of the face detection reliability, and the key point embedding data of the object, generated by embedding the arranged image, is assigned dependently to a lower level of the body feature reliability; and the operation of mapping the unique identification code comprises: an operation of classifying the tracklet data based on feature text within the tracklet data; and an operation of measuring the similarity between tracklet data classified by feature text.A computing device further comprising: an operation of determining the same object and assigning the same unique identification number when the similarity is greater than or equal to a reference value; wherein the reference value is variably determined according to the resolution and illumination values ​​of the camera that captured the image and the size and shape characteristics of the object detected in the image. Claim 12 A computing device according to claim 11, wherein the object detection data is generated on a frame-by-frame basis within an image, and the object detection data includes physical feature information and characteristic text of the object. Claim 13 A computing device according to claim 11, wherein the object tracking algorithm tracks an object based on the bounding box of the object in the image, and when a new object is detected in the image, determines whether the newly detected object is the same object by comparing the feature information of the previously detected object with that of the previously detected object. Claim 14 A computing device according to claim 11, wherein the operation of generating tracklet data further comprises: an operation of generating a 3D face model by 3D modeling the face of the object; and an operation of calculating face angle information by comparing the generated 3D face model with the face of the object; and wherein the operation of generating tracklet data is characterized by mapping the face angle information of the object to the tracklet data. Claim 15 A computer-readable recording medium storing a program that performs the same object tracking method described in any one of paragraphs 1 through 10. Claim 16 A computer program containing program code for executing the same object tracking method described in any one of claims 1 to 10, stored on a computer-readable recording medium.

Images