Method and apparatus for object character detection and integrated real-time and stored video searching for CCTV-based abnormal response.

Abstract

The present invention provides a characteristic detection method and apparatus that corrects errors that may occur when detecting the characteristics of an object from video footage, thereby improving the accuracy of object characteristic detection. [Solution] An object characteristic detection method provided in one embodiment of the present invention includes the steps of: acquiring a detection region containing the object from an image (S210); acquiring external appearance characteristic information of the object within the detection region using an artificial intelligence model (S220); acquiring state verification information of the object (S230); identifying changeable variable characteristic information from the external appearance characteristic information using the state verification information (S240); and generating final characteristic information based on the variable characteristic information (S250), wherein the final characteristic information includes, as auxiliary information, information from the variable characteristic information that has a reliability of a preset standard value or higher, or has a correlation of a preset threshold or higher.

Description

[[Technical Field]]

[0001] The present invention relates to a method and apparatus for detecting object characteristics from closed-circuit television video, and further relates to a method and apparatus for comprehensively searching real-time video and stored video by utilizing the detected object characteristics. [[Background Art]]

[0002] With the development of CCTV technology, the importance of searching for a specific object through video analysis in missing person searches, criminal investigations and the like has been increasing. In particular, the technology of accurately detecting object characteristics from CCTV video and searching for an object based on the detected characteristics plays a core role in missing person searches. However, current object characteristic detection technologies have various limitations. In particular, when a part of an object is occluded, there is a problem that the characteristics of the occluded part are incorrectly predicted, or when a plurality of objects overlap, the characteristics are mixed and false detection occurs. In addition, object characteristics may change over time due to detachable items such as hats and coats, which may cause a problem of decreased search accuracy. On the other hand, in current CCTV video surveillance systems, real-time video and stored video are operated as separate systems, making it difficult to comprehensively utilize both types of video. For example, when a user wants to search for a suspect found during real-time monitoring in stored video, the user needs to separately access the stored video system and perform a manual search, which takes a considerable amount of time. In addition, when tracking a suspect found in stored video in real time, it is necessary to alternately check the real-time monitoring system and the stored video system, which greatly reduces work efficiency. In particular, when an abnormal sign such as a fire outbreak or a person falling down is detected, it is necessary to quickly track the past movement route and grasp the current position of the person who was at the scene, but current segmented systems cannot effectively respond to such emergency situations. [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0003] Embodiments of the present invention provide a method for correcting errors that may occur when detecting the characteristics of an object from an image, thereby improving the accuracy of object characteristic detection. One embodiment of the present invention provides a method for preventing erroneous predictions about an object when a part of it is obscured. One embodiment of the present invention provides a method for accurately distinguishing and detecting the characteristics of each object, even when multiple objects are overlapping. One embodiment of the present invention provides a detection method that takes into account the influence of detachable items and is robust to changes in characteristics over time. Another embodiment of the present invention provides an effective object search system that can process real-time surveillance video and historically stored video through a single integrated interface. Another embodiment of the present invention provides a method for processing real-time video and stored video in a single integrated interface, and for automatically detecting objects using an artificial intelligence model for immediate search. Another embodiment of the present invention provides a method for selecting an object detected from a video and automatically searching for it in other videos. Another embodiment of the present invention provides a method for comprehensively analyzing and providing the time-based movement paths of detected objects. Another embodiment of the present invention provides a method for setting up multiple objects as a single group and tracking them on a group basis. [Means for solving the problem]

[0004] An object characteristic detection method provided in one embodiment of the present invention includes the steps of: acquiring a detection region containing the object from an image; acquiring external appearance characteristic information of the object within the detection region using an artificial intelligence model; acquiring state verification information of the object; identifying changeable variable characteristic information from the external appearance characteristic information using the state verification information; and generating final characteristic information based on the variable characteristic information. Here, the state verification information may include at least one of the object's pose vector value, the object's geometric features, and object segmentation information. Here, the final characteristic information may include the appearance characteristic information, which does not include the variable characteristic information. Here, the final characteristic information may include, as auxiliary information, information from the variable characteristic information that has a reliability of a predetermined standard value or higher, or that has a correlation of a predetermined threshold or higher. Here, if the verification information includes the posture vector values, the variable characteristic information may be characteristic information for body parts whose existence cannot be confirmed from the posture vector values. Here, the posture vector value includes vector values ​​for each part of the body, and if no vector value is detected for a specific part of the body, the final characteristic information can be generated without including the characteristics of that body part. Here, if the verification information includes the geometric features and object segmentation information, the step further includes determining a reference object from among a plurality of objects in the detection region, and the variable characteristic information may be characteristic information of an overlapping region that includes the characteristics of other objects that are not the reference object. Here, the reference object can be determined based on the area it occupies within the detection region and the distance between the center point of the detection region and the center point of the object. Here, the reference object may be the object with the largest area and the shortest distance among the plurality of objects. Here, the object segmentation information is obtained through an instance segmentation algorithm, and the overlapping regions between the multiple objects can be identified through the object segmentation information. Here, the characteristics of the superimposed region may not be included in the final characteristic information. Here, among the characteristics of the superimposed region, those characteristics that have a similarity to the characteristics of the reference object that is equal to or greater than a predetermined reference value may be included in the final characteristic information. An object characteristic detection device provided in one embodiment of the present invention includes: an object detection unit that acquires a detection region containing an object from an image; an appearance characteristic acquisition unit that acquires appearance characteristic information of the object within the detection region using an artificial intelligence model; a state verification information acquisition unit that acquires state verification information of the object; a variable characteristic identification unit that identifies changeable variable characteristic information from the appearance characteristic information using the state verification information; and a final characteristic information generation unit that generates final characteristic information based on the variable characteristic information. An object detection method provided by one embodiment of the present invention includes the steps of: detecting an object in a first video; providing the detected object to a user in a selectable form; receiving input of information of an object selected by the user from the provided objects; detecting the object selected by the user in a second video; and providing integrated information for the object selected by the user based on the detection results in the first video and the detection results in the second video, wherein the first video is a real-time video and the second video is a saved video, or the first video is a saved video and the second video is a real-time video. Here, the step in which an object is detected in the first video can be either by detecting the object from the first video using an artificial intelligence model, or by specifying and detecting the object from the first video by a user. Here, the detection of an object by the user can be performed by capturing and pasting an image of the object onto a still screen of the first video, or by specifying the area of ​​at least one object by dragging with the mouse on the still screen of the first video. In this step, detecting the object selected by the user in the second image allows for the detection of objects in the second image that satisfy the criteria set for image similarity and attribute similarity, respectively, with respect to the object selected by the user. Here, the step of providing the integrated information may include the steps of arranging the detection results in the first and second videos in chronological order and integrating them into a single time series, and displaying the time-dependent positions and movement paths of the selected object on a map according to the integrated time series. Here, the object may include at least one of the following: people, including missing persons, criminal suspects and wanted persons; animals; and things, including automobiles. Here, the real-time video includes multiple CCTV video feeds, and the step in which the detected object is provided to the user in a selectable format may display a thumbnail of the object detected in the real-time video, and at least one of the time the object was detected and the CCTV number may be displayed. Here, the step of detecting the selected object in the second video can be performed using saved video footage within a pre-set past time range based on the time when the object was detected in the real-time video. Here, the step of detecting the selected object in the second video can be performed using CCTV within a search area set from the position where the object was detected in the real-time video. Here, the integrated information for the object selected by the user may include at least one of the following: an image area displaying an image of the selected object; an attribute area displaying attribute information of the selected object; a detection list area including at least one of the detection time, CCTV location, place, and similarity information of the selected object; a timeline area displaying the detection time of the selected object in chronological order; and a detection map area displaying the movement path of the selected object. Another embodiment of the present invention provides an object detection method which includes the steps of: inputting conditions for an object group comprising a plurality of objects, wherein the group conditions include the distance between objects constituting the group and the duration for which the distance between objects is maintained; detecting the object group in a first video and a second video based on the input group conditions; and providing integrated information for the object group based on the detection results in the first video and the second video, wherein the first video is a real-time video and the second video is a saved video. Here, the conditions for the subject group may include at least one of the following for each subject: age range, costume color pattern, and costume type. In this step, the object group detection is performed by determining whether the multiple objects satisfy the group condition in the first video and the second video, respectively, and detecting the point in time when the group condition is not satisfied as the point in time when the group is disbanded. Here, the integrated information may include an image area displaying images of the objects constituting the group, an attribute area where the characteristics of the objects constituting the group are individually provided, a detection list area where the detection information of the group is listed in chronological order, a timeline area where the detection time of the group and the dissolution time of the group are visualized and displayed, and a detection map area where the movement path of the group and the dissolution location of the group are visualized and displayed. An object detection device provided by one embodiment of the present invention includes a communication unit that communicates with a server, a processor, and a memory that stores instruction words that cause the processor to perform a specific operation when executed by the processor, wherein the communication unit receives a first video and a second video from the server, the processor detects an object in the received first video, provides the detected object to the user in a selectable form, receives information of the object selected by the user from the provided objects, detects the object selected by the user in the received second video, and provides integrated information for the object selected by the user based on the detection results in the first video and the detection results in the second video, wherein the first video is a real-time video and the second video is a saved video, or the first video is a saved video and the second video is a real-time video. Another embodiment of the present invention provides an object detection device comprising: a communication unit that communicates with a server; a processor; and a memory that stores instruction words that cause the processor to perform a specific operation when executed by the processor, wherein the communication unit receives a first video and a second video from the server; the processor is input with conditions for an object group comprising a plurality of objects, the group conditions include the distance between objects constituting the group and the duration for which the distance between objects is maintained; the processor detects the object group in the received first video and second video based on the input group conditions; and provides integrated information for the object group based on the detection results in the first video and second video, wherein the first video is a real-time video and the second video is a stored video. [Effects of the Invention]

[0005] According to the present invention, the accuracy of object characteristic detection can be improved by correcting erroneous predictions of the artificial intelligence model using verification information that indicates the actual state of the object. In particular, reliable characteristic information can be generated even when part of the object is obscured, multiple objects overlap, or there are detachable items. Through the characteristic information detected according to the embodiments of the present invention, the accuracy and reliability of object retrieval in actual field situations such as missing person searches and criminal investigations can be greatly improved. Furthermore, because real-time and saved video footage can be processed through a single integrated interface, efficient object detection is possible using security video. In particular, the ability to instantly check the past movements of objects found in real-time video and to grasp the current location of objects found in saved video in real time enables rapid situational response. In addition, the ability to select objects using various methods such as video stilling, object area specification, and image capture greatly improves user convenience. Specifically, through the AI-based automatic object detection and group-unit search function for multiple objects, it is possible to track many objects simultaneously and intuitively grasp their movement paths over time. This can be particularly useful in situations where time is critical, as it allows for rapid understanding and response to object movements. [Brief explanation of the drawing]

[0006] [Figure 1] Fig. 1 is a functional block diagram showing the configuration of an object characteristic detection device according to the present invention. [Figure 2] Fig. 2 is a flowchart showing the overall processing procedure of an object characteristic detection method according to the present invention. [Figure 3] Fig. 3 is a diagram for explaining an object characteristic detection method using posture vector values according to a first embodiment of the present invention. [Figure 4] Fig. 4 is a flowchart showing the detailed processing procedure of an object characteristic detection method using posture vector values according to the first embodiment. [Figure 5] Fig. 5 is a diagram for explaining an object characteristic detection method using superimposed region information according to a second embodiment of the present invention. [Figure 6] Fig. 6 is a flowchart showing the detailed processing procedure of an object characteristic detection method using superimposed region information according to the second embodiment. [Figure 7] Fig. 7 is a diagram for explaining an object characteristic detection method using detachable item information according to a third embodiment of the present invention. [Figure 8] Fig. 8 is a flowchart showing the detailed processing procedure of an object characteristic detection method using detachable item information according to the third embodiment. [Figure 9] Fig. 9 is a block diagram showing the hardware configuration of the object characteristic detection device according to the present invention. [Figure 10] Fig. 10 is a configuration diagram showing the overall configuration of a CCTV-based target person search system according to the present invention. [Figure 11] Fig. 11 is a diagram showing a user interface of a CCTV-based target person search system according to an embodiment of the present invention. [Figure 12] Fig. 12 is a flowchart explaining a method in which information of a target person (target persons) is input and search is simultaneously performed on real-time video and stored video according to a fourth embodiment of the present invention. [Figure 13] Fig. 13 is a diagram explaining an example of an execution screen of a method in which information of a target person (target persons) is input and search is simultaneously performed on real-time video and stored video according to the fourth embodiment of the present invention. [Figure 14]This is a flowchart illustrating a method for searching for a subject selected in real-time video using stored video, according to a fifth embodiment of the present invention. [Figure 15] This figure illustrates an example of an execution screen for a method according to the fifth embodiment of the present invention, which searches for a subject selected in real-time video using saved video. [Figure 16] This flowchart illustrates a method for searching for a subject selected from stored video footage using real-time video footage, according to the sixth embodiment of the present invention. [Figure 17] This figure illustrates an example of an execution screen for a method that searches for a subject selected from saved video footage using real-time video footage, according to the sixth embodiment of the present invention. [Figure 18] This flowchart illustrates a method, according to the seventh embodiment of the present invention, in which information on a target group is input and searched simultaneously using real-time video and stored video. [Figure 19] This figure illustrates an example of an execution screen for a system in which information on a target group is input and searched simultaneously using real-time video and saved video, according to the seventh embodiment of the present invention. [Figure 20] This flowchart illustrates a method for searching for a group of subjects selected from real-time video using stored video, according to the eighth embodiment of the present invention. [Figure 21] This figure illustrates an example of an execution screen for a method according to the eighth embodiment of the present invention, which searches for a group of subjects selected in real-time video using saved video. [Figure 22] This flowchart illustrates a method for searching for a group of subjects selected from saved video footage using real-time video footage, according to the ninth embodiment of the present invention. [Figure 23] This figure illustrates an example of an execution screen for a method that searches for a group of subjects selected from saved video footage using real-time video footage, according to the ninth embodiment of the present invention. [Figure 24] This is a block diagram showing the internal configuration of a real-time and archived video integrated object search device according to one embodiment of the present invention. [Modes for carrying out the invention]

[0007] The main concept of the embodiments of the present invention is to extract object characteristics that may cause errors when detecting object characteristics from video footage, such as CCTV footage, and to either exclude those object characteristics from the final characteristics or utilize them as auxiliary information to improve the accuracy of object characteristic detection. For example, in a situation where CCTV footage is used to search for a subject such as a missing person or a criminal suspect, if the lower half of the subject's body is obscured, the artificial intelligence model may incorrectly detect the characteristics of the unseen lower half of the body (First Embodiment). Another example is when multiple people are filmed overlapping in a densely populated area, mixing their characteristics and potentially leading to incorrect detection of the subject's characteristics and a decrease in the accuracy of the subject search (Second Embodiment). Yet another example is when removable items such as hats or coats alter a person's characteristics over time, potentially reducing the accuracy of subject characteristic detection and subject search (Third Embodiment). The embodiments of the present invention propose a method for improving the accuracy of object characteristic detection and object retrieval even in such situations. On the other hand, while the following embodiments may be described assuming a CCTV-based surveillance system, the technical concept of the present invention can also be applied to other fields where the characteristics of an object must be accurately detected, such as robot vision, object recognition in autonomous vehicles, or quality inspection in industrial settings. Furthermore, for the sake of explanation, the object will be mainly assumed to be a person, but the present invention can also be applied to the detection of characteristics of animals and other objects such as vehicles. Based on the main concepts of the present invention described above, embodiments of the present invention will be described in detail below.

[0008] Figure 1 is a block diagram showing the configuration of the object characteristic detection device (100) of the present invention in terms of functional units. The object characteristic detection device (100) includes an object detection unit (110), an appearance characteristic acquisition unit (120), a state verification information acquisition unit (130), a variable characteristic identification unit (140), and a final characteristic information generation unit (150) in order to solve the problems described above. The object detection unit (110) acquires a detection area containing an object from the input video. A detection area that identifies the location and range of the object to be searched is acquired from the video input from a surveillance camera such as a CCTV. The detection area may be detected by, for example, a 'detection box' or a 'bounding box'. The appearance characteristic acquisition unit (120) acquires appearance characteristic information of objects within the detection area using an artificial intelligence model. Here, appearance characteristic information includes visually observed features of the object, such as the object's body (e.g., height, shoulder width, etc.), type and color of clothing, and whether or not accessories are worn. Such appearance characteristic information can be extracted through a deep learning-based object recognition model. The state verification information acquisition unit (130) acquires state verification information of the object. In this invention, 'state verification information' can be information that can be used to verify the reliability of the external characteristics information (i.e., the state of the object) of an object. External characteristics information of an object obtained from an object in a detection box may not accurately reflect the actual characteristics of the object. For example, in a large crowd, the upper body of person A may be visible in the video, but the lower body of person B may be part of another person's body that was filmed overlappingly. In this case, the external characteristics information (especially of the lower body) does not accurately reflect the external characteristics of A. Therefore, in this invention, various types of information that can verify whether or not the external characteristics information is reliable are used as state verification information. In this invention, state verification information may include 'pose vector values' related to the first embodiment, 'geometric features of the object' and 'object segmentation information' related to the second embodiment, or 'item information' related to the third embodiment. Here, pose vector values ​​may be obtained, for example, through a keypoint detection algorithm, and geometric features and object segmentation information may be obtained, for example, through an instance segmentation algorithm. Item information may also be obtained through various algorithms such as YOLO (You Only Look Once) object detection algorithms and semantic segmentation techniques. The variable characteristic identification unit (140) uses the state verification information to identify variable characteristic information from the appearance characteristic information that can be changed. The variable characteristic information may be defined differently from each other depending on the state verification information and the type of the corresponding embodiment. For example, in the first embodiment, the variable characteristic information may be the characteristic information of a body part for which there is no pose vector value for the object. In the second embodiment, the variable characteristic information may be the characteristic information of an 'overlapping region' where multiple objects overlap, and in the third embodiment, the variable characteristic information may be the characteristic information of a detachable item (e.g., hat, bag, coat). The final characteristic information generation unit (150) generates final characteristic information based on the variable characteristic information. The variable characteristic information identified in each embodiment may be removed from the final characteristic information depending on the situation. For example, in the first embodiment, the final characteristic information may be generated by excluding the characteristics of body parts for which no posture vector values ​​exist. In the second embodiment, the final characteristic information may be generated by excluding the characteristic information of the superimposed region, and in the third embodiment, the final characteristic information may be generated by excluding the characteristic information of detachable items. However, in various modified embodiments of the present invention, variable characteristic information may not be excluded but may be included in the final characteristic information as auxiliary information. Specifically, among the variable characteristic information, information that has a reliability of a predetermined standard value or a correlation of a predetermined threshold or higher may be used as auxiliary information. Here, the confidence threshold can be set based on the prediction probability of the artificial intelligence model. For example, a particular characteristic can only be included as supplementary information if the probability value output by the artificial intelligence model when predicting that characteristic is 0.8 or higher. Such a threshold can be adjusted according to the application field and the required accuracy, and can be optimized through experimental verification. Furthermore, the threshold for relevance can be set based on the statistical correlation or semantic relationship between characteristics. For example, in the first embodiment, if the upper body is a suit jacket, the possibility of wearing suit trousers for the shielded lower body may have a high relevance. In such cases, if the conditional probability calculated through a pre-built clothing combination database is 0.7 or higher, that characteristic can be included as supplementary information. Similarly, such thresholds can be adjusted according to the actual application environment and requirements. Such reference values ​​and thresholds can be flexibly adjusted according to the system's operating purpose and environment, and an automatic adjustment mechanism can be implemented through real-time feedback. This ensures both accuracy and flexibility in object retrieval. Thus, the object characteristic detection device (100) of the present invention can generate more reliable final characteristic information by correcting the appearance characteristic information detected by the artificial intelligence model through various state verification information. Such final characteristic information can be used to improve the accuracy of object searches, and its usefulness can be further enhanced in situations where the target is being searched for, such as missing persons or criminal suspects.

[0009] Figure 2 is a flowchart showing the overall processing steps of the object characteristic detection method of the present invention. Each step shown in Figure 2 corresponds to the operation of the object detection unit (110), appearance characteristic acquisition unit (120), state verification information acquisition unit (130), variable characteristic identification unit (140), and final characteristic information generation unit (150) described in Figure 1. Each step can be implemented by a computer program, hardware, or a combination thereof. Refer to Figure 2. In step S210, the object detection unit (110) acquires a detection area containing the object. An area that identifies the location and extent of the object to be searched is detected from the video input from a surveillance camera such as a CCTV. The area that identifies the location and extent of the object may be detected by, for example, a 'detection box' or a 'bounding box'. In step S220, the appearance characteristic acquisition unit (120) acquires appearance characteristic information of the object using an artificial intelligence model. The appearance characteristic information may include visually observed features of the object, such as the object's body (e.g., height, shoulder width), type and color of clothing, and whether or not accessories are worn. Such appearance characteristic information can be extracted through a deep learning-based object recognition model. In step S230, the state verification information acquisition unit (130) acquires state verification information of the object. The state verification information may include attitude vector values ​​related to the first embodiment, geometric features and object division information related to the second embodiment, or item information related to the third embodiment. In step S240, the variable characteristic identification unit (140) identifies the variable characteristic information using the state verification information. The variable characteristic information can be defined differently from the state verification information and the corresponding type of embodiment. For example, in the first embodiment, the variable characteristic information may be the characteristic information of a body part for which no pose vector value exists. In the second embodiment, the variable characteristic information may be the characteristic information of an 'overlapping region' where multiple objects overlap, and in the third embodiment, the variable characteristic information may be the characteristic information of a detachable item (e.g., hat, bag, coat). In step S250, the final characteristic information generation unit (150) generates final characteristic information based on the variable characteristic information. The variable characteristic information identified in each embodiment may be removed from the final characteristic information depending on the circumstances. However, in various modified embodiments of the present invention, the variable characteristic information may not be completely excluded but may be included in the final characteristic information as auxiliary information. For example, information with a high degree of reliability or information with a high correlation to the appearance characteristic information among the variable characteristic information may be used as auxiliary information. The final characteristic information generated through these stages will more accurately reflect the actual state of the object. Hereafter, the processing steps using the specific state verification information in the first to third embodiments will be described in more detail.

[0010] Figure 3 is a diagram illustrating an object characteristic detection method using attitude vector values ​​according to one embodiment of the present invention. In particular, Figure 3 illustrates the process of resolving a problem that may arise when an artificial intelligence model predicts the characteristics of a specific part of an object that is obscured. Refer to Figure 3. (a) shows the process of detecting the characteristics of a person (310) in a normal state, for example, when the entire body of the person (310) is observable. The object detection unit (110) acquires a detection box (301) that includes the person whose entire body is visible in the detection area. Meanwhile, the state verification information acquisition unit (130) acquires various vectors and joint points that represent the posture vector values ​​of the person (310). Specifically, the upper body vector (311) includes the central axis from the head, neck, and spine down to the pelvis, and the upper limb vectors extending from both shoulders to the elbows and wrists. The joint points (312) indicate the positions of major joints such as the head, neck, shoulders, elbows, wrists, and pelvis. The lower body vector (313) shows the vector extending from the pelvis through both knees to the ankles. Such posture vector values ​​can be obtained through a keypoint detection algorithm and are used as state verification information to determine whether each part of the body is actually present. (b) shows the object characteristic detection process when a specific part of a person is obscured by another object, for example, when a person is behind a vehicle or obstacle and their lower body is not visible. In such a situation, the state verification information acquisition unit (130) can acquire only the upper body vector (311) of the person (310) in the detection box (302), and the lower body vector is not detected. At this time, the appearance characteristic acquisition unit (120) acquires appearance characteristic information of the parts that can actually be observed in the detection area through an artificial intelligence model. For example, if the lower body is obscured as in Figure 3(b), the characteristics of the visible upper body (e.g., type and color of the jacket) are acquired as appearance characteristic information. The variable characteristic identification unit (140) identifies the characteristics of body parts for which no posture vector value exists as variable characteristic information. That is, based on posture vectors that do not include lower body vector values ​​(= state verification information), it identifies the characteristics related to the lower body as variable characteristic information. The final characteristic information generation unit (150) generates final characteristic information by excluding variable characteristic information. That is, the final characteristic information is generated so that it includes only appearance characteristic information, excluding the characteristics of body parts for which posture vector values ​​do not exist. Through this process, hallucination, in which artificial intelligence attempts to predict the characteristics of something that does not exist and derives incorrect results, is prevented. However, in modified embodiments of the present invention, if the predictive characteristics for an invisible object have a strong correlation with the characteristics of an observable part (e.g., the likelihood of wearing suit trousers when wearing a suit jacket), and the reliability of the prediction is high, this can be included as auxiliary information in the final characteristic information. Through this processing method, the first embodiment can effectively prevent the problem of artificial intelligence models mispredicting the characteristics of occluded body parts, and can utilize highly reliable predictions as supplementary information when necessary. This can significantly improve search accuracy, especially when searching for missing persons or criminal suspects, by preventing false detections due to incorrect predictions.

[0011] Figure 4 is a flowchart showing the detailed processing steps of the object characteristic detection method using attitude vector values ​​according to the first embodiment. Each step shown in Figure 4 is a sequential implementation of the processing steps described in Figure 3. Refer to Figure 4. In step S410, a detection area including the object's body is acquired. A detection box (301) is generated when the entire body of a person is visible, as in Figure 3(a), and a detection box (302) is generated when the lower half of the body is obscured, as in Figure 3(b). A detection box including the person (310) is generated in various situations. In step S420, an artificial intelligence model is used to acquire information on the external characteristics of the visible body parts of the object. The artificial intelligence model detects the visually observable characteristics of the detected person, such as the type and color of their clothing. At this time, even if the lower body is obscured, as shown in Figure 3(b), the artificial intelligence model acquires the characteristics of the visible upper body as external characteristic information. In step S430, the object's posture vector values ​​are acquired as state verification information. As explained in Figure 3(a), the upper body vector (311), joint points (312), and lower body vector (313) are acquired through the keypoint detection algorithm. On the other hand, in Figure 3(b), only the upper body vector (311) is acquired, and the lower body vector is not detected. Such posture vector values ​​are used as state verification information to determine whether each part of the body actually exists. In step S440, characteristics of body parts for which posture vector values ​​do not exist are identified as variable characteristic information. If a lower body vector is not detected, as in Figure 3(b), lower body-related characteristics (e.g., type and color of trousers) are identified as variable characteristic information. This is a process of identifying predictions based on incomplete information of the artificial intelligence model for parts that have not actually been observed. In step S450, the variable characteristic information is excluded to generate the final characteristic information. That is, the lower body-related characteristics identified as variable characteristic information are excluded, and the upper body characteristics that can actually be observed, as shown in Figure 3(b), are included in the final characteristic information. However, in various modified embodiments of the present invention, characteristics with high reliability among the variable characteristic information can be included in the final characteristic information as auxiliary information.

[0012] Figure 5 is a diagram illustrating an object characteristic detection method using superimposed region information according to a second embodiment of the present invention. In Figure 5, the object is represented as an elliptical object rather than a person in order to simplify the representation of the object so that the superposition situation and the process of determining the reference object can be conceptually clearly understood. Through such a simplified representation, geometric properties such as the superposition region, area, and distance relationship can be shown more clearly. The method described in this embodiment can be applied equally to various types of objects in reality, such as people and vehicles, and can be particularly effectively used in superposition situations between human objects in densely populated environments. Refer to Figure 5. Two objects (510, 520) exist within a single detection box (530), and these objects are detected overlapping each other. The object detection unit (110) detects object 1 (510) and object 2 (520) within the detection box (530). The appearance characteristic acquisition unit (120) acquires appearance characteristic information for each detected object. The artificial intelligence model detects visual characteristics of each object, such as the color, texture, and pattern of the body and clothing. At this stage, all visual characteristics are acquired regardless of whether the objects are superimposed or not. The state verification information acquisition unit (130) acquires the geometric characteristics of the object and object segmentation information as state verification information. Geometrically, the center points (511, 521) of each object and the center point (550) of the detection box (530) are calculated. In addition, the area (S1, S2) that each object occupies within the detection box and the distance information between the center points (D1, D2) are calculated. Specifically, the area of ​​object 1 (S1) and the distance between the center point (511) of object 1 and the center point (550) of the detection box (541=D1) are calculated, and the area of ​​object 2 (S2) and the distance between the center point (521) of object 2 and the center point (550) of the detection box (542=D2) are calculated. Simultaneously, precise boundary and mask information for each object is obtained through an instance segmentation algorithm. This information, along with its geometric features, is used to accurately understand the actual spatial relationships of the objects. The variable characteristic identification unit (140) identifies the superposition region (560) based on the geometric features and object division information acquired by the state verification information acquisition unit (130), determines the reference object, and then identifies the mixture of characteristics occurring in the superposition region as variable characteristic information. Specifically, the process first comprehensively analyzes geometric features and object segmentation information to accurately identify the overlapping region (560). The overlapping region is determined to be the area where the masks of two objects overlap in the object segmentation results. Next, a reference object is determined based on its geometric characteristics. In this embodiment, an object that is shorter in distance from the detection box center point (550) and occupies a larger area may be selected as the reference object. For example, in Figure 5, the area of ​​object 1 (S1) is larger than the area of ​​object 2 (S2), and D1 (541) is shorter than D2 (542), so object 1 (510) may be selected as the reference object. However, if the area and distance conditions are contradictory, for example, if one object has a large area but is far from the center point, and the other object has a small area but is close to the center point, the selection can be made according to predefined weighting values. For example, if area is judged to be a more important factor than distance, the object with the larger area may be selected as the standard object, and conversely, if distance is judged to be a more important factor, the object closer to the center point may be selected as the standard object. Alternatively, weighting values ​​can be assigned to area and distance respectively, and the object with the highest overall score can be selected as the standard object. Thereafter, characteristics that appear in the identified superimposed region (560) that are mixed in a way that differs from the actual characteristics of the reference object are identified as variable characteristic information. For example, if object 1 (510) is a person wearing a blue outfit and object 2 (520) is a person wearing a red outfit, then in the superimposed region (560), the blue and red may overlap and be perceived as purple, or the red of object 2 may be mistakenly perceived as the dominant color. The variable characteristic identification unit (140) compares such a region with the actual characteristics of the reference object and identifies the characteristics that have been deformed or mistakenly perceived due to superimposition as variable characteristic information. The final characteristic information generation unit (150) can generate final characteristic information by excluding variable characteristic information in the superimposed region. This makes it possible to accurately extract only the characteristics unique to the reference object. However, in various modified embodiments of the present invention, variable characteristic information can be included in the final characteristic information as auxiliary information without being completely excluded. For example, if the color of the reference object is blue, blue-green characteristics detected in the superimposed region have a high similarity to blue and can therefore be used as auxiliary information. Also, if a characteristic pattern of the reference object (e.g., a checkered pattern) is partially identified in the superimposed region, such pattern information can be used as auxiliary information to improve the accuracy of object retrieval.

[0013] Figure 6 is a flowchart showing the detailed processing steps of the object characteristic detection method using superimposed region information according to the second embodiment. Each step shown in Figure 6 is a sequential implementation of the processing steps described in Figure 5. Refer to Figure 6. In step S610, a detection area containing multiple objects is acquired. As shown in Figure 5, two objects (510, 520) exist within a single detection box (530), and these objects are detected overlapping each other. At this stage, only the presence of multiple objects is identified. In stage S620, appearance characteristic information is acquired for multiple objects. The artificial intelligence model detects visual characteristics such as the color, texture, and pattern of each object's clothing. At this stage, all visual characteristics are acquired regardless of whether the objects are superimposed or not. In step S630, the geometric features and object segmentation information of the object are acquired as state verification information. As explained in Figure 5, the geometric features include the center point of each object (511, 521), the center point of the detection box (550), the distance between the object center point and the detection box center point (D1, D2), and the area occupied by each object (S1, S2). Simultaneously, the precise boundary and mask information of each object are acquired through the object segmentation algorithm. This information is used as objective state verification information to verify the reliability of the appearance characteristic information. In step S640, the reference object is determined, and the characteristics of the superimposed region are identified as variable characteristic information. In this stage, the superimposed region is first accurately identified using state verification information. The portion where the masks of two objects overlap using object division information is determined as the superimposed region. Next, the reference object is determined based on the geometric features (area information and distance information) obtained in step S630. Once the reference object is determined in this way, the characteristics that appear in the identified superimposed region as being mixed in a way that differs from the actual characteristics of the reference object are identified as variable characteristic information. In step S650, the variable characteristic information of the superimposed region is excluded to generate the final characteristic information. In this stage, the variable characteristic information identified in S640, i.e., characteristics that have been mixed or transformed by superimposition, is excluded, and reliable final characteristic information containing only the characteristics unique to the reference object is generated. However, in various modified embodiments of the present invention, some of the variable characteristic information can be included in the final characteristic information as auxiliary information. For example, if the reference object is wearing clothing with a specific pattern (e.g., stripes), and that pattern is partially observed in the superimposed region, this can be included as auxiliary information. Also, even if the color of the reference object (e.g., dark blue) is mixed with the color of another object (e.g., gray) in the superimposed region and appears as a slightly lighter blue, it can be used as auxiliary information considering the continuity and similarity of the colors. Such auxiliary information can be usefully used to adjust the ranking of search results or evaluate reliability during object search.

[0014] Figure 7 is a diagram illustrating an object characteristic detection method using detachable item information according to a third embodiment of the present invention. The third embodiment provides a method for reliably searching for subjects such as missing persons or criminal suspects, even when they change their clothing or accessories over time. Refer to Figure 7. (a) shows the object, a person, wearing multiple removable items. The person is wearing various items such as a hat (711), glasses (712), a coat (713), and a bag (714). Such items can be easily removed or replaced with other items over time, which can reduce the accuracy of object retrieval. On the other hand, (b) represents the person's body shape information as a stick figure vector. Body shape characteristics that can be measured in a 2D image, such as shoulder width (721), height (722), and torso length (723), are characteristics that remain relatively constant even if clothing or accessories are changed. Here, height (722) is the total length from the top of the head to the toes measured along a vertical line, and shoulder width (721) indicates the horizontal distance between both shoulders. Torso length (723) is the value measured as the vertical distance from the shoulder line to the pelvis. Such body shape information can be estimated through a pose estimation algorithm even when the person is wearing clothing. The appearance characteristics acquisition unit (120) detects the visual characteristics of a person, which include not only the person's body shape information, the color and style of their clothing, but also the characteristics of any removable items they are wearing. For example, if a person is wearing a black hat and a gray coat, these characteristics are detected as appearance characteristics information. The state verification information acquisition unit (130) acquires item information such as hats (711), glasses (712), coats (713), and bags (714) as state verification information. In this process, object detection algorithms such as YOLO (You Only Look Once) and semantic segmentation techniques can be used to accurately identify items worn by a person. The variable characteristic identification unit (140) can identify the characteristics of easily detachable items such as hats (711), coats (713), and bags (714) as variable characteristic information based on the item information acquired by the state verification information acquisition unit (130). This is because such items have unstable characteristics that can easily change over time. Such variable characteristic information identification is not determined by predetermined fixed rules, but rather dynamically based on contextual information and usage patterns. Contextual information can include indoor / outdoor environment information, seasonal information, and time of day information. For example, the wearability of a particular item may change depending on the indoor / outdoor environment. A coat may be considered an essential item outdoors in winter, but it may be identified as a variable characteristic once indoors. Similarly, the wearability of items such as hats and sunglasses may change depending on seasonal factors, and the wearing pattern of a particular item may also change depending on the time of day. The variable characteristic identification unit (140) analyzes such contextual information to identify characteristics that are likely to change in the current situation. It can also learn a specific person's item wearing patterns (e.g., the habit of always wearing the same hat) through continuous observation and generate personalized variable characteristic information. The final characteristic information generation unit (150) generates final characteristic information by excluding variable characteristic information. That is, it excludes the characteristics of items that can be easily put on and taken off, such as hats, bags, and coats, and generates final characteristic information that includes only body shape information and characteristics that are relatively unlikely to change (e.g., glasses). In modified embodiments of the present invention, variable characteristic information can be included in the final characteristic information as auxiliary information without being completely excluded. Whether or not such auxiliary information can be included is determined based on contextual information such as indoor / outdoor environment, season, and time of day. For example, in a winter outdoor environment, a coat may be considered an essential item, and information on the type and color of the coat may be included as auxiliary information, while in summer, the style and color of a hat or sunglasses may be used as auxiliary information. In particular, by combining individual wearing patterns and contextual information obtained through long-term observation, more reliable auxiliary information can be generated. Specifically, for a subject who prefers to wear a black padded jacket in winter, this information, which combines seasonal information and personal preference, can be used as auxiliary information to adjust the priority of search results. Furthermore, if the wearing patterns of specific items show consistency depending on the time of day and location, such context-based pattern information can also be used as auxiliary information. In this way, the accuracy and efficiency of object searches can be improved through the selective inclusion of auxiliary information that utilizes contextual information.

[0015] Figure 8 is a flowchart showing the detailed processing steps of the object characteristic detection method using detachable item information according to the third embodiment. Refer to Figure 8. In step S810, the detection area including the object is acquired. At this time, the person may be wearing various removable items such as a hat (711), glasses (712), coat (713), and bag (714), as shown in Figure 7(a). In stage S820, external appearance characteristics information of the object is acquired. The artificial intelligence model detects the characteristics of the items being worn, along with visual characteristics such as the color, style, and texture of the person's clothing. Body shape information may also be acquired at this stage, and this is considered a stable characteristic that is unlikely to change. In stage S830, item information (e.g., glasses, hat, bag, coat) is acquired as state verification information. Object detection algorithms and semantic segmentation techniques are used to accurately identify and classify the various items worn by the person. In stage S840, variable characteristic information (e.g., hats, bags, coats) is identified from item information. Among the characteristics of an item, those that can be easily changed over time are identified as variable characteristic information. In this process, contextual information (indoors / outdoors, season, time of day) and usage patterns (frequency of wear, duration) are analyzed to dynamically determine variability. For example, a hat may be classified as an item worn temporarily in the summer, but a knit hat may be considered an essential item worn for long periods in the cold winter. In addition, an individual's usual wearing habits (e.g., always carrying a specific bag) may be considered and reflected in the determination of variability. In stage S850, variable characteristic information is excluded to generate final characteristic information. That is, characteristics of items that can be easily put on and taken off, such as hats, bags, and coats, are excluded, and final characteristic information is generated that includes only body shape information and characteristics that are relatively unlikely to change. In modified embodiments of the present invention, variable characteristic information can be included in the final characteristic information as auxiliary information without being completely excluded. For example, if a particular person usually prefers to wear a particular style of hat (e.g., a black baseball cap), this information can be included as auxiliary information to improve search accuracy.

[0016] Figure 9 is a block diagram showing the hardware configuration of the object characteristic detection device (900) of the present invention. The apparatus of the present invention includes a processor (910) and memory (920). It may also include additional components such as an input unit (not shown), a display unit (not shown), and a communication unit (not shown). The processor (910) is a hardware device that includes at least one processing unit such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Processing Unit). The CPU is responsible for general program execution and control, while the GPU accelerates inference for artificial intelligence models that require large-scale matrix operations. In particular, the NPU performs processing specialized for artificial intelligence calculations, such as extracting external appearance characteristic information of objects, calculating pose vector values, detecting overlapping regions, and estimating body shape information. The processor (910) uses this configuration to control the execution of the operations of the embodiments described earlier in Figures 2 to 8, and to control the overall operation of the object characteristic detection device (900). Specifically, in relation to the first embodiment, the processor (910) detects the posture vector value of the object and prevents hallucination by identifying and excluding the characteristics of non-existent body parts as variable characteristic information based on this value. In relation to the second embodiment, the processor (910) analyzes the geometric features (area, center point distance, etc.) and object segmentation information of objects within the superimposed region to identify the superimposed region, determine the reference object, and then identifies the characteristics mixed by superimposition as variable characteristic information and excludes them from the final characteristic information. In connection with the third embodiment, the processor (910) can identify detachable items of the object (e.g., hats, bags, coats, etc.), identify variable characteristic information considering contextual information (indoors / outdoors, season, etc.), and generate final characteristic information by excluding the variable characteristic information so that stable object retrieval is possible even if clothing or accessories are changed over time. The memory (920) consists of a program area, a model area, and a data area. The program area stores program code that implements the operation of each embodiment, the model area stores an appearance characteristic information extraction model, a posture estimation model, an object segmentation model, an item detection model, etc. The data area temporarily stores video data of detected objects, extracted characteristic information, and intermediate calculation results. The apparatus of the present invention may include the following additional configurations: An input unit (not shown) can provide an interface for receiving video input from CCTV or cameras. A storage unit (not shown) may be implemented as a large-capacity storage device for long-term storage of analysis results. A communication unit provides a network interface for sending and receiving data with other systems. In addition, a display unit for visually displaying analysis results and a user interface unit for adjusting analysis settings may be added. Through this hardware configuration, the present invention can accurately detect the characteristics of an object in real time. In particular, by utilizing the parallel processing capabilities of heterogeneous processors, it can effectively solve the problems that arise in the situations of each embodiment described above. The device of the present invention can be implemented in CCTV control systems and intelligent surveillance systems and can be used in various application fields such as searching for missing persons or crime suspects. The object characteristic detection method of the present invention can be particularly effectively utilized in CCTV-based anomaly response systems. For example, it can be used to accurately detect the characteristics of individuals present at a fire scene or a scene where a person has collapsed, and to track their past movements or determine their current location. Specifically, in a fire scene, situations frequently occur where parts of an object are obscured by smoke or flames, and in such cases, the posture vector-based characteristic detection method according to the first embodiment of the present invention can be effectively utilized. Furthermore, in evacuation situations, situations may occur where many people are densely packed together and overlapping, and in such cases, the overlapping region-based characteristic detection method according to the second embodiment of the present invention can be utilized. Moreover, during the evacuation process, people may remove or discard coats, bags, etc., and in such cases, the detachable item-based characteristic detection method according to the third embodiment of the present invention can be utilized to continuously track the same person despite changes in clothing. The final characteristic information generated by the object characteristic detection method of the present invention is used as accurate query information for object searching in the integrated search system of real-time video and stored video described below, enabling a rapid and accurate response when abnormal signs occur. In the following, as another embodiment of the present invention, a system and method for comprehensively searching for an object using real-time video and stored video, utilizing the final characteristic information generated by the object characteristic detection method described above, will be explained. In particular, an integrated search system will be described that can quickly track the past movements and determine the current location of a person who was at the scene when abnormal signs such as a fire or a person falling are detected. The main concept of other embodiments of the present invention is to process real-time video and (past) archived video in a single integrated interface, enabling users to efficiently search for desired objects through both real-time and archived video. Specifically, users can search using real-time and archived video by entering criteria for the object, or if the user does not enter criteria, they can select an object that has been automatically detected while monitoring real-time video, and the same object can then be searched using archived video. The reverse is also possible. Based on the main concepts of the other embodiments of the present invention described above, other embodiments of the present invention will be described in detail below. In particular, the “object” of the present invention may include at least one of persons, animals, and automobiles, including missing persons, criminal suspects, and wanted persons, but for the sake of explanation, the object will be assumed to be a person. Hereafter, the object may be referred to as a “target person.” Furthermore, the following description will be given as an example in which the technical idea of ​​the present invention is applied to a system for searching for a target person. Furthermore, for the sake of explanation, the target person will be assumed to be a missing person, a criminal suspect, or a wanted person. Furthermore, while the following other embodiments of the present invention are described assuming a system that detects subjects using security footage based on CCTV, the technical ideas of the other embodiments of the present invention can also be applied to other fields where integrated analysis of real-time and stored footage is required, such as traffic control in smart cities.

[0017] Figure 10 is a configuration diagram showing the overall configuration of a CCTV-based subject search system according to another embodiment of the present invention. As shown in Figure 10, this system includes numerous CCTVs (1011, 1012, 1013), a server (1020), and a user terminal (1030). Multiple CCTVs (1011, 1012, 1013) capture video in real time and transmit it to a server (1020). In this embodiment, three CCTVs are illustrated exemplarily, but in actual implementations, hundreds or more CCTVs may be connected to the system. Each CCTV is installed in a specific area or facility to monitor that area and may include a high-resolution camera and night vision capabilities. In particular, given the nature of target search, it is desirable to have a high resolution that can clearly identify human features. The server (1020) is responsible for processing and storing video footage received from each CCTV. The server (1020) processes the received video in real time and transmits it to the user terminal (1030), while also storing the received video in a database for future retrieval. In particular, the server (1020) includes an artificial intelligence-based object detection module that can automatically detect people in the video and extract and store the characteristics of the detected people (clothing, body type, etc.). Such feature extraction must be performed in real time for immediate retrieval, requiring high-performance processing capabilities capable of processing a large number of CCTV videos in parallel. The server (1020) also includes a database management system for efficient storage and retrieval of video data. The video data is stored along with metadata such as time and location, and the characteristic information of detected objects is also indexed to enable high-speed retrieval. In particular, the database must be optimized to allow simultaneous real-time and archived video retrieval. The user terminal (1030) communicates with the server (1020) and provides a user interface for searching for subjects. The user terminal (1030) also provides an integrated interface for real-time video monitoring and stored video retrieval, which is described in detail in Figure 11. The user terminal (1030) must be able to simultaneously monitor a large number of CCTV video feeds, including a high-resolution display, and must have the capability to smoothly play back real-time streaming video and stored video. The user terminal (1030) must also be able to handle various user interactions, such as setting search criteria, displaying search results, and tracking user movement. In particular, considering the urgency of target searches, it must be possible to perform quick searches and confirm results through an intuitive and efficient user interface. The specific configuration of such a user interface is explained in detail in Figure 11. Through the system configuration of the present invention, real-time video and stored video can be searched in an integrated manner, which can greatly contribute to the rapid identification of subjects and the understanding of their movements.

[0018] Figure 11 shows the user interface of a CCTV-based subject search system according to one embodiment of the present invention. As shown in Figure 11, the user interface of the present invention includes a search condition setting unit (1110), a real-time video display unit (1120), a saved video display unit (1130), a real-time video search unit (1140), a saved video search unit (1150), and a detection result display unit (1160). The search condition setting unit (1110) includes search direction (1111), target information (1112), group settings (1113), similarity conditions (1114), search area (1115), and search map (1116), etc. The search direction (1111) is the area that determines the direction of the search and can offer three options: bidirectional, real-time ==> save, and save ==> real-time. Two-way search can be used when attempting to search all video footage before and after the time a missing person report was filed, i.e., both archived and real-time video. Real-time ==> Archived search can be used when attempting to search for a person found in real-time video using archived video. Archived ==> Real-time search can be used when attempting to search for a person found in archived video using real-time video. The subject information (1112) is used when a system administrator (or user) (for example, when searching for a missing person) enters the criteria for a subject (e.g., missing person, suspect, etc.). First, the number of subjects can be selected, and a separate tab for each subject is generated according to the selected number. In each subject tab, an image of the subject can be registered through the photo registration area. In addition, attribute information for each subject can be entered, such as age group (e.g., late 20s), upper garment color pattern (e.g., black padding), lower garment color pattern (e.g., jeans), upper garment type (e.g., thick jacket), etc. In the group setting (1113), it is possible to set whether or not the relationships between subjects constitute a group when searching for multiple subjects. For example, by setting the distance between groups (e.g., 5m) and the duration (e.g., 30 seconds), subjects moving together within the set distance for a specified period of time can be recognized as a single group and searched for. For example, by setting the family and friends who were with a missing child as a group and searching for them, it becomes possible to capture the moment when the group breaks up at a specific point in time or to understand the overall movement of the group members. Examples of group settings will be described later in the 7th to 9th embodiments. The similarity criteria (1114) allows you to set threshold values ​​for image similarity and attribute similarity. Image similarity indicates the visual similarity to the registered photograph, while attribute similarity indicates the degree of match with the entered feature information. For example, if you set both image similarity and attribute similarity to 80% or higher, the search results will include individuals who meet both conditions. These values ​​are used as default settings or can be changed by the user. Figure 11 shows an example where default values ​​have been set. In the search area (1115), the search radius of the CCTV to be searched (e.g., the location where the missing person went missing, i.e., a radius of 3 km from the reference location), the search time range (e.g., the time the missing person went missing, i.e., saved footage from the 30 minutes prior to the reference time), and the selection method for the CCTV to be searched (based on a specified location, or whether individual selection is possible) can be set. These values ​​are either maintained at the initial settings or can be changed by the user. The CCTV search area set here can be visually confirmed through the search map (1116). In some cases, the user can directly adjust the CCTV reference point and area on the map. The search map (1116) displays the location and search radius of the selected CCTV, and the search area can be finely adjusted by zooming in / out and moving the map. The real-time video display unit (1120) displays at least one CCTV video (six examples are given, 1121-1126) in real time, and the stored video display unit (1130) plays back and displays at least one CCTV video (six examples are given, 1131-1136) from a specified specific past point in time. On the other hand, each of the CCTV images displayed on the stored image display unit (1130) can correspond to each of the CCTV images displayed on the real-time image display unit (1120). In Figure 11, each of the six images (1131-1136) on the stored image display unit (1130) is shown as corresponding to each of the images (1121-1126) on the real-time image display unit (1120). However, in some cases, only the stored images (1131, 1133) corresponding to a specific image (e.g., 1121, 1123) selected on the real-time image display unit (1120) may be enlarged and displayed on the stored image display unit (1130), or vice versa. The real-time video search unit (1140) may display subjects detected in real-time video (1121-1126) in the form of thumbnails (e.g., 1141-1143), and the archived video search unit (1150) may display thumbnails (e.g., 1151, 1152, 1154) of subjects detected in archived video (1131-1136). Each thumbnail may be provided with information such as the subject's identifier, the time of detection, and the CCTV where it was detected. Such subjects can be automatically detected through an artificial intelligence-based object detection algorithm. However, in some cases, the user can also directly specify and detect subjects in the CCTV video. The detection result display unit (1160) provides various information about the detected results, such as images (1161) of each detected subject, subject attributes (1162), a detection list (1163), a detection timeline (1164), and a subject detection map (1165). In other words, the detection result display unit (1160) can arrange the information detected in real-time video and stored video in chronological order and generate a single integrated time-series data. This integrated time-series data can be visualized through the detection timeline (1164), and the subject detection map (1165) can display the detection location and movement path at different times according to the integrated time series. Through this integrated time-series-based analysis, the user can intuitively grasp the overall movement path and patterns of the subjects. Specifically, the subject image (1161) displays an image of the detected subject, and the subject attributes (1162) displays detailed classification information such as age group, upper garment color pattern, lower garment color pattern, and upper garment type. The detection list (1163) displays the detection results for the subject. For example, information such as detection time, detection CCTV number, detection location, and similarity is displayed. The detection timeline (1164) visualizes the detection results in a timeline format, and the subject detection map (1165) displays the detected location and movement path on a map. As described above, the user interface of the subject search system according to the embodiment of the present invention provides various search scenarios in the search condition setting unit (1110), ranging from single subject searches to group-based searches of multiple subjects. The system searches for subjects based on the conditions entered by the user in real time and in stored video, or, if the user does not enter subject conditions, the system can monitor real-time video (or stored video) and the user can select a subject that has been automatically detected, allowing for immediate searching of the same subject in stored video (or real-time video). Furthermore, the search results can be visualized in various ways. Through the configuration of this user interface, a series of processes from setting conditions for subject search to confirming results can be efficiently executed. Based on the interface in Figure 11, various embodiments of the present invention will be described below. The fourth to sixth embodiments describe cases where the subjects are not set as a group (single subject or multiple subjects). In the fourth embodiment, a two-way search method is described in which information about the subject(s) is input and a search is performed simultaneously using real-time video and stored video. This makes it possible to comprehensively understand the movements of the subject(s) before and after the last sighting (or disappearance) time and location. This is explained in Figures 12 and 13. In the fifth embodiment, a case is described in which subjects (etc.) are automatically searched for in real-time video based on artificial intelligence without inputting subject information, and if the user selects subjects (etc.) to be detected from among these, the selected subjects (etc.) are automatically searched for and detected in past saved video. Through this embodiment, the past movement paths of the currently discovered subjects (etc.) can be confirmed. This is explained in Figures 14 and 15. In the sixth embodiment, an example is described in which subjects (etc.) are automatically searched for in the saved video based on artificial intelligence without any input of subject information, and if the user selects subjects (etc.) to be detected from among these, the selected subjects (etc.) are automatically searched for and detected in the real-time video. Through this embodiment, the current location of subjects (etc.) found in the saved video can be confirmed. This is explained in Figures 16 and 17. The seventh to ninth embodiments describe cases in which multiple subjects are set as a group. In the seventh embodiment, a two-way search method is described in which information of the target group is input in the fourth embodiment, and a search is performed using real-time video and saved video. This makes it possible to comprehensively understand the movements of the target group before and after the last sighting (or disappearance) time and location. This is explained in Figures 18 and 19. In the eighth embodiment, an example is described in which, as in the fifth embodiment, subjects are detected in real-time video based on artificial intelligence without inputting information about the subject group, and if the user selects a subject group to search for from among these, the selected subject group is automatically detected in the saved video. Through this embodiment, it is possible to check the past movement paths of the subject group discovered in real-time video. This is explained in Figures 20 and 21. In the ninth embodiment, an example is described in which, as in the sixth embodiment, subjects are detected in the saved video based on artificial intelligence without inputting information about the subject group, and if the user selects a subject group to search for from among these, the selected subject group is automatically detected in the real-time video. Through this embodiment, the current location of the subject group searched in the saved video can be confirmed. This is explained in Figures 22 and 23.

[0019] Figure 12 is a flowchart illustrating a method for inputting information about a subject (etc.) and searching using real-time video and stored video, according to the fourth embodiment of the present invention. Refer to Figure 12. The bidirectional search method of the fourth embodiment includes the following steps. First, the search criteria for the subject are entered through the search criteria setting unit (1110) (S1210). The search criteria include bidirectional search (1111) and subject information (1112) where image or attribute information (age group, upper garment color pattern, lower garment color pattern, upper garment type, etc.) for the subject may be entered. Similarity criteria (1114) and search area (1115) may also be set together. Next, the system detects the subject in the real-time video displayed on the real-time video display unit (1120) based on the entered search conditions (S1220). This can be performed through an artificial intelligence-based object detection algorithm, and the detected subject is displayed in thumbnail form on the real-time video search unit (1140). Next, the system detects the subject in the saved video displayed on the saved video display unit (1130) based on the same search criteria (S1230). The detected subject is displayed in thumbnail form on the saved video search unit (1150). Finally, the system integrates the detection results from real-time video and saved video and displays them on the detection result display unit (1160) (S1240). The integrated detection results provide detailed information through the subject image (1161), subject attributes (1162), and detection list (1163), and in particular, the detection information is visualized in chronological order through the detection timeline (1164). The detection location and movement path are also displayed on the map through the subject detection map (1165). Through this two-way search process, it is possible to simultaneously understand the subject's past movements and current location based on the last sighting, which can be a very effective approach in searching for a person.

[0020] Figure 13 illustrates an example of an execution screen for a system in which information about a subject (etc.) is input and searched using real-time video and saved video, according to the fourth embodiment of the present invention. In Figure 13, areas requiring special attention are indicated separately with reference numbers beginning with "13". Refer to Figure 13. Specifically, the search condition setting unit (1110) is used to set detailed conditions for bidirectional target search. Such conditions can be entered by the user. Refer to box number 1310. In the search direction (1111), "two-way" is selected to search both real-time and archived video together. In the subject information (1112), single subject search is selected, and the subject's characteristics are entered. Specifically, the age range is set to "late 20s," the upper garment color pattern to "black padding," the lower garment color pattern to "jeans," and the upper garment type to "thick jacket." The search time is set to "2024-02-14 09:20," and the location is set to "Jamsil Saenae Station." While such time and location are usually set based on the time and place where the subject went missing, it goes without saying that the time and location can be changed according to user input. The group settings (1113) field is blank. This is because the example in Figure 13 is presented as if a single subject was entered. The similarity criteria (1114) and search area (1115) are maintained at their default values ​​or can be changed by the user. In Figure 13, they are presented as default values, similar to the example in Figure 11. The real-time video display unit (1120) displays the current video from the CCTV selected according to the set conditions in real time. In particular, it can be confirmed that a person similar to the search target has been detected in the video from CCTV 31123) (1321). This is a person who satisfies all of the set image similarity and attribute similarity conditions, and the real-time video search unit (1140) may automatically extract and display a thumbnail (1341) of that person. The saved video display unit (1130) can display saved video from each CCTV within a predetermined range (e.g., 30 minutes) based on the search time (09:20) set in the real-time video display unit (1120). As the saved video is automatically played back, if a subject that meets the conditions set in the search condition setting unit (1110) is detected using the object search algorithm based on artificial intelligence, the detected subject may be displayed in thumbnail form in the saved video search unit (1150). In the example in Figure 13, subjects identical to the search target (i.e., those that meet the similarity conditions) are detected in the videos of CCTV 4 (1134) and CCTV 5 (1135) on the saved video display unit (1130) (1331, 1332), and thumbnails (1351, 1352) are displayed in the saved video search unit (1150) according to the detection results. The detection result display unit (1160) comprehensively analyzes and provides information on the subject detected from real-time video and stored video. In the example in Figure 13, the subject image (1161) displays the image of the detected subject (1361). The displayed image can be set in various ways. Various methods are possible through pre-setting, such as displaying all detected images, displaying the image with the highest similarity among the frontal images as a representative image, or displaying the most recent image. Here, the frontal image with the highest resolution is shown as the representative image (1361). The subject attributes (1162) provide various attribute information extracted from the detected subjects according to the artificial intelligence-based analysis of the detected results. The detection list (1163) displays each detection in chronological order, allowing you to check the detection time, CCTV number / location, and similarity information for each item. The detection timeline (1164) visually displays the detection history according to the detection results, and this can be linked with the subject detection map (1165) to display the travel route during that time period as arrows on the map. In the subject detection map (1165) of Figure 13, the route taken by the subject who went missing at Jamsil-Senae Station at 09:20, the time the search was set up, between 08:50 (30 minutes prior to the set time) and the current time of 09:25 is shown. Through this two-way search, the subject's movements around the time they went missing (09:20) can be tracked in real time and through archived video footage, providing highly effective information for quickly locating the person. Figure 13 illustrates the case where a single subject is entered as a search criterion. However, in some cases, multiple subjects who are not a group (e.g., Subject 1, Subject 2) may be entered as a search criterion. In this case as well, the bidirectional search process for subjects according to the fourth embodiment can be applied independently to each subject. That is, if information on multiple subjects is entered in the search criterion setting unit (1110), the system will perform a separate search for each subject and provide individual detection results. For example, when searching for two subjects, an independent timeline is generated for each subject, making it possible to understand their individual movements. On the other hand, the case where multiple subjects are set as a group and entered as a criterion will be described later in the seventh embodiment.

[0021] Figure 14 is a flowchart illustrating a method for searching for a subject selected in real-time video using stored video, according to a fifth embodiment of the present invention. First, real-time video of the location specified via the search condition setting unit (1110) is displayed on the real-time video display unit (1120) (S1410). At this time, the real-time video displayed may be CCTV video of the location and time set by the user via the search condition setting unit (1110). Next, at least one subject is detected in the real-time video displayed on the real-time video display unit (1120) through an object detection algorithm based on artificial intelligence, and the detected subject is displayed on the real-time video search unit (1140) (S1420). The detected subject may be displayed in thumbnail form, and the CCTV number may be provided along with the subject's identifier, the time of detection, etc. Next, the user selects at least one subject from the detected subjects displayed on the real-time video search unit (1140) (S1430). This selection can be performed using various methods, such as mouse clicks or screen touches. On the other hand, if no target person is automatically detected at step S1420, or if a target person is automatically detected but the desired target person is not found, and the user finds the desired target person in the real-time video, the user can stop the real-time video and select the target person by using the clipboard function linked to the system's operating system, such as capturing or selecting the target person's image in the stopped real-time video (e.g., specifying an area by dragging with the mouse), and pasting the image of the target person into the real-time video search unit (1140) to select the target person. Next, the system analyzes the characteristics of the subject selected by the real-time video search unit (1140) and uses an artificial intelligence-based object search algorithm to detect the same subject (i.e., one that meets the similarity criteria) in the stored video displayed on the stored video display unit (1130) (S1440). At this time, the search may be performed within a predetermined time period (e.g., 30 minutes) and geographical range (e.g., radius of 3 km), and the detected results may be displayed in thumbnail form on the stored video search unit (1150). Finally, the system integrates the detection results from real-time video and saved video and displays them on the detection result display unit (1160) (S1450). For example, the image of the subject detected in the saved video (1161) and the attributes of the detected subject (1162) may be displayed, and the detection list (1163) may also be displayed. Furthermore, the contents of the detection list (1163) can be visualized and the movement routes and times of the subjects can be displayed in chronological order on the timeline (1164) and the subject detection map (1165).

[0022] Figure 15 illustrates an example of an execution screen for a method according to the fifth embodiment of the present invention, which searches for a subject selected in real-time video using saved video. In Figure 15, areas requiring special attention are indicated separately with reference numbers beginning with "15". Refer to Figure 15. The search condition setting unit (1110) is configured with conditions for searching for subjects selected from real-time video in the saved video. Refer to the box with reference number 1510. In the search direction (1111), "Real-time ==> Saved" is selected to perform a linked search of real-time video and saved video. In the fourth embodiment described in Figure 13, the user enters the conditions for the subject, so no information is displayed in this item (1112). Furthermore, the group settings (1113) item is also blank. The similarity conditions (1114) and search area (1115) are exemplified as being set to their default values, as in Figure 13. According to the set conditions, the real-time video display unit (1120) plays and displays the current video from the selected CCTV in real time. While the real-time video is being played, subject 1 is detected in the video from CCTV 3 (1123) (1523) and subject 2 is detected in CCTV 4 (1524) based on artificial intelligence. The detected subjects are displayed as subject 1 (1543) and subject 2 (1544) in the real-time video search unit (1140). Let's assume the user selected Subject 1 (1543) from Subject 1 (1543) and Subject 2 (1544). Meanwhile, the saved video display unit (1130) displays and plays back the saved CCTV footage (1131-1136) corresponding to the real-time images (1121-1126) of each CCTV displayed on the real-time video display unit (1120). Saved footage from 08:55, which is 30 minutes prior to 09:25 (the time when subject 1 was detected) (as set as the search time range), can be played back. The system detects subjects identical to subject 1 (1543) selected by the user from the stored video footage (1131-1136) (i.e., subjects that meet the similarity criteria). Subjects that meet the similarity criteria are detected in the video footage of CCTV 4 (1134) and CCTV 5 (1135), respectively (1534, 1535), and these are displayed in the stored video search unit (1150) (1554, 1555). At this time, the identifier of the detected subject, the detection time, etc. may also be displayed. The detection result display unit (1160) comprehensively analyzes and provides information on the subject detected in real-time video and saved video. In the example in Figure 15, the subject image (1161) displays the image of the detected subject (1561). The displayed image can be set in various ways. Various methods are possible through pre-setting, such as displaying all detected images, displaying the image with the highest similarity among the frontal images as a representative image, or displaying the most recent image. Here, it is exemplified that all images of subject 1 detected in the saved video are displayed. The subject attributes (1162) provide various attribute information of the subject according to the artificial intelligence-based analysis of the detected results. The detection list (1163) displays each detection in chronological order, allowing you to view the detection time, CCTV location, specific location, and similarity information for each item. The detection timeline (1164) visually displays the detection history according to the detection results, and this is linked to the subject detection map (1165) to display the movement route during that time period as arrows on the map. In the subject detection map (1165) of Figure 15, the route taken by the subject during the 30-minute search time range is illustrated, with 09:25, the time when subject 1 was detected on the real-time video display unit, as the reference point. In this way, by searching for subjects similar to the one selected in real-time video from the archived video, it is possible to comprehensively track the subject's movement path during a certain period prior to the time of detection using the archived video. Such an embodiment can provide effective information not only for finding subjects, but especially for revealing the past movements of criminal suspects. Figure 15 illustrates the case where a single subject is entered as a search criterion. However, in some cases, multiple subjects who are not a group (e.g., Subject 1, Subject 2) may be entered as a search criterion. In this case as well, the detection process for stored video for subjects according to the fifth embodiment can be applied independently to each subject. That is, if information on multiple subjects is entered in the search criterion setting unit (1110), the system will perform a separate search for each subject and provide individual detection results. For example, when searching for two subjects, an independent timeline is generated for each subject, making it possible to understand their individual movements. On the other hand, when multiple subjects are set as a group and entered as a criterion, this will be described later in the eighth embodiment.

[0023] Figure 16 is a flowchart illustrating a method for searching for subjects selected from saved video footage using real-time video footage, according to the sixth embodiment of the present invention. First, the saved video from the location specified via the search condition setting unit (1110) is played back and displayed on the saved video display unit (1130) (S1610). At this time, the saved video displayed may be CCTV footage from the location and time set by the user via the search condition setting unit (1110). Next, at least one subject is detected in the stored video on the stored video display unit (1130) through an object detection algorithm based on artificial intelligence, and the detected subject is displayed on the stored video search unit (1150) (S1620). The detected subject may be displayed in thumbnail form, and the CCTV number may be provided along with the subject's identifier, the time of detection, etc. Next, the user can select at least one subject from the detected subjects displayed in the saved video search unit (1150) (S1630). This selection can be performed in various ways, such as by touching the screen with a mouse click. If, in the S1620 stage, no target person is automatically detected based on artificial intelligence, or if a target person is detected but the user's desired target person is not found, and the user finds the desired target person in the saved video, the user can stop the saved video and select the target person by using the clipboard function linked to the system's operating system, such as capturing or selecting the target person's image in the stopped saved video (e.g., specifying an area by dragging with the mouse), and pasting the image of the target person into the saved video search unit (1150) to select the target person. Next, the system analyzes the characteristics of the subject selected by the stored video search unit (1150) and uses an artificial intelligence-based object search algorithm to detect similar subjects in the real-time video displayed on the real-time video display unit (1120) (S1640). The detected results are displayed in thumbnail form on the real-time video search unit (1140). Finally, the system integrates the detection results from the saved video and the real-time video and displays them on the detection result display unit (1160) (S1650). For example, the image of the subject detected in the real-time video (1161) and the attributes of the detected subject (1162) may be displayed, and the detection list (1163) may also be displayed. Furthermore, the contents of the detection list (1163) can be visualized and the movement routes and times of the subjects can be displayed in chronological order on the timeline (1164) and the subject detection map (1165).

[0024] Figure 17 illustrates an example of an execution screen for a method that searches for a subject selected from saved video footage using real-time video footage, according to the sixth embodiment of the present invention. In Figure 17, areas requiring special attention are indicated separately with reference numbers beginning with "17". Refer to Figure 17. The search condition setting unit (1110) has conditions set for searching for subjects selected from saved video footage in real-time video. Refer to the box with reference number 1710. In the search direction (1111), "Saved ==> Real-time" is selected to perform a linked search of saved and real-time videos. In the fourth embodiment described in Figure 13, the user enters the conditions for the subjects, so no information is displayed in this item (1112). Furthermore, the group settings (1113) item is also blank. The similarity conditions (1114) and search area (1115) are exemplified as being set to their default values, as in Figure 13. According to the set conditions, the saved video footage from the selected CCTV is played back and displayed on the saved video display unit (1130). During the playback of the saved video, subject 1 is automatically detected in the video from CCTV 1 (1131) (1731), subject 2 is automatically detected in CCTV 3 (1133) (1733), and subject 3 is automatically detected in CCTV 4 (1134) and CCTV 6 (1136) (1734, 1736). The detected subjects are displayed on the saved video search unit (1150) as subject 1 (1751), subject 2 (1753), and subject 3 (1754, 1756). Let's assume the user selected subject 3 (1754 or 1756) from the three subjects mentioned above. The real-time video display unit (1120) plays back and displays the real-time video (1121-1126) of each CCTV corresponding to the saved images (1131-1136) of each CCTV displayed on the saved video display unit (1130). The system detects individuals identical to (i.e., meeting the similarity criteria for) individual 3 selected by the user from real-time video (1121-1126). Individuals identical to individual 3 are detected in the video of CCTV 6 (1126) (1726), and this is displayed in the real-time video search unit (1140) (1746). At this time, the individual's identifier, detection time, etc., may also be displayed. The detection result display unit (1160) comprehensively analyzes and provides information on the subjects detected in the saved video and real-time video. In the example in Figure 17, the subject image (1161) displays an image of the detected subject 3 (1761). As explained in Figure 13, images can be displayed in various pre-set formats, but here it is exemplified as displaying the most recent detection image of subject 3 detected in the saved video and real-time video. The detection list (1163) and detection timeline (1164) visually display the detection history according to the detection results, and these are linked to the subject detection map (1165) to display the movement route during that time period as arrows on the map. In the subject detection map (1165) in Figure 17, the route taken by subject 3 from 09:15, the time of the first detection of subject 3 in the saved video display unit, to the present time (09:25) is illustrated. In this way, by searching for subjects similar to those selected from saved video footage in real-time video, it is possible to comprehensively track the movement path of the subjects from the initial detection point to their current location using real-time video. Such an embodiment can provide effective information not only for determining the current location of a subject, but especially for quickly determining the current location of a suspected criminal.

[0025] Figure 18 is a flowchart illustrating a method for inputting information on a target group and searching using real-time and stored video footage, according to the seventh embodiment of the present invention. Refer to Figure 18. The bidirectional search method of the seventh embodiment includes the following steps. First, the search criteria for the target group are entered through the search criteria setting unit (1110) (S1810). In the search criteria, bidirectional search is selected in the search direction (1111), and image or attribute information for each target is entered in the target information (1112). In addition, the group settings (1113) specify the distance between groups (e.g., 5m) and the duration (e.g., 30 seconds), and subjects moving together within the specified distance for the specified time are recognized as a single group. Similarity conditions (1114) and search area (1115) can also be set at the same time. Next, the system detects groups of subjects in the real-time video displayed on the real-time video display unit (1120) based on the entered search conditions (S1820). This is performed automatically through an artificial intelligence-based object detection algorithm, and subjects who meet the group setting conditions are detected together. The detected groups of subjects are displayed in thumbnail form on the real-time video search unit (1140). Next, the system detects the target group in the saved video displayed on the saved video display unit (1130) based on the same search conditions (S1830). At this time, whether or not the group has been disbanded is also checked, and the time and location of the group's disbandment can also be recorded as important detection information. Finally, the system integrates the detection results from real-time video and saved video and displays them on the detection result display unit (1160) (S1840). The integrated detection results are organized and displayed in groups, and the group joining and dismantling points may be specially highlighted.

[0026] Figure 19 illustrates an example of an execution screen for a system in which information on a target group is input and a search is performed using real-time video and saved video, according to the seventh embodiment of the present invention. In Figure 19, areas requiring special attention are indicated separately with reference numbers beginning with "19". The rest of the figure is the same as in Figure 13. Refer to the box with reference number 1911. The search condition setting section (1110) has detailed conditions set for a bidirectional subject group search. In the search direction (1111), "bidirectional" is selected to search both real-time video and saved video together. In the subject information (1112), multiple subjects are selected, and information for subject 1 and subject 2 has been entered (1911). In particular, the group settings (1113) were set to a distance of "5m" between groups and a duration of "30 seconds" (1913). This means that two subjects will be recognized as a single group when they are moving together for 30 seconds or more at a distance of 5m or less. The similarity criteria (1114) and search area (1115) were maintained at their default settings. The real-time video display unit (1120) displays the current video feed from the CCTV selected according to the set conditions, played back in real time. In the real-time video feed from CCTV 3 (1123), a group of subjects similar to the search target group was detected (1921). These subjects satisfy the group setting conditions and are displayed as Group 1 (1941) in the real-time video search unit (1140). The archived video display unit (1130) plays back footage from a predetermined time prior to the set search time (09:20) for each CCTV (e.g., from 08:50, which is 30 minutes prior). In the footage from CCTV 3 (1133) and CCTV 5 (1135), individuals similar to the search target group were detected (1933, 1935), and these were displayed as Group 1 (1951, 1952) in the archived video search unit (1150). The detection result display unit (1160) comprehensively analyzes and provides information on the group detected from real-time video and saved video. While subject images (1161) can be displayed in various ways, an example is shown where images of all group members are displayed (1961), and subject attributes (1162) may provide individual characteristics of each member. Although not shown, group attributes may also be provided in some cases. As illustrated in Figure 13, a detection list (1163), a detection timeline (1164), and a subject detection map (1165) are provided. These may also display the group's movement path. Although not shown, points where subjects in a group are separated or joined may be particularly highlighted. Through this, the formation and dissolution points of groups, and their movement paths within the group, can be intuitively understood. This type of group-based, bidirectional search allows for searches that consider the relationships between multiple individuals, providing highly effective information, particularly in missing person searches, to determine the presence or absence of companions and the point of separation. It can also be used in criminal investigations to identify accomplices and their connections.

[0027] Figure 20 is a flowchart illustrating a method for searching for a group of subjects selected in real-time video using stored video, according to the eighth embodiment of the present invention. First, real-time video of the location specified via the search condition setting unit (1110) is displayed on the real-time video display unit (1120) (S2010). At this time, the real-time video displayed may be CCTV video of the location and time set by the user via the search condition setting unit (1110). Next, a large number of subjects are automatically detected in the real-time video displayed on the real-time video display unit (1120) through an artificial intelligence-based subject detection algorithm, and the detected subjects are displayed on the real-time video search unit (1140) (S2020). Each detected subject may be displayed in thumbnail form, and the subject's identifier, detection time, CCTV number, etc., may be provided together. Next, the user can select multiple subjects from the detected subjects displayed in the real-time video search unit (1140) and designate them as a single group (S2030). Next, the system analyzes the characteristics of the individuals designated as a group by the user and uses an artificial intelligence-based object search algorithm to detect the group of individuals in the saved video displayed on the saved video display unit (1130) (S2040). At this time, the search is performed within a pre-specified time period and geographical range, and the detected results are displayed in thumbnail form on the saved video search unit (1150). Finally, the system integrates the detection results from the real-time video and the saved video and displays them on the detection result display unit (1160) (S2050).

[0028] Figure 21 illustrates an example of an execution screen for a method according to the eighth embodiment of the present invention, which searches for a group of subjects selected in real-time video using saved video. In Figure 21, parts requiring special attention are indicated separately with reference numbers beginning with "21". The rest of the content is the same as in Figure 15. Refer to Figure 21. The search condition setting unit (1110) has basic settings for searching real-time video in conjunction with saved video. In the search direction (1111), "Real-time ==> Saved" is selected. The subject information (1112) and group setting (1113) items are blank, because in the 8th embodiment, the user does not pre-enter subject or group conditions, but instead selects a group from the subjects automatically detected in the real-time video. In the real-time video display unit (1120), subject 1 and subject 2 are automatically detected in the video from CCTV 3 (1123) based on artificial intelligence (2123-1, 2123-2), and another subject 3 is also detected in CCTV 4 (1124) (2124). The detected subjects are displayed as individual thumbnails (2141, 2142, 2143) in the real-time video search unit (1140). Assume that the user has selected Subject 1 (2141) and Subject 2 (2142) together from the subjects displayed in the real-time video search unit (1140) and designated them as a single group. In this case, the user can designate subjects as a group in the real-time video search unit (1140) in the following various ways. One possible selection method involves using keyboard combinations. For example, a user can select multiple subjects by holding down the Control key and sequentially clicking the thumbnails of Subject 1 (2141) and Subject 2 (2142). Alternatively, by using the Shift key to select the first and last subjects, all subjects in between can be selected at once. Another selection method utilizing mouse drag is also possible. For example, if a user drags the mouse in the real-time video search unit (1140) to specify an area containing thumbnails of multiple subjects, all subjects within that area will be selected as a group. Another method is to utilize the system's automated group recommendation function. The system can analyze the location, movement patterns, and external characteristics of individuals detected in real-time video and automatically recommend individuals who are likely to be grouped together. Users can then select one of these recommended groups or modify its members as needed. Another method of selection is through the context menu. By right-clicking on a specific person's thumbnail, a menu appears in which the "Create Group" option can be selected. After selecting this option, additional people to include in the group can be chosen. Refer to Figure 21 again. The saved video display unit (1130) plays and displays the saved videos (1131-1136) corresponding to each CCTV on the real-time video display unit (1120). The saved videos are played from 08:55, which is 30 minutes prior to the time set as exemplified from when the user selected the group (09:25). The system detects the group selected by the user in the saved video footage (1131-1136). In Figure 21, the target group is detected in the video footage of CCTV 2 (1132) and CCTV 3 (1133) (2132, 2133), and this is displayed as Group 1 (2154, 2155) in the saved video search unit (1150). The detection result display unit (1160) comprehensively analyzes and provides information on the groups detected from real-time video and saved video. The subject images (1161) may display images of the subjects constituting the group in various ways (2161), and the subject attributes (1162) provide the characteristics of each member. Although not shown in the diagram, group attributes may also be provided in some cases. As explained in Figure 15, a detection list (1163), a detection timeline (1164), and a subject detection map (1165) are provided to display the movement paths of the subject group. In particular, the subject detection map (1165) shows the movement paths of the group from 09:15, when it was first discovered, to the present time of 09:25, using arrows. This method allows multiple individuals discovered during real-time monitoring to be designated as a group, and their past movements can be tracked. This can be particularly useful for immediately verifying the past movements of a suspect group discovered in real time, or for understanding the past travel routes of a missing person and their companions.

[0029] Figure 22 is a flowchart illustrating a method for searching for a group of subjects selected from stored video footage using real-time video footage, according to the ninth embodiment of the present invention. Refer to Figure 22. The search method of the ninth embodiment includes the following steps. First, the saved CCTV footage set through the search condition setting unit (1110) is played back and displayed on the saved footage display unit (1130) (S2210). At this time, the saved footage displayed may be saved CCTV footage from a location and a specific past point in time set by the user through the search condition setting unit (1110). Next, as the stored video in the stored video display unit (1130) is played back, a large number of subjects may be automatically detected through the object detection algorithm of the artificial intelligence base, and the detected subjects will be displayed in the stored video search unit (1150) (S2220). Each detected subject will be displayed in thumbnail form, and the CCTV number may be provided along with the subject's identifier, the time of detection, etc. Next, the user selects multiple subjects from those displayed in the saved video search unit (1150) and designates them as a single group (S2230). Next, the system analyzes the characteristics of the individuals designated as a group by the user and uses an artificial intelligence-based object search algorithm to detect the group of individuals in the real-time CCTV video displayed on the real-time video display unit (1120) (S2240). The detected results are displayed as thumbnails on the real-time video search unit (1140). Finally, the system integrates the detection results from the saved video and the real-time video and displays them on the detection result display unit (1160) (S2250). The integrated detection results are organized and displayed by group, and the movement paths of the groups are visualized.

[0030] Figure 23 illustrates an example of an execution screen for a method that searches for a group of subjects selected from saved video footage using real-time video footage, according to the ninth embodiment of the present invention. In Figure 23, parts requiring special attention are indicated separately with reference numbers beginning with "23". The rest of the content is the same as in Figure 17. Refer to Figure 23. The search condition setting unit (1110) has basic settings for searching saved videos in conjunction with real-time videos. In the search direction (1111), "Saved ==> Real-time" is selected. The subject information (1112) is left blank (2310), because in the ninth embodiment, the user does not pre-enter any conditions for subjects or groups, and instead selects a group from the subjects automatically detected in the saved videos. The saved video display unit (1130) displays the saved video from the selected CCTV. As the saved video is played back, subject 1 and subject 2 are detected in the video from CCTV 3 (1133) based on artificial intelligence (2333-1, 2333-2), subject 1 and subject 2 are detected again in the video from CCTV 4 (1134) (2334-1, 2334-2), and subject 3 is detected in the video from CCTV 6 (1136) (2336). The detected subjects are displayed as individual thumbnails (2351, 2352, 2353, 2354, 2355) in the saved video search unit (1150). It is assumed that the user has selected Subject 1 (2351) and Subject 2 (2352) together from the subjects displayed in the saved video search unit (1150) and designated them as a single group. Such group designation can be performed in various ways as explained in Figure 21. The real-time video display unit (1120) plays and displays the real-time video (1121-1126) of each CCTV corresponding to the saved video (1131-1136) of each CCTV in the saved video display unit (1130). In the video of CCTV 5 (1125), a group of subjects similar to the selected subject group is detected (2325), and this is displayed as group 1 (2341) in the real-time video search unit (1140). The detection result display unit (1160) comprehensively analyzes and provides information on the groups detected from the saved video and real-time video. The subject image (1161) displays images of the subjects that make up the group (2361), and the subject attributes (1162) may provide the characteristics of each member individually. Although not shown in the diagram, group attributes may also be provided in some cases. Additionally, the detection list (1163), detection timeline (1164), and subject detection map (1165) display the movement paths of the subject groups. In particular, the subject detection map (1165) shows the movement paths of the group from 09:15, when they were first discovered, to the current time of 09:25, using arrows. This method allows multiple individuals identified in archived video footage to be designated as a group, and their current locations can be tracked in real time. This can be particularly useful for quickly confirming the current location of a group of suspects identified in archived video footage, or for determining the current location of a missing person and their companions.

[0031] Figure 24 is a block diagram showing the internal configuration of a real-time and archived video integrated object search device according to one embodiment of the present invention. Refer to Figure 24. This system includes a processor (2410), memory (2420), and a communication unit (2430). The processor (2410) controls the overall operation of the system, particularly enabling object retrieval by processing real-time and stored video through a single integrated interface. As illustrated in Figures 11 to 23, the processor (2410) performs the process of searching based on user-inputted object criteria, or searching for objects (or groups of objects) automatically detected in real-time or stored video, if selected by the user, in other video. The processor (2410) may include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an NPU (Neural Processing Unit), with the GPU and NPU being particularly useful for accelerating artificial intelligence-based tasks such as real-time object detection, feature extraction, and object re-identification. Memory (2420) stores program code and data. Specifically, the execution code of the application providing the integrated interface, weight data for the artificial intelligence model for object detection and re-identification, and search history data are stored in memory (2420). User-entered object conditions, selected object feature information, group setting information, and search result data are also temporarily stored. Memory (2420) can be implemented using various types of storage devices such as RAM, ROM, and flash memory. The communications unit (2430) is responsible for data communication with the server (1020). All data communication, including real-time video reception, saved video request / reception, and search request / response, is conducted through the communications unit (2430). The communications unit (2430) includes wired communication (Ethernet, etc.) and wireless communication (Wi-Fi, LTE, 5G, etc.) interfaces to enable stable transmission and reception of video data in various network environments. The various embodiments of the present invention described above can be provided to an electronic device to be executed by a processor in the state of an application that is implemented in a computer-executable program code and stored on various non-transitory computer-readable media. The aforementioned non-temporary readable media refers to media that store data semi-permanently and can be read by devices, rather than media that store data for short periods, such as registers, caches, and memory. Specifically, the various applications or programs mentioned above may be stored and provided on non-temporary readable media such as CDs, DVDs, hard disks, Blu-ray discs, USB drives, memory cards, and ROMs. Although the invention has been described above with reference to the drawings and embodiments, this does not mean that the scope of protection of the present invention is limited by the drawings or embodiments. A person skilled in the art will understand that the invention can be modified and altered in various ways without departing from the spirit and scope of the invention as described in the following claims. [Explanation of symbols]

[0032] 910 Processor 920 memory 2410 Processor 2420 memory 2430 Communications Department

Claims

1. A method for detecting the characteristics of an object from an image, The steps include: obtaining the detection region containing the object from the video, A step of acquiring external appearance characteristic information of an object within the detection area using an artificial intelligence model, The step of obtaining state verification information of the aforementioned object, Using the aforementioned state verification information, the step of identifying variable characteristic information from the aforementioned appearance characteristic information that can be changed, The step of generating final characteristic information based on the aforementioned variable characteristic information includes, The final characteristic information includes, as auxiliary information, information from the variable characteristic information that has a reliability of a predetermined standard value or higher, or that has a correlation of a predetermined threshold or higher. A method for detecting the characteristics of an object.

2. In claim 1, The aforementioned status verification information is, A method for detecting the characteristics of an object, comprising at least one of the object's pose vector value, the object's geometric features, and object segmentation information.

3. In claim 1, The aforementioned final characteristic information is, A method for detecting the characteristics of an object, which includes the appearance characteristic information but does not include the variable characteristic information.

4. In claim 2, If the verification information includes the attitude vector value, A method for detecting the characteristics of an object, wherein the variable characteristic information is characteristic information for a body part whose existence cannot be confirmed from the posture vector value.

5. In claim 4, The aforementioned posture vector value includes vector values ​​for each part of the body. A method for detecting the characteristics of an object, which generates the final characteristic information without including the characteristics of a specific body part if no vector value is detected for that body part.

6. In claim 2, If the verification information includes the geometric features and object segmentation information, The process further includes the step of determining a reference object from among a plurality of objects within the detection region, A method for detecting the characteristics of an object, wherein the variable characteristic information is characteristic information of a superimposed region that includes the characteristics of an object other than the reference object.

7. In claim 6, The aforementioned reference object is, A method for detecting the characteristics of an object, which is determined based on the area it occupies within the detection region and the distance between the center point of the detection region and the center point of the object.

8. In claim 7, The aforementioned reference object is, A method for detecting the characteristics of an object, wherein the object is the one with the largest area and the shortest distance among the aforementioned plurality of objects.

9. In claim 6, The object segmentation information is obtained through an object segmentation (instance segmentation) algorithm. A method for detecting the characteristics of an object, wherein the overlapping regions between the plurality of objects are identified through the object division information.

10. In claim 9, A method for detecting the characteristics of an object, wherein the characteristics of the superimposed region are not included in the final characteristic information.

11. In claim 9, A method for detecting the characteristics of an object, wherein, among the characteristics of the superimposed region, those characteristics that have a similarity to the characteristics of the reference object that is equal to or greater than a preset reference value are included in the final characteristic information.

12. In claim 1, The aforementioned video is CCTV footage. A method for detecting the characteristics of an object, wherein the object is a person in the vicinity of a site where abnormal signs, including a fire or collapse, have occurred.

13. In claim 1, The aforementioned final characteristic information is, A method for detecting the characteristics of an object, which is used as a search criterion for searching for an object that was at the scene of a crime, based on the time when abnormal signs, including fire or collapse, were detected.

14. In claim 1, The aforementioned final characteristic information is, A method for detecting the characteristics of an object, which is used as query information for object retrieval in a system that integrates real-time video and stored video for searching.

15. A device for detecting the characteristics of an object from an image, An object detection unit that acquires a detection area containing an object from the video, An appearance characteristic acquisition unit that acquires appearance characteristic information of an object within the detection area using an artificial intelligence model, A state verification information acquisition unit that acquires state verification information of the aforementioned object, A variable characteristic identification unit identifies variable characteristic information that can be changed from the appearance characteristic information using the state verification information, Includes a final characteristic information generation unit that generates final characteristic information based on the variable characteristic information, The final characteristic information includes, as auxiliary information, information from the variable characteristic information that has a reliability of a predetermined standard value or higher, or that has a correlation of a predetermined threshold or higher. Object characteristic detection device.

Images