Control system, method, and computer program for monitoring subject

Abstract

A control system for monitoring a subject is disclosed. The present system comprises: a setting unit that sets, on the basis of interest information about a subject, a camera for acquiring an image of the subject; an activity information generation unit that generates activity information of the subject including an activity image of the subject by analyzing the image received from the camera on the basis of a photo of the subject included in the interest information; and a subject state determination unit that detects whether or not an abnormal activity state of the subject occurs, on the basis of the number of detections of the activity image of the subject included in the activity information of the subject.

Description

Control system, method and computer program for monitoring subjects

[0001] The present invention relates to a control system, method and computer program capable of monitoring subjects such as criminals based on CCTV (closed circuit television) camera images.

[0002] For criminals with a high risk of repeating the same crime, such as sex offenders, there is currently a policy in place where the relevant government agency periodically collects and discloses the criminal's personal information to those around them, even after they have served their sentence and returned to their daily lives.

[0003] Details of the policy are as follows:

[0004] Current Criminal Management Policy

[0005] 1. Subject notification information: A total of 8 items of information, including name, age, resident registration address (including road name and building number), residential address (including road name and building number), physical information (height and weight), photo (front, left side, right side, full body), summary of sexual crime (date of judgment, name of crime, sentence), sexual violence criminal record, and presence of electronic device.

[0006] 2. Update cycle of target notification information

[0007] - Photo: Taken once a year when the subject appears in person at a police station / correctional facility / treatment facility.

[0008] - Other personal information: The subject must report directly within 20 days from the date the change occurred.

[0009] 3. Information collected from subjects: name, resident registration number, address on resident registration, residential address, work address, contact information, physical information (height, weight), vehicle number, etc.

[0010] 4. Renewal Procedure

[0011] - Providing information collected from the subject through attendance at the police station or reporting of changes in residence.

[0012] - The Ministry of Justice, which has received the results of verification of the authenticity and change of registration information from the competent police station, verifies the contents and then registers the changed information in the registration information book.

[0013] - When the Ministry of Justice sends public information, the Ministry of Gender Equality and Family creates a sex offender public information base and discloses the information.

[0014] 5. Related laws and regulations

[0015] - Special Act on the Punishment of Sexual Crimes (abbreviated as the Sexual Violence Punishment Act) and its Enforcement Decree

[0016] - Act on the Protection of Children and Youth from Sexual Abuse (abbreviated as the Youth Sexual Protection Act) and its Enforcement Decree

[0017]

[0018] The current criminal management policy requires criminals to voluntarily report changes in their personal information, such as changes in their residential address or workplace, to the relevant government agency.

[0019] Therefore, if a criminal intentionally or accidentally fails to report, it may be difficult for the relevant government agency to individually check and examine the changed personal information of the criminal, which may create a loophole in management to prevent the criminal from committing the same crime again during the period of failure to report.

[0020] The present invention has been devised to solve the above-described problem, and the purpose of the present invention is to provide a control system, method and computer program for monitoring a subject, which can identify the activity area of ​​a subject based on the interest information of the subject, such as a criminal, and collect and analyze the subject's video through a closed circuit television (CCTV) camera installed in the identified activity area to confirm the authenticity of the subject's activity area.

[0021] In order to achieve the above-described purpose, a control system for monitoring a subject according to an embodiment of the present invention includes a camera setting unit that sets a camera to acquire an image of the subject based on interest information about the subject, an activity information generation unit that analyzes an image received from the camera based on a photo of the subject included in the interest information to generate activity information of the subject including an activity image of the subject, and a subject status determination unit that determines whether an abnormal activity of the subject occurs based on the number of detections of activity images of the subject included in the activity information of the subject.

[0022] In addition, a method for monitoring a subject according to an embodiment of the present invention for achieving the above-described purpose includes a step of setting a camera for obtaining an image of the subject based on interest information about the subject, a step of analyzing an image received from the camera based on a photo of the subject included in the interest information to generate activity information of the subject including an activity image of the subject, and a step of detecting whether an abnormal activity state of the subject occurs based on the number of detections of activity images of the subject included in the activity information of the subject.

[0023] In addition, a computer program stored in a computer-readable recording medium according to an embodiment of the present invention for achieving the above-described purpose may include a program code for executing the method for monitoring the subject described above.

[0024] In addition, a computer-readable recording medium according to an embodiment of the present invention for achieving the above-described purpose may record a program for executing the method for monitoring the subject described above.

[0025] According to the present invention, by determining whether the criminal's actual activity area, such as his residence or workplace, is authentic, it is possible to more accurately check and track the criminal's residential activities without the criminal's voluntary reporting.

[0026] In addition, according to the present invention, by collecting and analyzing images only through CCTV cameras installed in the main living area of ​​a specific criminal rather than all CCTV cameras installed in the jurisdictional area, the amount of image analysis data can be minimized, thereby minimizing system resources required for image analysis.

[0027] In addition, according to the present invention, by providing a user interface that can compare a photo of a criminal registered with a government agency and an image collected through a CCTV camera, there is an effect that allows a control agent to intuitively check and inspect whether the control system is operating normally.

[0028] FIG. 1 is a block diagram showing a control system for monitoring a subject according to one embodiment of the present invention.

[0029] FIG. 2 is a flowchart illustrating a process for determining whether a control system according to one embodiment of the present invention is operating normally.

[0030] Figure 3 is a diagram showing target activity information according to one embodiment of the present invention.

[0031] FIG. 4 is a diagram illustrating a subject activity video verification interface according to one embodiment of the present invention.

[0032] FIG. 5 is a flowchart illustrating a process for determining the authenticity of a target activity area according to one embodiment of the present invention.

[0033] FIG. 6 is a diagram showing target activity data according to one embodiment of the present invention.

[0034] Figure 7 is a diagram showing a process of resetting a target utilization area according to one embodiment of the present invention.

[0035] FIG. 8 is a diagram illustrating an interface that displays an existing path of a subject and a new path of the subject according to one embodiment of the present invention.

[0036] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the attached drawings.

[0037] In the present invention, the 'subject' may preferably be a criminal, but is not limited thereto, and the subject may be a concept that includes all targets that are subject to monitoring.

[0038] Figure 1 is a block diagram illustrating a control system for monitoring subjects according to one embodiment of the present invention. The control system (1000) illustrated in Figure 1 may be a control system capable of collecting images from all CCTV cameras installed on roads or in living spaces, such as traffic control CCTVs and crime prevention CCTVs, and detecting and selectively controlling various events.

[0039] This control system (1000) can collect images not only from CCTV cameras directly linked to the control system (1000), but also from all CCTV control systems (hereinafter referred to as “external systems”) operated by other government agencies or companies. For example, images collected from parking lot entrance cameras in everyday life, cameras that recognize license plates installed in the city center, and fixed / mounted cameras that crack down on illegal parking can be targets of the expanded collection range. Here, the external system collects and stores information on the subject’s image, shooting time, and shooting camera installation location, and the control system (1000) can request and receive the subject’s image, shooting time, and location information from the external system.

[0040] The control system (1000) according to the present invention may include all or part of an information acquisition unit (110), a camera setting unit (120), an activity information generation unit (130), a status determination unit (140), an operation unit (150), and a storage unit (160).

[0041] The information acquisition unit (110) can acquire information of interest to the subject. Regarding offenders with a high risk of repeating the same offense, such as sex offenders, the competent government agency (e.g., the Ministry of Justice) periodically collects personal information on the offender and notifies those around them of this information. Here, information of interest may include information notified by the competent government agency regarding the subject and information collected from the subject.

[0042] At this time, the notification information may include name, age, resident registration address, residential address, physical information, photo, summary of the crime, criminal record, and whether or not an electronic device is attached.

[0043] In addition, the collected information collected from the subject according to the need for information update, such as changes in notification information, may include name, resident registration number, resident registration address, residential address, workplace address, contact information, physical information, and vehicle number.

[0044] Meanwhile, information of interest may further include additional information about the subject. Here, the additional information may include communication information of the subject's electronic devices (e.g., communication information of a criminal's attached device, such as an electronic bracelet, communication information of a criminal's portable device, such as a criminal's smartphone, etc.).

[0045] The camera setting unit (120) can set a camera to acquire an image of a subject based on the interest information acquired by the information acquisition unit (110). Here, the camera setting unit (120) can include an activity area setting unit (121) that sets an activity area of ​​the subject based on at least one of a resident registration address, a residential address, and a workplace address of the subject included in the interest information, and a camera selection unit (122) that sets a camera installed within the activity area of ​​the subject as a camera for acquiring an image of the subject according to predetermined conditions. At this time, the predetermined conditions can include at least one of the number of cameras within the activity area of ​​the subject, a camera shooting direction, and a camera installation location.

[0046] The activity information generation unit (130) can generate activity information of the subject including the activity image of the subject by analyzing the camera image received from the camera set in the camera setting unit (120) based on the photo of the subject included in the information of interest.

[0047] The status determination unit (140) can determine the status of at least one of the control system (1000) and the target based on the activity information generated by the activity information generation unit (130). For example, the status determination unit (140) may include a system status determination unit (141) and a target status determination unit (142).

[0048] The system status judgment unit (141) can determine whether the control system is operating normally based on the subject activity video included in the subject activity information generated by the activity information generation unit (130) and the subject's photo included in the interest information. Here, the subject activity video may include at least one of the subject video and the subject's vehicle video.

[0049] In addition, the subject status judgment unit (142) can judge whether an abnormal activity state of the subject has occurred based on the number of detections of the subject's activity images included in the subject's activity information.

[0050] The above-described information acquisition unit (110), camera setting unit (120), activity information generation unit (130), and status determination unit (140) may be configured as a single processor mounted and operated on a server that is connected to, for example, the operation unit (150) and the storage unit (160) in communication, or as separate processors that perform the functions of each unit, and may be built on a separate operation server together with the operation unit (150).

[0051] The operation unit (150) performs the functions of an operation center and may be constructed as a separate operation server. This operation unit (150) may include an alarm unit (151) and an interface unit (152).

[0052] The alarm unit (151) can provide operational information including the status of at least one of the system (1000) and the target determined by the status determination unit (140) in the form of an alarm or the like to users of the integrated control center or a server of an external organization (e.g., an operation server of a competent organization such as the National Police Agency or the Ministry of Justice, which are criminal-related organizations).

[0053] The interface unit (152) can provide users of the integrated control center with operational information including the status of at least one of the system (1000) and the target determined by the status determination unit (140) through an interface screen. The interface unit (152) can include a HID (Human interface device), such as a keyboard and monitor, used by users to control the operation of the computer system.

[0054] The storage unit (160) can store various information, data, programs, etc. for the operation of the control system (1000).

[0055]

[0056] Below, a control method for monitoring a subject according to one embodiment of the present invention is described in terms of the operation process of the control system.

[0057] FIG. 2 is a flowchart illustrating a process for determining whether a control system is operating normally based on subject activity information according to an embodiment of the present invention. Referring to FIG. 2 , an information acquisition unit (110) can acquire information of interest of a subject (S100). Specifically, the information acquisition unit (110) can acquire information of interest for each subject by linking with a criminal information database operated by a government agency in charge where the subject's personal information is stored. The information acquisition unit (110) can store information of interest for each subject in a database in a storage unit (160), and when the subject's personal information stored in the criminal information database is updated, the updated information can be acquired to update the information of interest stored in the storage unit (160).

[0058] Here, information of interest to the subject may include notification information and collection information. In step S100, the information acquisition unit (110) may selectively extract only pre-defined essential data, excluding personal information such as resident registration numbers, from the data included in the notification information and collection information, and create and store the information of interest. For example, the essential data may include the subject's name, photo, residential address, workplace address, and vehicle registration number.

[0059] In addition, the activity area setting unit (120) can set the activity area of ​​the subject based on at least one of the subject's resident registration address, residential address, and workplace address included in the interest information acquired through step S100 (S200). For example, the activity area setting unit (120) can set the activity area of ​​the subject as a radius centered on at least one of the subject's resident registration address, residential address, and workplace address on the control map provided through the interface unit (152). For example, the radius can be set to 200 m, etc. centered on the address, and the address can also be directly entered / edited by a user (e.g., a control agent) through the interface unit (152).

[0060] In addition, the camera selection unit (122) can select a camera installed within the target's activity area set through step S200 as a camera for acquiring an image of the target according to predetermined conditions (S300). At this time, the predetermined conditions may include at least one of the number of cameras within the target's activity area, the camera shooting direction, and the camera installation location. For example, the number of cameras may be 20 or less, the camera shooting direction may be forward or reverse, and the camera installation location may be designated according to a combination of cameras installed on a route from a residence to a bus / taxi stop / subway station / convenience store within a certain distance. Meanwhile, the above-described predetermined conditions may also be directly input / modified by the user through the interface unit (151).

[0061] And, the activity information generation unit (130) can detect an activity image of the subject by receiving an image captured by the camera selected through step S300, analyzing the photo of the subject included in the information of interest acquired by the information acquisition unit (110) through step S100 and the camera image received from the camera selected through step S300 (S400). The activity information generation unit (130) can detect an activity image of the subject by using an artificial neural network model trained to detect an activity image of the subject from an image received from a camera by learning the photo of the subject included in the information of interest acquired by the information acquisition unit (100) through S100.

[0062] Meanwhile, in order to save video analysis resources, the activity information generation unit (130) may not detect the target activity images from all images received from the cameras selected through step S300, but may only detect a predetermined number of target activity images or target activity images corresponding to a predetermined time.

[0063]

[0064] In addition, the activity information generation unit (130) can generate subject activity information based on the subject activity image detected through step S400 (S500). FIG. 3 is an example showing subject activity information according to an embodiment of the present invention, and the activity information generation unit (130) can include subject identification information (401), subject activity image (402), shooting time (403), and camera information (404, 405) as shown in FIG. 3. At this time, the subject identification information (401) is information that can identify the subject, such as a name, and can be blinded to protect personal information. In addition, the subject activity image (402) can be a frame that includes an object with the highest similarity to the subject among a plurality of consecutive frames constituting an image received from a camera. In addition, the camera information can include camera identification information (ID) (404) and location (405), etc.

[0065] Meanwhile, in order to save operating resources of the control system (1000) according to one embodiment of the present invention, the activity information generation unit (130) may generate target activity information based only on a predetermined number of target activity images or target activity images corresponding to a predetermined time, rather than all target activity images detected through step S400.

[0066] In addition, the activity information generation unit (130) can generate a modified activity image of the subject by performing 3D modeling of the facial area of ​​the activity image of the subject based on a photograph of the subject, and an artificial neural network model for subject detection can additionally learn the modified activity image of the subject to more accurately detect the activity image of the subject from the image received from the camera.

[0067] For example, the activity information generation unit (130) can generate a deformed subject activity image by performing 3D modeling based on a photograph of the subject and deforming the subject's face according to various camera angles, lighting directions, and image deformation situations. The reason for converting the image in this way is that in the case of the subject activity image, since the image is taken by a CCTV camera installed at a height of 5 to 6 m, the subject's face cannot be captured horizontally, and therefore, there is a need to resolve the problem of having to identify the subject only with the nose and chin of the face depending on the subject's actions such as wearing a hat or bowing the head. In order to solve this, the activity information generation unit (130) according to the present invention can generate subject activity information that further includes the deformed subject activity image in addition to the subject identification information, subject activity image, shooting time, and camera information illustrated in FIG. 3.

[0068] In addition, the storage unit (160) can store the subject activity information generated through step S500 (S600). At this time, the storage unit (160) can sort and store the subject activity information based on a time series such as day, week, month, etc. according to the shooting date and time.

[0069] In addition, the system status judgment unit (141) can judge whether the control system is operating normally based on the subject activity video and the subject's photo (S700). For example, the system status judgment unit (141) can receive subject identification information (e.g., name, etc.) from the user through the interface unit (152), retrieve the subject activity video and subject interest information matching the input subject identification information from the storage unit (160), and display the result through the subject activity video verification interface (500) as shown in FIG. 4 through the interface unit (152).

[0070] At this time, the subject activity video verification interface (500) may include a subject interest information display area (520) including a subject photo (521), a subject activity video display area (510) including a subject activity video (512), a total activity video count display area (531), a matching count display area (532), a matching rate display area (533), and a user operation area (540).

[0071] The user operation area (540) may include a target identification information input area (541) to be searched, a search period input area (542), and various optional function operation areas (not shown).

[0072] Each image (512) displayed in the subject activity image display area (510) may include a check box (511) that allows the user to check whether it is authentic, and the user may intuitively and simultaneously compare the subject photo (521) included in the subject interest information display area (510) and the subject activity image (512) to check whether the subject activity image is authentic (511). At this time, the input value through the check box (511) may constitute verification information.

[0073] Accordingly, the user can check the checkbox (511) of the matching target activity video while comparing the target activity video (512) with the target photo (521) of the target interest information, and the interface unit (152) can display the number of checked target activity videos in the matching number display area (532), and calculate the ratio of the number of matching videos based on the total target activity videos and display it in the matching rate display area (533).

[0074] In this case, the system status judgment unit (141) can judge whether the control system (1000) according to the present invention is operating normally based on the number of matches or the size of the match rate at step S700. For example, in a situation where there are a predetermined number or more of target activity videos (512) and the number of matches or the match rate with the target photo (521) is less than a predetermined value (e.g., 1%), the control system (1000) according to one embodiment of the present invention, particularly the activity information generation unit (130), may be judged to be operating abnormally due to a failure in the system.

[0075] In addition, the user can estimate that the subject's activity area has been changed to a place other than the address on the resident registration, residential address, and workplace address by checking the number of matches or the size of the match rate displayed on the subject activity video verification interface (500).

[0076] Alternatively, the system status determination unit (141) may provide an input interface (not shown) together with the activity image verification interface (500) through which the user can directly determine whether the control system (1000) according to an embodiment of the present invention is operating normally after checking the activity image verification interface (500) displayed through the interface unit (152). For example, if a situation arises where the user suspects a malfunction of the control system (1000), such as when the activity image (511) of the subject displayed in the activity image display area (510) is different from the subject photo, the user may input an abnormal operation of the system (1000), and on the other hand, if it is determined that the activity image of the subject matches the subject photo by a certain percentage or more, the control system (1000) may input a normal operation.

[0077] Meanwhile, as illustrated in FIG. 4, unlike the method in which the user intuitively and simultaneously compares the subject photo (521) and the subject activity video (512) included in the subject interest information display area (510) and checks (511) the authenticity of the subject activity video to determine whether the control system (100) is operating normally (i.e., the method in which the control system (100) is operating normally based on the user's manual input), the system status determination unit (141) may be implemented to automatically determine the number of matches or the size of the match rate based on the subject activity video and the subject's photo without the user's manual input to determine whether the control system (1000) is operating normally.

[0078] According to this invention, by providing a user interface that can compare a photo of a criminal registered with a government agency and an image collected through a CCTV camera, there is an effect that allows a control agent to intuitively check and inspect whether the control system is operating normally.

[0079] Meanwhile, if the system status judgment unit (141) determines that the system is operating abnormally, the abnormal operation of the control system (1000) can be notified (S710). For example, the operation unit (150) can display a system abnormal operation notification message to the user through the interface unit (152) or provide an alarm to the user through the alarm unit (151).

[0080] In this case, the user can resolve the malfunction so that the system operates normally, such as by improving the performance through learning of the target detection artificial neural network model equipped in the activity information generation unit (130) of the system (1000).

[0081] However, if the system status judgment unit (141) determines that the system is operating normally, a process for judging the authenticity of the actual activity area of ​​the subject as shown in Fig. 5 can be performed.

[0082]

[0083] FIG. 5 is a flowchart illustrating a process for determining the authenticity of a subject's actual activity area based on the subject's activity status data according to one embodiment of the present invention. Referring to FIG. 5, the subject status determination unit (142) generates subject activity status data by calculating the number of detections of the subject's activity images included in the subject's activity status data for a predetermined period of time, and stores the generated subject activity status data in the storage unit (160) (S800).

[0084] FIG. 6 is a diagram showing subject activity status data according to one embodiment of the present invention, and the subject activity status data may include subject identification information (601), activity status information (602), and information on the number of activity images detected per predetermined period (e.g., the number of activity images detected per day from July 1 to July 11, the total, average, previous month average, etc.) (603).

[0085] Since step S800 is before the activity status information of the subject is analyzed, the activity status information (602) may have no value in step S800.

[0086] In addition, the subject status determination unit (142) can analyze the generated subject activity status data to determine whether abnormal activity of the subject has occurred (S900). For example, the subject status determination unit (142) can determine that abnormal activity of the subject has occurred if the number of detected activity images of the subject remains below a predetermined number for a predetermined period of time by analyzing the subject activity status data stored through step S800, or if the average number of detected activity images of the subject is below a predetermined value for a predetermined period of time.

[0087] For example, referring to FIG. 6, it can be confirmed that for the subject "Park xx", the subject's activity video was detected 0 times from July 4 to July 7 and from July 9 to 11, and the subject's activity video was detected once on July 8. At this time, when analyzing based on past data, it is highly likely that the 1 case detected on July 8 occurred due to a system error, etc. Therefore, the subject status determination unit (142) can determine that abnormal activity has occurred for the subject "Park xx", if the number of detected activity videos remains at "0" for 3 days or more or the average for 7 days or more is 50% of the average of the previous cycle or the previous month. At this time, since it is more accurate to determine abnormal activity based on the trend, the average may preferably be a daily moving average.

[0088] Meanwhile, step S900 may include a step of displaying a notification of abnormal activity of the subject through the interface unit (152) or notifying a relevant agency (e.g., an operation server of a competent agency such as the National Police Agency or the Ministry of Justice) through the alarm unit (151).

[0089] According to this invention, by determining whether the criminal's actual activity area, such as his residence or workplace, is authentic, it is possible to more accurately check and track the criminal's residential activities without the criminal's voluntary reporting.

[0090] And, if the occurrence of abnormal activity of the subject is detected through step S900, the activity information generation unit (130) can check whether the activity image of the subject is detected by using at least one of the subject's photo, vehicle number, and additional information included in the subject's interest information from the image collected through the camera installed in an area other than the activity area set in advance through the activity area setting unit (121) through step S200 during a predetermined period of abnormal activity (S1000). Here, the activity image of the subject detected in step S1000 may be a concept that further includes the subject's vehicle.

[0091] And, in the S1000 step, the search priority to check whether the target activity video is detected in another area can be set to the subject's photo as the first priority, the vehicle number as the second priority, and additional information as the third priority.

[0092] Specifically, referring to FIG. 6, the predetermined period during which the subject's abnormal activity continued can be set to "July 4 to July 11," and the activity information generation unit (130) can check whether an activity video of the subject is detected using a photo of the subject among the videos collected through cameras installed in other areas. If the video of the subject is not detected using the photo of the subject during the predetermined period during which the subject's abnormal activity continued, the activity information generation unit (130) can check whether an image of the subject's vehicle is detected among the videos collected through cameras installed in other areas using the vehicle number among the information of interest.

[0093] If the subject's vehicle image is not detected using the license plate number during a predetermined period of time during which the subject's abnormal activity continued, the activity information generation unit (130) can receive the subject's location information using the subject's electronic device communication information among the subject's interest information, and can determine whether the subject's activity is detected in another area based on the subject's location information. For example, the time information and location information of the communication information of the subject's electronic bracelet or smartphone can be received, and the subject's activity information can be detected and determined using the camera image of the corresponding location information area.

[0094] As another example, the activity information generation unit (130) can receive time information and location information in communication information such as the subject's email and card usage, and detect and determine the subject's activity information using camera images of the corresponding location information area.

[0095] Meanwhile, the activity information generation unit (130) may include a step (S1100) of generating activity information of the subject based on the activity image of the subject detected in the other area when the activity image of the subject is detected in the other area.

[0096] Meanwhile, the activity area setting unit (121) can reset the target activity area based on the installation location information of the camera that captured the target activity image included in the target activity information generated through step S1100 (S1200).

[0097] In this case, the interface unit (152) can display the installation location information of the camera that captured the subject activity video included in the subject activity information on the control map. For example, the activity area setting unit (121) can reset an area within a certain range (702) that includes the installation locations (701) of multiple cameras that captured the subject vehicle video as the subject activity area, as shown in FIG. 7.

[0098] Additionally, the alarm unit (151) may provide a notification to inform the user that the subject is active in another area, or may notify a relevant agency (e.g., an operation server of a competent agency such as the National Police Agency or the Ministry of Justice).

[0099] Meanwhile, according to one embodiment of the present invention, if the occurrence of abnormal activity of the subject is detected through step S900, the interface unit (152) can display a screen that can compare the movement path of the subject's vehicle before the occurrence of the abnormal event and the movement path of the subject's vehicle after the occurrence of the abnormal event. At this time, the movement path of the vehicle can be determined using the subject's vehicle number from images collected through cameras installed in other areas as well as the activity area of ​​the subject set through step S200.

[0100] For example, referring to FIG. 8, the interface unit (152) provides the existing main route (801) of the subject's vehicle and the changed main route (802) of the subject's vehicle after the occurrence of the abnormal event, through which the user can intuitively confirm that the subject's main activity location has changed from the residence (803) to a new area of ​​interest (804), and can estimate the subject's changed expected activity location and movement route based on the changed area (804). In addition, the user can estimate the situation in which the subject moved to the residence, disguised himself at the residence, and is not using the activity or registered vehicle.

[0101]

[0102] The present invention has been described so far, focusing on preferred embodiments thereof. All embodiments and conditional examples disclosed herein are intended to assist those skilled in the art in understanding the principles and concepts of the present invention. Those skilled in the art will appreciate that the present invention can be implemented in modified forms without departing from its essential characteristics.

[0103] Therefore, the disclosed embodiments should be considered illustrative rather than restrictive. The scope of the present invention is determined by the claims, not the foregoing description, and all differences within the scope equivalent thereto should be construed as being encompassed by the present invention.

[0104] Meanwhile, the methods for the various embodiments of the present invention described above can be implemented as programs and provided to servers or devices. Accordingly, each device can access the server or device where the program is stored and download the program.

[0105] In addition, the methods according to the various embodiments of the present invention described above may be implemented as programs and stored and provided on various non-transitory computer-readable media. A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specifically, the various applications or programs described above may be stored and provided on non-transitory computer-readable media, such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, or ROM.

[0106] In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and various modifications can be made by a person having ordinary skill in the art to which the present invention pertains without departing from the gist of the present invention claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present invention.

Claims

1. A camera setting unit that sets a camera to acquire an image of a subject based on information of interest in the subject; An activity information generation unit that generates activity information of the subject, including an activity image of the subject, by analyzing an image received from the camera based on a photo of the subject included in the interest information; and A control system for monitoring a subject, including a subject status determination unit that determines whether abnormal activity of the subject has occurred based on the number of detections of activity images of the subject included in the activity information of the subject.

2. In paragraph 1, The above information of interest is: A control system for monitoring a subject, characterized in that it includes the subject's photo, name, physical information, contact information, resident registration address, residential address, workplace address, and vehicle number.

3. In paragraph 1, The above camera settings section, A control system for monitoring a subject, characterized in that it includes an activity area setting unit that sets the activity area of ​​the subject based on at least one of the subject's resident registration address, residential address, and workplace address included in the above interest information.

4. In paragraph 3, The above camera settings section, A control system for monitoring a subject, characterized in that it further includes a camera selection unit that selects a camera installed within the activity area of ​​the subject as a camera for obtaining an image of the subject according to predetermined conditions.

5. In paragraph 4, The above conditions are, A control system for monitoring a subject, characterized in that it includes at least one of the number of cameras within the subject's activity area, the camera shooting direction, and the camera installation location.

6. In paragraph 1, The above activity information generation unit, A control system for subject monitoring characterized in that it learns a photograph of the subject and detects an activity image of the subject from an image received from the camera.

7. In paragraph 6, The above activity information generation unit, A control system for subject monitoring characterized in that it generates activity information of the subject including identification information of the subject, activity video of the subject, shooting time, and camera information.

8. In paragraph 1, The activity video of the above subject is, A control system for monitoring a subject, characterized in that the frame includes an object with the highest similarity to the subject among a plurality of consecutive frames constituting an image received from the camera.

9. In paragraph 6, The above activity information generation unit, A control system for subject monitoring characterized in that it learns by creating a modified subject activity image by performing 3D modeling of the facial area among the subject's activity images based on a photograph of the subject.

10. In paragraph 1, A control system for subject monitoring, characterized in that it further includes a system status judgment unit that judges whether the control system is operating normally based on the subject activity video and the subject's photo.

11. In paragraph 10, A control system for subject monitoring, characterized in that it further includes an interface unit that displays an activity image verification interface that can verify the authenticity of the subject's activity image based on the subject's photo and the subject's activity image included in the subject's interest information.

12. In paragraph 11, The above activity video verification interface is, A control system for subject monitoring, characterized in that it includes a check box that can compare the subject photo and the subject activity video and check the authenticity of the subject activity video.

13. In paragraph 12, The above system status judgment unit, A control system for subject monitoring characterized in that it determines whether the control system is operating normally based on the number of matches or the size of the match rate between the subject photo and the subject activity video according to the input value through the check box.

14. In paragraph 3, The above subject status judgment unit is, A control system for monitoring a subject, characterized in that it is determined that abnormal activity of the subject has occurred when the number of detections of activity images of the subject included in the activity status data of the subject remains below a predetermined number for a predetermined period of time or when the average number of detections of activity images of the subject is below a predetermined value for a predetermined period of time.

15. In paragraph 14, The above activity information generation unit, A control system for monitoring a subject, characterized in that it determines whether a video of a subject's activity is detected by using at least one of the subject's photo, vehicle number, and additional information among the subject's interest information from a video collected through a camera installed in an area other than a preset activity area during a predetermined period of abnormal activity.

16. In paragraph 15, The above additional information is: Contains communication information of the electronic device of the subject, A control system for monitoring a subject, characterized in that the search priority of the additional information in the other area is given priority to the vehicle number compared to the communication information of the electronic device of the subject.

17. In paragraph 15, The above activity information generation unit, A control system for subject monitoring, characterized in that when an activity image of a subject is detected in the above other area, subject activity information is generated based on the activity image of the subject detected in the above other area.

18. In paragraph 17, The above activity area setting section is, A control system for monitoring a subject, characterized in that the activity area of ​​the subject is reset based on the installation location information of a camera that captured the activity image of the subject detected in the above-mentioned other area.

19. In paragraph 17, A control system for monitoring a subject, characterized in that it further includes an interface unit that displays the installation location information of a camera that captured an activity image of a subject detected in the above-mentioned other area on a control map.

20. In paragraph 15, A control system for monitoring a subject, characterized in that it further includes an interface unit that displays on a screen the movement path of the subject's vehicle before the occurrence of an event for the above-mentioned abnormal activity and the movement path of the subject's vehicle after the occurrence of an event for the above-mentioned abnormal activity.

21. In a method for monitoring subjects, A step of setting a camera to acquire an image of a subject based on interest information about the subject; A step of generating activity information of the subject including an activity image of the subject by analyzing an image received from the camera based on a photo of the subject included in the interest information; and A method for monitoring a subject, comprising: a step of detecting whether an abnormal activity state of the subject has occurred based on the number of detections of activity images of the subject included in the activity information of the subject.

22. A computer program stored on a computer-readable recording medium and including a program code for executing the method for monitoring a subject described in paragraph 21.

23. A computer-readable recording medium having recorded thereon a computer program for executing the method for monitoring subjects described in Article 21.

Images