Crime prevention systems, crime prevention methods, and computer-readable recording media
By utilizing Person Re-Identification and a neural network model, the crime prevention system achieves accurate location and alerts for suspicious actions in surveillance camera images, solving the problem of difficulty in identifying suspicious actions in existing technologies and improving the crime prevention capabilities of surveillance systems.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2022-12-09
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies struggle to effectively identify suspicious actions by individuals in surveillance camera footage, especially those involving changes in appearance such as clothing, which increases the difficulty of crime prevention.
The crime prevention system uses multiple camera images captured sequentially in a time sequence. It tracks and processes the area of a person using Person Re-Identification technology, and compares template image data with a neural network model to determine suspicious behavior, including reliability assessment and occlusion detection, and outputs alarm information.
It can accurately identify and alert on suspicious activities, improving the effectiveness of surveillance systems in crime prevention.
Smart Images
Figure CN116259000B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to crime prevention systems, methods for preventing crime, and techniques for using computer-readable recording media containing crime prevention procedures to identify suspicious behavior of persons captured in camera images. Background Technology
[0002] Patent Document 1 discloses technology related to an image publishing system for publishing images captured by a surveillance camera or similar device. This image publishing system includes a camera and a publishing device. The publishing device includes a personal authentication database. The publishing device determines the person (e.g., a child) captured by the camera by referring to the personal authentication database containing data from the camera images. The data of the determined person, along with the camera image data, is transmitted to a receiving device via a network.
[0003] Existing technical documents
[0004] Patent documents
[0005] Patent Document 1: Japanese Patent Application Publication No. 2009-225398 Summary of the Invention
[0006] Consider using camera images of people captured by surveillance cameras, etc., for crime prevention. There is a possibility that these camera images may contain individuals engaging in suspicious activities. One example of suspicious activity could be an action such as suddenly changing appearance to evade surveillance. If such suspicious activities can be identified from camera images captured by surveillance cameras, it could be useful for crime prevention.
[0007] This disclosure was made in view of the above-mentioned problems, and its purpose is to provide a technique that can identify suspicious behavior of a person being filmed by a surveillance camera.
[0008] To achieve the aforementioned objectives, this disclosure provides a crime prevention system that monitors the actions of individuals contained in multiple camera images captured sequentially in a time sequence by a surveillance camera. The crime prevention system includes: a memory for storing multiple camera images; and a processor for processing the multiple camera images stored in the memory. The processor is configured to perform: tracking processing, detecting person regions within the multiple camera images respectively, and identifying individuals contained in the multiple camera images in a time sequence based on person image data contained in the person regions; and suspicious action determination processing, determining that individuals suspected of suspicious action are contained in the camera images if the identification based on the tracking processing fails midway.
[0009] In the crime prevention system disclosed herein, multiple camera images may be included, including a first camera image and a second camera image following the first camera image. In the memory, template image data containing person image data of the person region of the first camera image is stored. This is configured to enable tracking processing to identify the person in the second camera image by comparing the person image data contained in the person region of the second camera image with the template image data using a neural network model.
[0010] In addition, in the crime prevention system disclosed herein, the processor may also be configured to prohibit the determination and processing of suspicious actions in the obscured area when the area of a person is obscured.
[0011] In addition, in the crime prevention system disclosed herein, the processor may also be configured to prohibit suspicious action determination processing if the identification based on tracking processing fails midway and if the identification based on tracking processing has not been continued for a predetermined time before the failure.
[0012] Furthermore, in the crime prevention system disclosed herein, the processor may also be configured to perform notification processing that sends alarm information from the output device when a person deemed to be engaging in suspicious activity is included in the camera image.
[0013] Furthermore, to achieve the aforementioned objectives, this disclosure provides a crime prevention method in which a computer monitors the actions of individuals contained in multiple camera images captured sequentially in a time sequence by a surveillance camera. The computer is configured to detect individual regions within the multiple camera images, perform tracking processing to identify individuals within the multiple camera images in a time sequence based on image data of individuals contained within those regions, and determine that individuals exhibiting suspicious behavior are present in the camera images if the identification based on the tracking processing fails midway.
[0014] Furthermore, to achieve the aforementioned objectives, this disclosure applies to a computer-readable recording medium containing a crime prevention program that enables a computer to monitor the actions of individuals contained in multiple camera images captured sequentially in a time sequence by a surveillance camera. The crime prevention program is configured to enable the computer to: detect individual regions within the multiple camera images, perform tracking processing to identify individuals within the multiple camera images in a time sequence based on image data of individuals contained within those individual regions, and determine that individuals exhibiting suspicious behavior are contained within the camera images if the identification based on the tracking processing fails midway.
[0015] According to the technology disclosed herein, it is possible to identify suspicious activity contained in the camera images of one or more surveillance cameras. Therefore, the camera images of one or more surveillance cameras can be used for crime prevention. Attached Figure Description
[0016] Figure 1 This is a structural diagram of a crime prevention system according to an embodiment of the present disclosure.
[0017] Figure 2 This is a block diagram illustrating an example of the structure of a crime prevention system.
[0018] Figure 3 This is a block diagram illustrating the functionality achieved by executing the crime prevention program 32 by the processor 20 of the management server 10.
[0019] Figure 4 It is a graph that uses bounding box BOX1 to represent the human image data IMG1 detected from the camera image, showing the human region.
[0020] Figure 5 This is a diagram illustrating an example of a database of template image data.
[0021] Figure 6 This diagram is used to illustrate the occlusion detection process.
[0022] Figure 7 This is a flowchart illustrating the routines of the processing implemented by the crime prevention system.
[0023] (Symbol Explanation)
[0024] 2: Surveillance camera; 4: Output device; 10: Management server; 20: Processor; 30: Memory; 32: Crime prevention program; 34: Data; 100: Crime prevention system; 201: Tracking processing unit; 202: Reliability determination processing unit; 203: Obstruction detection processing unit; 204: Suspicious action determination processing unit; 205: Notification processing unit. Detailed Implementation
[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, when the number, quantity, amount, range, etc., of each element are mentioned in the embodiments shown below, the technical concept involved in the present disclosure is not limited to the mentioned quantity, unless specifically stated or explicitly determined in principle. In addition, the structures, etc., described in the embodiments shown below are not necessarily necessary for the technical concept involved in the present disclosure, unless specifically stated or explicitly determined in principle.
[0026] Implementation
[0027] 1. Overview of the crime prevention system
[0028] Figure 1 This is a structural diagram of a crime prevention system according to an embodiment of this disclosure. The crime prevention system 100 identifies suspicious individuals from camera images of one or more surveillance cameras. Suspicious individuals here refer to those who engage in suspicious activities such as changing clothing or altering their appearance in public places where surveillance cameras are installed.
[0029] The crime prevention system 100 includes one or more surveillance cameras 2, an output device 4 accessible to administrators, and a management server 10. The surveillance cameras 2 are installed, for example, inside commercial facilities or on public roads where people pass through. The surveillance cameras 2 continuously record images of passing crowds in a time sequence. There are no restrictions on the type, number, or location of the surveillance cameras 2. Figure 1 The illustration shows an example of camera images from a surveillance camera 2 that captures images of people walking on a road. The camera images captured by one or more surveillance cameras 2 are sent to a management server 10. The management server 10 can be directly connected to one or more surveillance cameras 2, or it can be remotely connected to one or more surveillance cameras 2 via a communication network.
[0030] The management server 10 performs a "tracking process" that receives camera images captured sequentially over time by one or more surveillance cameras 2 and tracks people in the camera images. This tracking process utilizes a technique called Person Re-Identification. In Person Re-Identification, firstly, regions containing people are detected from the input camera images. Hereinafter, the image information of the people contained in the detected regions is referred to as "person image data." The detected person image data is then input as a retrieval query into a neural network model used for Person Re-Identification.
[0031] Additionally, template image data of candidate destinations is input into the neural network model. Template image data is data associated with identification IDs for person images identified in past camera footage. In the neural network model, the template image data of the candidate destination that has the highest similarity to the person image data used as the search query input is output as the result. Furthermore, if no template image data of a candidate destination similar to the person image data exists, data associated with a new identification ID for that person image data is stored as part of the new template image data. The management server 10 performs Person Re-Identification-based tracking processing on the input time-series camera images, identifying people according to the time sequence.
[0032] Here, the identification of person image data based on tracking processing sometimes fails midway. In cases where tracking processing failures are frequent, the reliability of the tracking processing is first considered to be reduced. Such situations may arise, for example, if the template image data used as a candidate destination contains unsuitable images, if the camera image is unclear, or if the person has an unrecognizable appearance. The management server 10 is configured to perform a "reliability determination process" that determines whether the reliability of the tracking processing has decreased. In the reliability determination process, it is determined whether the tracking processing of the person image data being tracked can continue uninterrupted for a predetermined time.
[0033] Furthermore, tracking sometimes fails when occlusion occurs during tracking. Occlusion here refers to a situation where the person in the person area cannot be correctly identified because other objects overlap the foreground side of the person. The management server 10 is configured to perform "occlusion detection processing" to detect the occurrence of occlusion during tracking of the person image data that is the object. In occlusion detection processing, for example, if the proportion of the foreground area of the person image relative to the bounding box of the person area is lower than a predetermined proportion, occlusion is determined to have occurred.
[0034] Furthermore, in the process of tracking, if the appearance of a person suddenly changes midway, the tracking process may sometimes fail. Such a situation may occur, for example, when a suspicious person disguises themselves to evade tracking.
[0035] Based on the above, in the crime prevention system 100 of this embodiment, when tracking of a target person is performed with high reliability and no obstruction occurs, if the tracking process fails, it is determined that the target person has made an appearance change as a suspicious action. Hereinafter, this process will be referred to as "suspicious action determination process".
[0036] When the management server 10 determines that a person has engaged in suspicious activity, it outputs an alarm message from the output device 4 via the communication network. This process will be referred to as "notification processing" below. Examples of the output device 4 include a monitor in a monitoring room displaying images from the surveillance camera 2, or a monitor's portable terminal. Examples of alarm messages include the display of a message urging attention to suspicious activity, and sound notifications.
[0037] According to the crime prevention system 100 described above, suspicious activities contained in the camera images of one or more surveillance cameras 2 can be identified. Therefore, the camera images of one or more surveillance cameras 2 can be used for crime prevention.
[0038] 2. The structure of the crime prevention system
[0039] Figure 2 This is a block diagram illustrating an example of the structure of a crime prevention system. The crime prevention system 100 includes one or more surveillance cameras 2, a management server 10, and an output device 4.
[0040] The management server 10 functions as a computer processing device. Typically, the management server 10 includes one or more processors 20 (hereinafter referred to as processor 20) and one or more memories 30 (hereinafter referred to as memory 30) combined with the processors 20. The memory 30 stores one or more crime prevention programs 32 (hereinafter referred to as crime prevention programs 32) and neural network models 33 that can be executed by the processors 20, as well as various associated data 34.
[0041] Various processes can be performed using the processor 20 by executing the crime prevention program 32. Figure 3 This is a block diagram illustrating the functionality achieved by executing the crime prevention program 32 by the processor 20 of the management server 10. (See diagram for example.) Figure 3 As shown, the processor 20 includes: a tracking processing unit 201 for performing tracking processing; a reliability determination processing unit 202 for performing reliability determination processing; an occlusion detection processing unit 203 for performing occlusion detection processing; a suspicious action determination processing unit 204 for determining suspicious actions contained in the camera image; and a notification processing unit 205 for performing notification processing. Referring also to... Figure 1 This explains the function of processor 20.
[0042] The tracking processing unit 201 is a function block for performing tracking processing based on Person Re-Identification. The tracking processing unit 201 detects person image data from camera images captured by the surveillance camera 2. Figure 4In the diagram, the bounding box BOX1 represents the person region of the person image data IMG1 detected from the camera image. The method for detecting person image data from the camera image can utilize well-known techniques, so its description is omitted.
[0043] The data 34 in the memory 30 contains a database of template image data. Figure 5 This diagram illustrates an example of a database of template image data. In the template image data, identification IDs are associated with person image data identified from past camera images. The tracking processing unit 201 has a neural network model 33 for Person Re-Identification. The neural network model 33 compares the person image data with the database of template image data stored in the memory 30, and searches for people in the template image data of the same person as the person image data based on their similarity. Furthermore, if no template image data with high similarity to the person image data exists, it is registered in the database after being associated with the identification ID as new person image data. The tracking processing unit 201 tracks the people contained in the camera images by sequentially performing the above processing on multiple camera images that are consecutive in a time sequence.
[0044] For example, when multiple camera images include a first camera image and a second camera image following the first camera image, in the tracking processing for the second camera image, the person image data of the person region contained in the second camera image is compared with template image data. The template image data at this time includes at least the person image data of the person region contained in the first camera image. Therefore, in the tracking processing for the second camera image, it is possible to identify the person image data with the person region contained in the first camera image, thereby enabling tracking of people according to a time sequence. Furthermore, the neural network model 33 used in the tracking processing can apply a known model for Person Re-Identification, so detailed explanation is omitted.
[0045] The reliability determination processing unit 202 is a function block for performing reliability determination processing. In the reliability determination processing, when tracking processing of the target image fails, it determines whether the tracking processing before the failure continued with high reliability. Here, the reliability metric is the duration for which tracking processing of the target image continues uninterrupted. The reliability determination processing unit 202 calculates the duration Tc of the tracking processing before failure and compares it with a predetermined time Tth. Here, the predetermined time Tth is a value preset as a threshold for determining high reliability in the determination processing. If the duration Tc is greater than the predetermined time Tth, the reliability determination processing unit 202 determines that the tracking processing continued with high reliability.
[0046] The occlusion detection processing unit 203 is a function block used for occlusion detection processing. Figure 6 This diagram illustrates the occlusion detection process. Figure 6 The example shown illustrates a state where a portion of another image data IMG2 overlaps on the foreground side of the person region of image data IMG1, which is enclosed by bounding box BOX1. In the occlusion detection process, the occlusion detection processing unit 203 calculates the ratio R of the foreground region of image data IMG1 relative to the bounding box BOX1. Furthermore, if the calculated ratio R is greater than a predetermined ratio Rth, the occlusion detection processing unit 203 determines that occlusion has occurred.
[0047] The suspicious action determination processing unit 204 is a function block used to perform suspicious action determination processing. In the suspicious action determination processing, if the tracking processing fails, the reliability determination processing determines that the state is highly reliable, and the occlusion detection processing does not detect the occurrence of occlusion, the suspicious action determination processing unit 204 determines that the person in the person image data being targeted has suspicious action.
[0048] The notification processing unit 205 is a functional block for performing notification processing. During notification processing, if the notification processing unit 205 determines that there is suspicious activity in the suspicious activity determination process, it outputs an alarm message from the output device 4. The method of outputting to the output device 4 is not limited. For example, the output method could be either an audible alert from a speaker or a display output such as a pop-up on a monitor.
[0049] 3. Specific examples of using crime prevention systems to identify suspicious activity
[0050] Next, refer to Figure 7This describes an application example of using the crime prevention system 100 to determine suspicious actions according to the implementation method. The crime prevention system 100 is a system that outputs an alarm for suspicious actions by a person when the camera images (IMG) of one or more surveillance cameras include suspicious actions performed by that person. Figure 7 This is a flowchart illustrating the routine of the processing implemented by the crime prevention system. The crime prevention program 32 is executed by the processor 20 of the management server 10. Figure 7 The flowchart illustrates a routine. Additionally, it shows a portion of the crime prevention methods described in embodiments of this disclosure.
[0051] exist Figure 7 In step S100 of the illustrated routine, tracking processing is performed on the image data of people contained in the camera images captured by one or more surveillance cameras 2. In the next step S102, it is determined whether the tracking processing in step S100 results in failed image data of people. As a result, if the determination is not considered valid, the processing of this routine ends; if the determination is considered valid, the processing proceeds to step S104.
[0052] In step S104, the reliability determination process determines whether the tracking process prior to the failure has a high reliability status. Typically, this process checks whether the duration Tc of the tracking process prior to the failure exceeds a predetermined time Tth. If the determination is deemed valid, the process proceeds to step S106. Otherwise, if the determination is not deemed valid, the process ends. This prevents the execution of suspicious action determination processes in subsequent steps.
[0053] In step S106, it is determined whether occlusion has been detected during the occlusion detection process. If the determination is successful, the process ends. This prevents the execution of subsequent suspicious action determination processes. Alternatively, if the determination is not successful, it is determined that suspicious action is present in the camera image, and the process proceeds to step S108. In step S108, alarm notification processing is performed, sending alarm information from the output device 4.
[0054] As can be seen from the above description, the crime prevention system 100 according to the embodiment can identify suspicious actions involving appearance alteration from camera images. As a result, alarm information can be notified from the output device 4, thus being useful for crime prevention.
[0055] 4. Variations
[0056] The crime prevention system 100 described in the embodiments can also be applied in the following variations.
[0057] 4-1. Suspicious Action Determination and Handling
[0058] In the process of determining suspicious behavior, the conditions for using the occlusion determination process and the reliability determination process are not mandatory. That is, in the process of determining suspicious behavior, if the behavior is deemed valid in step S102, the process of step S104 and step S106 can be skipped and the process can proceed to step S108.
[0059] 4-2. Follow-up Processing
[0060] There are no limitations to the methods used in tracking processing if people can be identified between camera images in a time sequence.
Claims
1. A crime prevention system for monitoring actions of a person included in a plurality of camera images successively imaged in accordance with a time series by a surveillance camera, wherein, The crime prevention system has the following features: The memory stores the images from the plurality of cameras; and The processor processes the multiple camera images stored in the memory. The processor is configured to perform: The tracking process involves detecting the person regions within the multiple camera images, and identifying the people within the multiple camera images according to a time sequence based on the person image data contained within those person regions; and Suspicious action determination processing: If the identification based on the tracking process fails midway, the person deemed to be engaging in suspicious action is included in the camera image. The processor is configured to disable the suspicious action determination process for the person's image data if the identification based on the tracking process for the person's image data fails midway, and if the duration of the tracking process for the person's image data before the failure is less than a predetermined time.
2. The crime prevention system according to claim 1, wherein, The plurality of camera images includes a first camera image and a second camera image following the first camera image. The memory stores template image data including the image data of the person area from the first camera image. The tracking process is configured to identify the person in the second camera image by comparing the person image data contained in the person region of the second camera image with the template image data using a neural network model.
3. The crime prevention system according to claim 1, wherein, The processor is configured to prohibit the determination and processing of suspicious actions in the area where the character area is obscured, in the event of such obstruction.
4. The crime prevention system according to any one of claims 1 to 3, wherein, The processor is configured to also perform notification processing, in the event that a person deemed to be engaging in the suspicious activity is present in the camera image, to send an alarm message from an output device.
5. A crime prevention method in which a computer executes monitoring of an action of a person included in a plurality of camera images successively imaged by a monitoring camera in accordance with a time series, wherein The crime prevention methods include: Detect the person regions of the people contained in the images from the multiple cameras respectively; Perform tracking processing to identify people contained in the camera images based on the person image data contained in the person area according to a time sequence; and If the identification based on the tracking process fails midway, the individuals deemed to be engaging in suspicious activity are included in the images from the multiple cameras. The computer is configured to disable the suspicious action determination process for the person's image data if the identification based on the tracking process for the person's image data fails midway, and if the duration of the tracking process for the person's image data before the failure is less than a predetermined time.
6. A computer-readable recording medium configured to record a crime prevention program that causes a computer to perform surveillance of the actions of persons contained in multiple camera images captured sequentially in a time sequence by a surveillance camera, wherein... The crime prevention procedure is executed by a computer: Detect the person regions of the people contained in the images from the multiple cameras respectively; Tracking processing is performed to identify people contained in multiple camera images based on the person image data contained in the person area according to the time sequence; as well as If the identification based on the tracking process fails midway, individuals deemed to be engaging in suspicious activity are included in the camera image. The crime prevention procedure causes the computer to: if the identification based on the tracking process of the person's image data becomes a failure midway, and if the duration of the tracking process of the person's image data before becoming a failure is less than a predetermined time, then prohibit the suspicious action determination process based on the person's image data.