Suspicious person determination device, suspicious person determination method and interphone system

Abstract

To provide a suspicious person determination device that can appropriately determine a person included in a video frame of a camera as a suspicious person.MEANS FOR SOLVING THE PROBLEM: A suspicious person determination device comprises: a person detection unit that acquires appearance information of a person contained in a video frame captured by a camera; and a person determination unit that determines whether the person is a suspicious person or not, based on the appearance information and determination time. The person determination unit determines whether the person included in the video frames over the determination time is the same person based on the appearance information, and when the same person is included in the video frame throughout the determination time, the person determination unit determines that the same person is the suspicious person.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a suspicious person determination device, a suspicious person determination method, and an intercom system. [Background technology]

[0002] 2. Description of the Related Art Conventionally, systems for detecting suspicious persons using imaging devices such as cameras have been used.

[0003] Patent Document 1 discloses a system that identifies a person from facial features captured by a camera, and determines that the identified person is suspicious if they appear within the camera's field of view with a certain frequency. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2014 / 132841 Summary of the Invention [Problem to be solved by the invention]

[0005] However, suspicious people often dislike being seen on camera and do not clearly expose their faces to the camera.

[0006] In such a case, the system described in Patent Document 1 may not be able to obtain sufficient facial information related to the facial features of the person captured by the camera, and may not be able to determine whether the person is suspicious. Furthermore, even if the system is able to determine whether the person is suspicious, it may take a long time to make the determination.

[0007] A non-limiting example of the present disclosure contributes to providing a suspicious person determination device that can appropriately determine a person included in a video frame of a camera as a suspicious person. [Means for solving the problem]

[0008] A suspicious person determination device according to one embodiment of the present disclosure includes a person detection unit that acquires appearance information of a person included in a video frame from a camera, and a person determination unit that determines whether the person is a suspicious person based on the appearance information and a determination time, and the person determination unit determines whether the people included in the video frames spanning the determination time are the same person based on the appearance information, and if the same person is included in the video frames spanning the determination time, determines that the same person is a suspicious person.

[0009] A suspicious person determination device according to one embodiment of the present disclosure includes a person detection unit that acquires appearance information of a person included in a video frame of a camera, and a person determination unit that determines whether the person is a suspicious person based on the appearance information and a determination time. When the person detection unit detects a first person included in the video frame and detects a second person included in the video frame before a release waiting time has elapsed after the first person frames out of the video frame, the person determination unit determines whether the second person is the same person as the first person based on the first appearance information of the first person and the second appearance information of the second person, and if the second person is the same person as the first person, determines that the same person is a suspicious person.

[0010] An intercom system according to an embodiment of the present disclosure includes the suspicious person determination device.

[0011] In one embodiment of the method for determining a suspicious person according to the present disclosure, a detection unit acquires appearance information of a person included in a video frame of a camera, a person determination unit determines whether the person is a suspicious person based on the appearance information and a determination time, and the person determination unit determines whether the people included in the video frames spanning the determination time are the same person based on the appearance information, and if the same person is included in the video frames spanning the determination time, determines that the same person is a suspicious person.

[0012] In one embodiment of the present disclosure, a method for determining a suspicious person includes a person detection unit acquiring appearance information of a person included in a video frame of a camera, and a person determination unit determining whether or not the person is a suspicious person based on the appearance information and a determination time, and when the person detection unit detects a first person included in the video frame and detects a second person included in the video frame before a release waiting time has elapsed after the first person frames out of the video frame, the person determination unit determines whether or not the second person is the same person as the first person based on the first appearance information of the first person and the second appearance information of the second person, and if the second person is the same person as the first person, determines that the same person is a suspicious person.

[0013] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0014] According to an embodiment of the present disclosure, a person included in a video frame of a camera can be appropriately determined to be a suspicious person.

[0015] Further advantages and benefits of an embodiment of the present disclosure will become apparent from the specification and drawings. Such advantages and / or benefits may be provided by some of the embodiments and features described in the specification and drawings, respectively, but not necessarily all of them may be provided to obtain one or more identical features. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a suspicious person determination device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of the configuration of a parent device and a child device according to an embodiment; [Figure 3] FIG. 10 is a diagram showing an example of person detection according to an embodiment; [Figure 4]FIG. 10 is a diagram showing an example of a watch list according to an embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a first suspicious person determination according to an embodiment; [Figure 6] 10 is a flowchart showing an example of an operation of the suspicious person determination device in the first suspicious person determination operation according to the embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a second suspicious person determination according to an embodiment. [Figure 8] 10 is a flowchart showing an example of the operation of the suspicious person determination device in the second suspicious person determination operation according to the embodiment. [Figure 9] FIG. 10 is a diagram showing an example of a third suspicious person determination according to an embodiment. [Figure 10] 10 is a flowchart showing an example of the operation of the suspicious person determination device in the third suspicious person determination operation according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or redundant explanation of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

[0018] The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0019] <System configuration> 1 is a diagram showing an example of the configuration of a suspicious person determination device 1 according to an embodiment. In explaining the suspicious person determination device 1 according to the embodiment, the suspicious person determination device 1 will be explained using as an example a suspicious person determination device 1 that uses an intercom system of a residence A including a base unit 2 and a sub-unit 3.

[0020] As shown in FIG. 1, the suspicious person determination device 1 includes a master unit 2 and a slave unit 3.

[0021] The suspicious person determination device 1 may be configured with a slave unit 3 equipped with an imaging device such as a camera, without including the master unit 2. The suspicious person determination device 1 may also be configured with an imaging device (for example, a surveillance camera) installed in building A1 and the master unit 2, without including the slave unit 3.

[0022] Base unit 2 is installed inside (inside) house A, such as in the living room or kitchen, and handset 3 is installed outside house A, near the entrance.

[0023] The base unit 2 communicates with the slave unit 3 via a wired or wireless connection. For example, as shown in Fig. 1, the base unit 2 may be configured to be communicably connected to the slave unit 3 via a communication line L1.

[0024] A person inside the house uses the base unit 2, and a person outside the house uses the handset 3, and a call is made between the person inside the house and the person outside the house.

[0025] <Block configuration> Block configuration of base unit 2 Fig. 2 is a diagram showing an example of a block configuration of the base unit 2 and the handset 3. As shown in Fig. 2, the base unit 2 has a control unit 21, a ROM 22a, a RAM 22b, a handset communication I / F unit 23, an audio signal control unit 24, a microphone 24a, a speaker 24b, an audio signal bus 25, a display / operation unit 26, a short-range wireless control unit 27, a wireless LAN communication I / F unit 27a, a wireless LAN communication unit 27b, and a wired LAN communication I / F unit 28.

[0026] The control unit 21 controls the overall operation of the base unit 2. The control unit 21 may be configured, for example, by a CPU (Central Processing Unit).

[0027] The control unit 21 includes a person detection unit 21a, a face detection unit 21b, and a person determination unit 21c.

[0028] The person detection unit 21a uses an existing image recognition technology to detect a person P captured by the camera 34 of the slave unit 3. The person detection unit 21a measures the time during which the detected person P is detected.

[0029] The person detection unit 21a acquires appearance information related to the features of the detected person P. For example, the appearance information includes information about the clothing of the person P, and is stored in the ROM 22a and the RAM 22b.

[0030] The face detection unit 21b detects the face of the person P detected by the person detection unit 21a. For example, the face detection unit 21b stores face information related to the facial features of the detected person P in the ROM 22a and the RAM 22b.

[0031] The person determination unit 21c determines whether or not the person P detected by the person detection unit 21a is a suspicious person, based on the face information, appearance information, and determination time.

[0032] For example, if the face detection unit 21b continues to detect facial information of a person P who is unacquainted with the resident of house A (hereinafter referred to as the resident) for a certain period of time, the person determination unit 21c determines that person P loitering within the grounds (hereinafter referred to as the grounds), such as the garden of house A, is a suspicious person.

[0033] In this case, the base unit 2 may be configured to display on the display / operation unit 26 a message saying, "A suspicious person is loitering in the garden. Would you like to contact a security company?"

[0034] The ROM 22a stores, for example, a program for the control unit 21 to control the base unit 2. The RAM 22b stores, for example, data for the control unit 21 to execute calculation processing and data for controlling each connected unit. Note that instead of the ROM 22a and the RAM 22b, a storage device such as a flash memory, an HDD (Hard Disk Drive), or an SSD (Solid State Drive) may be used.

[0035] The handset communication I / F unit 23 is an interface that communicatively connects the base unit 2 and the handset 3. The base unit 2 communicatively connected to the handset 3 via the handset communication I / F unit 23 becomes capable of making and receiving calls with the handset 3.

[0036] The audio signal control unit 24 is electrically connected to the control unit 21, the microphone 24a, and the speaker 24b. The audio input to the microphone 24a is converted into analog audio data by the audio signal control unit 24. The speaker 24b outputs sound based on the analog audio data output from the audio signal control unit 24. The audio signal controlled by the audio signal control unit 24 is controlled by the control unit 21 via an audio signal bus 25.

[0037] Display / operation unit 26 accepts operations by people in the residence where base unit 2 is installed. Display / operation unit 26 displays images based on digital video data output from control unit 21. Display / operation unit 26 may be a touch panel type liquid crystal display device.

[0038] Short-range wireless control unit 27 controls wireless LAN communication I / F unit 27a. Short-range wireless control unit 27 controls wireless communication between base unit 2 and devices with wireless communication functions that are located in a short distance from base unit 2. For example, short-range wireless control unit 27 controls wireless communication between a smartphone app in house A and base unit 2. At that time, base unit 2 may notify the smartphone of information about a visitor who has arrived.

[0039] Furthermore, for example, when the base unit 2 and the handset 3 are connected via wireless communication, the short-distance wireless control unit 27 may control the wireless communication between the base unit 2 and the handset 3.

[0040] The wireless LAN communication I / F unit 27a is an interface that connects the base unit 2 and a device having a wireless communication function so that they can communicate with each other.

[0041] The wireless LAN communication unit 27b performs wireless communication (short-distance wireless communication) in accordance with a communication standard such as Wi-Fi (registered trademark).

[0042] The wired LAN communication I / F unit 28 is an interface that communicatively connects the base unit 2 with a device having a wired communication function. The wired LAN communication I / F unit 28 may be an interface that connects the base unit 2 to the Internet 100.

[0043] Block configuration of handset 3 As shown in FIG. 2, the handset 3 includes a control unit 31, a ROM 32a, a RAM 32b, a base unit communication I / F unit 33, a camera 34, a call button 35, a microphone 36, and a speaker 37.

[0044] The control unit 31 controls the overall operation of the handset 3 .

[0045] The ROM 32a stores, for example, a program for the control unit 31 to control the slave unit 3. The RAM 32b stores, for example, data for the control unit 31 to execute calculation processing and data for controlling each connected unit.

[0046] The base unit communication I / F unit 33 is an interface that connects the handset 3 and the base unit 2 so that they can communicate with each other.

[0047] Call button 35 is pressed, for example, by a visitor outside the residence. Based on the operation of this call button 35, control unit 31 controls camera 34 to capture an image of the area outside the residence. The image captured by this camera 34 is displayed, for example, on display / operation unit 26 of base unit 2.

[0048] Furthermore, when a visitor presses the call button 35, the control unit 31 controls the microphone 36 and the speaker 37. The voice input to the microphone 36 is output from the speaker 24b of the base unit 2. The speaker 37 outputs the voice input to the microphone 24a of the base unit 2.

[0049] <Person detection> FIG. 3 is a diagram showing an example of a person P detected by the person detection unit 21a.

[0050] As shown in Fig. 3, the person detection unit 21a detects a person P included in a video frame of a camera. The person included in the camera image may be detected in a rectangular shape. The person detection unit 21a may detect the person in a rectangular region R as shown in Fig. 3.

[0051] When the person P moves, the region R follows the movement of the person P. The person detection unit 21a may acquire the direction and speed of movement of the person P by referring to the coordinates of the angles Ra, Rb, Rc, and Rd of the region R before and after the person P moves.

[0052] The region R does not necessarily have to be set to surround the entire body of the person P, but may be set to surround an area necessary for determining the person P, such as the area from the shoulders to the head of the person P. Furthermore, the region R may be set to a circular or elliptical shape instead of a rectangular shape, depending on the posture and posture of the person P.

[0053] Appearance information Furthermore, the person detection unit 21a acquires appearance information of the detected person P. The appearance information is information based on the characteristics of the person P's appearance.

[0054] The appearance information includes body type information C1 relating to body type, appearance information C2 relating to appearance, and foot information C3 relating to feet.

[0055] The physical information C1 includes information about the physical characteristics of person P, such as height, build, posture, limb length, walking style, mouth movements when speaking, and eye movements. - Appearance information C2 includes information about person P's clothing, such as the type, color, pattern, and design of the clothes they are wearing (long sleeves, short sleeves, long pants, shorts, etc.), and the type, color, shape, and material of any accessories they are wearing (sunglasses, masks, necklaces, earrings, etc.). The foot information C3 includes information about the feet of the person P, such as the type, color, pattern, shape, and material of shoes (sandals, sneakers, etc.).

[0056] The person detection unit 21a acquires, for example, body type information C1, appearance information C2, and foot information C3, and digitizes the acquired information to create appearance information. The person detection unit 21a transmits the digitized appearance information to the person determination unit 21c.

[0057] Based on the received appearance information, the person determination unit 21c determines whether person P1 and person P2 are the same person. This determination is made based on the similarity between the appearance information of person P1 and the appearance information of person P2. For example, the similarity between the appearance information is expressed as a numerical value between 0 and 100.

[0058] When the similarity threshold is 80, if the similarity between person P1 and person P2 is 83, the person determination unit 21c determines that person P1 and person P2 are the same person, whereas if the similarity between person P1 and person P2 is 37, the person determination unit 21c determines that person P1 and person P2 are not the same person.

[0059] If the similarity of the appearance information is equal to or greater than a threshold, the individual determination unit 21c determines that the individuals P1 and P2 are the same individual. On the other hand, if the similarity of the appearance information is less than the threshold, the individual determination unit 21c determines that the individuals P1 and P2 are not the same individual.

[0060] Face information The face detection unit 21b detects the face of the person P detected by the person detection unit 21a, and acquires face information relating to facial features.

[0061] The face information acquired by the face detection unit 21b includes information based on features such as the contour of the face of person P and the positions of the eyes and nose. For example, the face information includes information on the contour of the face, the shape of the face, the distance between the eyes, the shape of the nose, the position of the mouth, etc.

[0062] The person determination unit 21c may determine a person based on either the appearance information or the face information, or may determine a person based on both the appearance information and the face information.

[0063] <Watchlist> The watch list L will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the watch list L.

[0064] The watch list L is a list in which people detected by the person detection unit 21a are registered. The watch list L mainly stores people, registration date and time, and update date and time. The registration date and time item stores the date and time when the person detection unit 21a first detected the person. The update date and time item stores the date and time when the person detection unit 21a most recently (last) detected the person.

[0065] For example, the person detection unit 21a first detected person X1 at 9:38 on January 26, 2021. Thereafter, the person detection unit 21a detected person X1 multiple times, and the last time person X1 was detected was at 23:14:07 on August 26, 2022. Therefore, in the watch list L, the date and time when person detection unit 21a first detected person X1 is registered in the registration date and time column, and the date and time when person X1 was last detected is registered as the update date and time.

[0066] For example, the watch list is created as follows: Any person detected by the person detection unit 21a for the first time, including visitors to the resident, is registered on the watch list. The resident operates the base unit 2 to check the watch list L and delete from the watch list L any person who is determined not to be a suspicious person, such as a family member, relative, or acquaintance.

[0067] As a result, the number of people who are acquainted with the resident decreases and the number of people who are not acquainted with the resident increases on watchlist L. As a result, watchlist L becomes a list that contains many people who may be determined to be suspicious.

[0068] Furthermore, the configuration may be such that, when a resident sees a person captured by the camera 34 and has a suspicious impression of the person, the resident can register the person in the watch list L.

[0069] In addition, if a resident determines that a person registered on the watch list L is not suspicious, the resident may be able to delete that person from the watch list L.

[0070] The people registered in the watch list L are linked to face information and appearance information, and are stored in the ROM 22a of the master unit 2 or the ROM 32a of the slave unit 3. The watch list L may also be stored in both the ROM 22a of the master unit 2 and the ROM 32a of the slave unit 3.

[0071] If the person determination unit 21c determines, based on the face information and appearance information, that the person registered in the watch list L and the person detected by the person detection unit 21a are the same person, the person determination unit 21c determines that the person detected by the person detection unit 21a is a suspicious person because the person is the same as the person registered in the watch list.

[0072] <Suspicious person determination> In this embodiment, first to third suspicious person determinations are executed. In the suspicious person determination, a person P who has been in the premises for a predetermined time is determined to be a suspicious person.

[0073] When a predetermined time has elapsed since the person detection unit 21a detected the person P, that is, when the time during which the person detection unit 21a detected the person P has elapsed the determination time, the suspicious person determination device 1 determines that the person P is a suspicious person. Note that the determination time can be changed by the base unit 2 or a communication device (smartphone, tablet, etc.) connected to the base unit 2 so as to be able to communicate with it.

[0074] Suspicious people may dislike being seen on camera and avoid looking directly at the camera. Suspicious people may also try to deceive the camera and those around them by disguising themselves to change their appearance.

[0075] If the disguised person P is recognized as another person Q, the person detection unit 21a will need to set different judgment times for person P and person Q, which may make the process of judging a suspicious person complicated.

[0076] The suspicious person determination device 1 of this embodiment determines whether or not a detected person is the same person even if the appearance of the person changes during the process, and performs appropriate suspicious person detection.

[0077] First suspicious person determination The first suspicious person determination will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the first suspicious person determination. The person P before disguise is person P11 (first person), and the person P after disguise is person P12 (second person).

[0078] A person P11 enters the angle of view of the camera 34 and is captured by the camera 34. For example, a person P11 who has invaded a site is captured by the camera 34. Frames of the video captured by the camera 34 are extracted at regular intervals T.

[0079] The person detection unit 21a detects the person P11 included in the frame F11. The person detection unit 21a detects the position of the person P11 based on the coordinates Ra, Rb, Rc, and Rd of the four corners of the region R.

[0080] The person detection unit 21a may compare the region R of the frame F11 with the region R of the previously extracted frame, and acquire the characteristics of the person P11 based on the change in brightness.

[0081] The person detection unit 21a acquires appearance information of the person P11 based on the appearance of the person P11. For example, the person detection unit 21a acquires body type information C1a, appearance information C2a, and foot information C3a of the person P11. The face detection unit 21b detects the face of the person P11 and acquires face information of the person P11.

[0082] Person P11 is disguised with sunglasses and a hoodie. The person detection unit 21a detects person P12 included in frame F12.

[0083] With reference to frame F12, the face detection unit 21b acquires face information of the person P12, and the person detection unit 21a acquires appearance information of the person P12. For example, the person detection unit 21a acquires body type information C1b, appearance information C2b, and foot information C3b of the person P12.

[0084] The person determination unit 21c compares the facial information of the person P11 and the person P12, and determines whether the person P11 and the person P2 are the same person.

[0085] When it is not possible to determine whether person P11 and person P12 are the same person by comparing the facial information of person P11 and person P12, person determination unit 21c determines whether person P11 and person P12 are the same person based on appearance information.

[0086] For example, even if the acquired facial information of person P12 is insufficient because the person has disguised themselves and changed their appearance, the person determination unit 21c compares the appearance information of persons P11 and P12 and determines whether person P11 and person P12 are the same person.

[0087] Since the appearance information includes information based on the characteristics unique to each person, such as height and build, even if person P11 disguises himself or herself as person P12, person determination unit 21c determines that person P11 and person P12 are the same person.

[0088] If it is determined that person P11 and person P12 are the same person, the suspicious person determination device 1 sets a determination time for the currently detected person P12, and when the determination time has elapsed, the suspicious person determination device 1 executes a suspicious person detection process and determines that person P11 is a suspicious person.

[0089] The first suspicious person determination operation will be described with reference to Fig. 6. Fig. 6 is a flow chart showing an example of the operation of the suspicious person determination device 1 in the first suspicious person determination operation.

[0090] The suspicious person determination device 1 acquires frames from the video captured by the camera 34 at intervals of a period T (S1). For example, the suspicious person determination device 1 acquires frame F12 shown in FIG.

[0091] The suspicious person determination device 1 determines whether or not a person is detected in the acquired frame (S2). If the suspicious person determination device 1 determines that a person is not detected in the acquired frame (NO in S2), the suspicious person determination device 1 resets the determination time (S3), and the flow returns to the processing of S1.

[0092] If the suspicious person determination device 1 determines that a person has been detected in the acquired frame (YES in S2), the suspicious person determination device 1 acquires appearance information (S4).

[0093] In this process, the suspicious person determination device 1 executes a process of acquiring face information and appearance information of the person P12 included in the frame F12.

[0094] It is determined whether the detected person can be identified based on the face information (S5).

[0095] If the suspicious person determination device 1 is unable to identify the detected person based on the face information (NO in S5), it compares the appearance information of the person detected this time with the person detected previously (S6).

[0096] The person P12 detected this time (included in frame F12) is a suspicious person who disguises himself / herself with sunglasses and a hoodie to change his / her appearance to make it harder for him / her to be photographed by the camera 34. For this reason, the suspicious person determination device 1 may not be able to identify the person P12 based on the facial information. In this process, the suspicious person determination device 1 executes a process to compare the similarity of the appearance information of the person P11 and the person P12.

[0097] The suspicious person determination device 1 determines whether the degree of similarity between the appearance information of the person detected this time and the person detected last time exceeds a threshold value (S7).

[0098] If the suspicious person determination device 1 determines that the similarity of the appearance information is less than the threshold (NO in S7), the suspicious person determination device 1 registers the person detected in process S2 in a watch list L (S8) and resets the determination time (S9). The flow returns to process S1.

[0099] On the other hand, if the suspicious person determination device 1 determines that the similarity of the appearance information is equal to or greater than the threshold value (YES in S7), the flow proceeds to processing in S11.

[0100] Here, since the similarity of the appearance information is equal to or greater than the threshold, the suspicious person determination device 1 determines that the person P11 and the person P12 are the same person, and executes processing to measure a determination time.

[0101] Returning to the explanation of the processing of S5, if the suspicious person determination device 1 can identify the detected person based on the face information (YES in S5), the suspicious person determination device 1 determines whether the detected person is a person registered in the watch list L (S10). If the suspicious person determination device 1 determines that the detected person is not a person registered in the watch list L (NO in S10), the flow proceeds to the processing of S8.

[0102] When the suspicious person determination device 1 determines that the detected person is a person registered in the watch list L (YES in S10), it sets a determination time for that person (S11).

[0103] The suspicious person determination device 1 determines whether the set determination time has elapsed (S12). If the suspicious person determination device 1 determines that the determination time has not elapsed (NO in S12), the flow returns to the processing of S1.

[0104] If the suspicious person determination device 1 determines that the determination time has elapsed (YES in S12), the suspicious person determination device 1 determines that the person is a suspicious person and performs a suspicious person detection process (S13).

[0105] In the suspicious person detection process of S13, the suspicious person determination device 1 may take measures to notify residents, security management companies, police, etc. that a suspicious person has been detected using an alarm sound, message, image, etc.

[0106] The suspicious person determination device 1 may update the appearance information of a person determined to be suspicious, and at this time, may acquire the behavioral pattern of the person determined to be suspicious and add it to the appearance information as suspicious behavior. This makes it possible to take safety measures based on suspicious information such as the suspicious person's abnormal behavior or long stay.

[0107] With this configuration, the suspicious person determination device 1 can detect a suspicious person based on facial information (by facial recognition). Furthermore, even if the person cannot be identified sufficiently based on facial information, the suspicious person determination device 1 can detect a suspicious person based on appearance information.

[0108] If the acquired facial information is insufficient, the suspicious person determination device 1 can determine, based on appearance information, that the person P11 detected last time and the (disguised) person P12 detected this time are the same person, and detect the suspicious person.

[0109] Since the appearance information is information based on the characteristics unique to each person, such as height and body type, the suspicious person determination device 1 can determine the person by referring to the appearance information.

[0110] Second suspicious person detection The second suspicious person determination will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the second suspicious person determination.

[0111] The second suspicious person determination is a suspicious person determination when person P leaves the field of view of camera 34 (frame out), and after a while, person P enters the field of view of camera 34 (frame in). Hereinafter, person P before frame out will be referred to as person P21 (first person), and person P after frame in will be referred to as person P22 (second person).

[0112] The person detection unit 21a detects a person P21 included in the frame F21. Based on the region R, the person detection unit 21a detects the position of the person P21.

[0113] The person detection unit 21a acquires appearance information of the person P21. For example, the person detection unit 21a acquires body type information C1a, appearance information C2a, and foot information C3a of the person P21.

[0114] The face detection unit 21b acquires face information of the person P21.

[0115] Person P21 leaves the frame. Frame F21a is the frame after person P21 leaves the frame. Frame F21a shows an empty garden.

[0116] While out of frame, person P21 disguises himself / herself. For example, person P21 uses sunglasses to disguise himself / herself. After a while, person P22 disguises himself / herself and enters the frame (frame F22).

[0117] The person detection unit 21a detects a person P22 included in the frame F22.

[0118] The person detection unit 21a acquires appearance information of the person P22. For example, the person detection unit 21a acquires body type information C1b, appearance information C2b, and foot information C3b of the person P12. The face detection unit 21b acquires face information of the person P22.

[0119] The person determination unit 21c determines whether the person P21 and the person P22 are the same person based on the facial information of the person P21 and the person P22.

[0120] When it is not possible to determine whether the person P21 and the person P22 are the same person by comparing the facial information of the person P21 and the person P22, the person determination unit 21c determines whether the person P21 and the person P22 are the same person based on the appearance information.

[0121] If it is determined that the person P21 and the person P22 are the same, the suspicious person determination device 1 sets a determination time (second determination time) for the person P22. If the sum of the determination time (first determination time) set for P21 and the determination time set for the person P22 is equal to or greater than a predetermined value, the suspicious person determination device 1 determines that the person P21 (person P22) is a suspicious person.

[0122] If the acquired facial information is insufficient, the suspicious person determination device 1 can determine, based on appearance information, that the person P21 detected last time and the person P22 detected this time (who has entered the frame in disguise) are the same person, and detect the suspicious person.

[0123] The second suspicious person determination operation will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the suspicious person determination device 1 in the second suspicious person determination operation.

[0124] The processes of S1, S2, S4 to S13 are the same as those in Fig. 6, and therefore will not be described here. Compared to the flow in Fig. 6, the flow in Fig. 8 executes processes of S21 to S24 instead of the process of S3.

[0125] In the case of the second suspicious person determination operation, the frame F21a acquired in the process of S1 is a frame acquired after the person P21 has gone out of the frame, so no person is detected in the frame F21a (NO in S2).

[0126] If the result of S2 is NO, the suspicious person determination device 1 sets a determination time (S21) and a release waiting time (S22). For example, the suspicious person determination device 1 sets a determination time and a release waiting time for the person P21 who has left the frame. The release waiting time can be changed by the base unit 2 or a communication device (smartphone, tablet, etc.) connected to the base unit 2 so as to be able to communicate with the base unit 2.

[0127] The suspicious person determination device 1 determines whether the cancellation waiting time has elapsed (S23). If the suspicious person determination device 1 determines that the cancellation waiting time has not elapsed (NO in S23), the flow returns to the processing of S1. For example, the suspicious person determination device 1 will determine whether or not the frame-out person P21 is a suspicious person until the cancellation waiting time has elapsed.

[0128] If the suspicious person determination device 1 determines that the cancellation waiting time has elapsed (YES in S23), the suspicious person determination device 1 resets the determination time (S24), and the flow returns to the processing of S1.

[0129] With this configuration, even if a person who is thought to be a suspicious person frames out and then disguises himself / herself and frames back in, the suspicious person determination device 1 can execute the suspicious person detection process. To be able to deal with cases where a different person frames in after the person frames out, the suspicious person determination device 1 may also execute the suspicious person detection process for people registered in the watch list L.

[0130] In the second suspicious person determination, since it is possible that another person will come into the frame after person P21 frames out, a determination time and a release waiting time may also be set for persons registered in the watch list L who meet the conditions.

[0131] For example, the determination time and release waiting time may be set for a person whose update date and time of the watch list L has been updated a predetermined number of times or more in the past week, or for a person who has been captured in a frame a predetermined number of times or more within a predetermined time. The determination time and release waiting time may be set for multiple people.

[0132] Third suspicious person detection The third suspicious person determination will be described with reference to Fig. 9. Fig. 9 is a diagram showing an example of the third suspicious person determination.

[0133] The third suspicious person determination is the detection of a suspicious person in a detection area S within the frame. Hereinafter, the person P before going out of the frame will be referred to as person P31 (first person), and the person P after coming into the frame will be referred to as person P32 (second person).

[0134] Unlike the second suspicious person determination, in the third suspicious person determination, the suspicious person determination device 1 sets a detection area S within the frame and determines whether or not a person within the detection area S is a suspicious person.

[0135] By referring to the coordinates of the angles Ra, Rb, Rc, and Rd of the region R and the detection area S, the person detection unit 21a determines whether or not a person has entered the detection area S. In this case, it is sufficient that any of the angles Ra, Rb, Rc, and Rd is within the detection area S, and the suspicious person determination device 1 performs the third suspicious person determination in the detection area S.

[0136] Unlike the first and second suspicious person determinations, the suspicious person determination device 1 performs the suspicious person determination based on appearance information, and therefore the face of person P31 does not need to be included in the detection area S. In the third suspicious person determination, the person is not determined based on face information.

[0137] For example, as shown in Fig. 9, person P31 in frame F31 enters the frame from the left and moves to the right. Frame F31a is a frame after person P31 leaves the frame, so no one is in the frame. Person P32 in frame F33 enters the frame from the right and moves to the left.

[0138] In this way, even if the direction of movement of persons P31 and P32 is different before and after they go out of frame, the suspicious person determination device 1 can determine whether persons P31 and P32 are the same person based on appearance information.

[0139] Furthermore, by setting the detection area S, for example, when a person walking outside the fence (outside the premises) enters the frame of the suspicious person determination device 1, it is possible to prevent the suspicious person determination device 1 from erroneously determining that the person is a suspicious person. This allows the suspicious person determination device 1 to narrow down the number of people to be determined as suspicious or not.

[0140] The third suspicious person determination operation will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the operation of the suspicious person determination device 1 in the third suspicious person determination operation.

[0141] The processes of S1, S8, S9, 11 to S13, and S21 to S24 are the same as those in Fig. 8, and therefore will not be described. Compared to the flow in Fig. 8, the flow in Fig. 10 executes processes of S31 to S34 after the process of S1.

[0142] After the process of S1, the suspicious person determination device 1 determines whether or not a person has been detected in the detection area S (S31). If the suspicious person determination device 1 determines that a person has not been detected in the detection area S (NO in S31), the flow proceeds to the process of S21.

[0143] When the suspicious person determination device 1 determines that a person has been detected in the detection area S (YES in S31), ), the appearance information of the person detected in the detection area S is acquired (S32), and compared with the appearance information of the person detected previously (S33).

[0144] The suspicious person determination device 1 determines whether the similarity between the appearance information of the person detected this time and the person detected last time exceeds a threshold value (S34).

[0145] If the suspicious person determination device 1 determines that the similarity of the appearance information is less than the threshold value (NO in S34), the flow proceeds to the processing of S8.

[0146] On the other hand, if the suspicious person determination device 1 determines that the similarity of the appearance information is equal to or greater than the threshold value (YES in S34), the flow proceeds to the processing in S11.

[0147] With this configuration, the suspicious person determination device 1 determines people and detects suspicious people within a specific area, which can prevent unnecessary suspicious person determination of people outside the set area, such as people walking outside a fence. Also, because the suspicious person determination device 1 detects people within a specific area, it is expected to determine whether the person is suspicious with high accuracy and in a short time.

[0148] (Variation) The suspicious person determination device 1 may be composed of only the slave unit 3. The control unit 31 of the slave unit 3 may have a person detection unit, a face detection unit, and a person determination unit, similar to the control unit 21 of the master unit 2, and may be configured to perform suspicious person determination.

[0149] <Summary of the embodiment> As described above, the suspicious person determination device of this embodiment comprises a person detection unit that acquires appearance information of a person included in a camera's video frame, and a person determination unit that determines whether the person is a suspicious person based on the appearance information and a determination time, and the person determination unit determines whether the people included in the video frames spanning the determination time are the same person based on the appearance information, and if the same person is included in the video frames spanning the determination time, determines that the same person is a suspicious person.

[0150] The suspicious person determination device 1 can detect a suspicious person based on facial information. Furthermore, even if the suspicious person determination device 1 cannot determine a person based on facial information, it can detect a suspicious person based on appearance information based on individual characteristics of each person, such as height and body type.

[0151] Although the embodiments have been described above with reference to the drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications or alterations within the scope of the claims. It is understood that such modifications or alterations also fall within the technical scope of the present disclosure. Furthermore, the components in the embodiments may be combined in any manner without departing from the spirit of the present disclosure.

[0152] In the above-described embodiments, the notation "... part" used for each component may be replaced with other notations such as "... circuitry," "... assembly," "... device," "... unit," or "... module."

[0153] The present disclosure can be realized by software, hardware, or software linked to hardware. Each functional block used in the description of the above embodiments may be partially or entirely realized as an LSI, which is an integrated circuit, and each process described in the above embodiments may be partially or entirely controlled by a single LSI or a combination of LSIs. The LSI may be composed of individual chips, or may be composed of a single chip that includes some or all of the functional blocks. The LSI may have data input and output. Depending on the degree of integration, the LSI may be called an IC, system LSI, super LSI, or ultra LSI.

[0154] The integrated circuit method is not limited to LSI, but may be realized by a dedicated circuit, a general-purpose processor, or a dedicated processor. Also, a field programmable gate array (FPGA) that can be programmed after LSI manufacturing, or a reconfigurable processor that can reconfigure the connections and settings of circuit cells within the LSI, may be used. The present disclosure may be realized as digital processing or analog processing.

[0155] Furthermore, if an integrated circuit technology that can replace LSI emerges due to advances in semiconductor technology or other derivative technologies, it is natural that such technology may be used to integrate functional blocks. The application of biotechnology, etc. is also a possibility. [Industrial Applicability]

[0156] The present disclosure is useful as a suspicious person determination device. [Explanation of symbols]

[0157] 1. Suspicious person detection device 2 Base unit 3 Handsets 21 Control section 21a Person detection unit 21b Face detection unit 21c Person recognition section

Claims

1. a person detection unit that acquires appearance information of a person included in an image frame of a camera; a person determination unit that determines whether the person is a suspicious person based on the appearance information and the determination time, the person determination unit determines whether the person included in the video frames spanning the determination time is the same person based on the appearance information, and if the same person is included in the video frames spanning the determination time, determines that the same person is a suspicious person. Suspicious person detection device.

2. the person detection unit acquires the appearance information from a plurality of video frames spanning the determination time; the person determination unit determines whether the person included in each of the video frames is the same person based on the similarity of the appearance information in the plurality of video frames; The suspicious person determination device according to claim 1 .

3. The appearance information includes information about the person's body shape, information about clothes worn by the person, and information about accessories worn by the person. The suspicious person determination device according to claim 2 .

4. A detection area is set in the video frame, the person determination unit determines whether the person in the detection area is the same person as The suspicious person determination device according to claim 3 .

5. a person detection unit that acquires appearance information of a person included in an image frame of a camera; a person determination unit that determines whether the person is a suspicious person based on the appearance information and the determination time, the person detection unit detects a first person included in the video frame; If a second person is detected in the video frame before a release waiting time has elapsed since the first person left the video frame, the person determination unit determines whether the second person is the same person as the first person based on the first appearance information of the first person and the second appearance information of the second person, and if the second person is the same person as the first person, determines that the same person is a suspicious person; Suspicious person detection device.

6. When the person determination unit determines that the second person is the same person as the first person, determining whether the same person is a suspicious person based on a first determination time set for the first person and a second determination time set for the second person; The suspicious person determination device according to claim 5 .

7. When the sum of the first determination time and the second determination time is equal to or greater than a predetermined value, the person determination unit determines that the same person is a suspicious person; The suspicious person determination device according to claim 6.

8. A detection area is set in the video frame, the person determination unit determines whether the second person is the same person as the first person in the detection area based on the first appearance information and the second appearance information; The suspicious person determination device according to claim 5 .

9. A suspicious person determination device according to any one of claims 1 to 8, Intercom system.

10. The detection unit acquires appearance information of a person included in an image frame of the camera, the person determination unit determines whether the person is a suspicious person based on the appearance information and the determination time; the person determination unit determines whether the person included in the video frames spanning the determination time is the same person based on the appearance information, and if the same person is included in the video frames spanning the determination time, determines that the same person is a suspicious person. Method for determining suspicious persons.

11. The person detection unit acquires appearance information of a person included in an image frame of the camera, the person determination unit determines whether the person is a suspicious person based on the appearance information and the determination time; the person detection unit detects a first person included in the video frame; If a second person is detected in the video frame before a release waiting time has elapsed since the first person left the video frame, the person determination unit determines whether the second person is the same person as the first person based on the first appearance information of the first person and the second appearance information of the second person, and if the second person is the same person as the first person, determines that the same person is a suspicious person; Method for determining suspicious persons.

Images