Information Processing System

Abstract

To solve a problem of degradation in security or safety.SOLUTION: The information processing system includes: a registration list storing biological information of a character; detection means for detecting a character taking a specific action from a picked-up image picked up by a camera; control means for extracting biological information of the detected character taking the action from the picked-up image and according to a registration state against the registration list of the extracted biological information, extracting other biological information from the detected character taking the action and for processing to be registered into the registration list.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present invention relates to an information processing system, and more particularly to an information processing system for monitoring suspicious persons. [Background technology]

[0002] In areas such as buildings and sites where security is required, and in urban areas where safety is required, surveillance cameras are installed to monitor suspicious individuals. For example, Patent Documents 1 and 2 disclose technologies that detect individuals behaving suspiciously from images captured by a surveillance camera, and output an alarm and location information if the facial information of the individual who engaged in suspicious behavior is registered on a blacklist. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-246799 [Patent Document 2] International Publication No. WO2017 / 029718 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the above-mentioned technology, if a person captured on a surveillance camera is registered on a blacklist, that person is treated as suspicious, but the technology does not address people who are not registered. As a result, it is not possible to take appropriate action against people who are not registered on the blacklist, or conversely, people who are registered on the whitelist, even if they behave suspiciously. Furthermore, if an appropriate image for comparison cannot be obtained for a person behaving suspiciously, it is not possible to take appropriate action against that person's subsequent behavior. This creates the problem of reduced security and safety.

[0005] Therefore, an object of the present invention is to provide an information processing system that can solve the problem of reduced security and safety. [Means for solving the problem]

[0006] An information processing system according to one embodiment of the present invention includes: a registration list storing biometric information of people; a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; Equipped with The structure is as follows.

[0007] A program according to one aspect of the present invention includes: In the information processing device, a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; To realize The structure is as follows.

[0008] A control method according to one aspect of the present invention includes: an information processing device having a registration list storing biometric information of a human body, Detects people engaging in specific behavior from images captured by a camera, extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and performing a process of extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; The structure is as follows. [Effects of the Invention]

[0009] With the above-described configuration, the present invention can quickly deter a person who has engaged in suspicious behavior from engaging in further behavior, thereby improving security and safety. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to a first embodiment of the present invention. [Figure 2] 2 is a block diagram showing the configuration of the control server disclosed in FIG. 1. FIG. [Figure 3] 2 is a flowchart showing the operation of the control server disclosed in FIG. 1; [Figure 4] 2 is a flowchart showing the operation of the control server disclosed in FIG. 1; [Figure 5] FIG. 10 is a block diagram showing the configuration of an information processing system according to a second embodiment of the present invention. [Figure 6] FIG. 6 is a block diagram showing the configuration of the control server disclosed in FIG. 5. [Figure 7] 6 is a flowchart showing the operation of the control server disclosed in FIG. 5. [Figure 8] 6 is a flowchart showing the operation of the control server disclosed in FIG. 5. [Figure 9] 10 is a flowchart showing the operation of the information processing system according to the third embodiment of the present invention. [Figure 10] FIG. 10 is a block diagram showing the configuration of an information processing system according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] <Embodiment 1> A first embodiment of the present invention will be described with reference to Figures 1 to 4. Figures 1 and 2 are diagrams for explaining the configuration of an information processing system. Figures 3 and 4 are diagrams for explaining the operation of the information processing system.

[0012] [composition] The information processing system of the present invention is a system for monitoring people in places where security and safety are required, such as data centers, offices, and event venues, and for quickly preventing further actions by the person if suspicious behavior is detected. In the present invention, when a gate is installed in an area where security is required, the area surrounding the gate is defined as a specific area A to be monitored. In the present invention, suspicious behavior is defined as, for example, loitering, entering a restricted area, or leaving something behind. However, the specific area and suspicious behavior described above are merely examples and are not limited to the above.

[0013] 1, the information processing system includes a surveillance camera C1 that takes images to monitor a specific area A, a control server 1, an authentication gate G that is an entrance / exit to a predetermined area, and an authentication camera C2 that takes images of people passing through the authentication gate G for authentication purposes. Each component will be described in detail below.

[0014] The control server 1 is configured with one or more information processing devices each equipped with a calculation device and a storage device. As shown in Fig. 2, the control server 1 includes a detection unit 11 (detection means), a control unit 12 (control means), and an authentication unit 13, which are constructed by the calculation device executing a program. The control server 1 also includes a registration list 15 formed in the storage device.

[0015] The registration list 15 is a so-called blacklist that registers facial image information of people who have been designated as suspicious persons in advance. The registered facial image information may be the facial image itself, or may be feature amounts extracted from the facial image. Note that the information of suspicious persons registered in the registration list 15 is not limited to facial image information, and may also be biometric information indicating other physical features.

[0016] The detection unit 11 detects a person engaging in specific behavior from a captured image captured within a specific area A. Specifically, the detection unit 11, as an example, extracts a person from the captured image. For example, the detection unit 11 detects a moving object from a video image and detects the person from its outline, color, etc. The detection unit 11 then extracts the movement trajectory of the extracted person and analyzes the movement trajectory to detect a person engaging in pre-registered suspicious behavior (specific behavior). Suspicious behavior includes, for example, loitering behavior such as staying in a certain area for a certain period of time or moving back and forth, entering a pre-set no-entry zone, or leaving an object behind. Such behavior can be detected using existing technology by detecting the person's movement trajectory and the person and surrounding objects. When the detection unit 11 detects a person engaging in suspicious behavior, it passes the captured image to the control unit 12.

[0017] When the control unit 12 acquires a captured image of a person exhibiting suspicious behavior, it extracts a facial image of the person from the captured image and extracts feature amounts of the facial image. Then, the control unit 12 determines whether the extracted facial image matches a facial image in the registration list 15, which is a blacklist. Note that the determination of whether the facial images match is performed, for example, by calculating a score such as the similarity of feature amounts extracted from each facial image, and determining that the facial images match if the score is higher than a threshold value. Note that the process of determining whether the facial images match may be performed by any method.

[0018] When the control unit 12 determines that the face image of a person exhibiting suspicious behavior is registered in the registration list 15, it sets a denial setting for the request made by that person. In this case, it sets a denial setting for passing through the authentication gate G. For example, it adds gate-passing denial information to the face image registered in the registration list 15.

[0019] The authentication unit 13 uses an authentication camera C2 installed near the authentication gate G to capture a facial image of a person approaching the gate, and performs authentication to determine whether the facial image has been registered in advance before opening or closing the authentication gate G. At this time, the authentication unit 13 determines whether the facial image captured by the authentication camera C2 matches a facial image in a registration list 15, which is a blacklist. In particular, the authentication unit 13 determines whether the captured facial image matches a facial image in the registration list 15 that has been assigned gate passage denial information. Note that the determination of whether the facial images match is similar to that described above, for example, by calculating a score such as the similarity of feature amounts extracted from each facial image, and determining that the facial images match if the score is higher than a threshold value.

[0020] If the captured facial image matches a facial image with gate denial information, the authentication unit 13 keeps the authentication gate G closed to deny the person passage. In other words, even if the person in the captured facial image has the authority to pass through the authentication gate G, the authentication unit 13 keeps the authentication gate G closed if the person is set as gate denial in the registration list 15. As a result, the person cannot pass through the authentication gate G, and is denied entry to or exit from the predetermined area.

[0021] If the captured facial image matches a facial image to which gate refusal information has been added, the authentication unit 13 may determine that the authentication process for that person has failed and output a message to that effect. In this way, it is possible to hide from the person the fact that they are being watched.

[0022] [Operation] Next, the operation of the control server 1 will be described with reference to the flowcharts of FIGS.

[0023] First, the control server 1 analyzes the image captured by the surveillance camera C1 to detect a person behaving suspiciously (step S1). If the control server 1 detects a person behaving suspiciously (Yes in step S2), it identifies the face area of ​​the person and extracts features of the face image (step S3).

[0024] Next, the control server 1 determines whether the facial image of the person exhibiting suspicious behavior is registered in the registration list 15, which is a blacklist, based on the feature values ​​of the facial image of the person (step S4). If the facial image of the person is registered in the registration list 15 (Yes in step S4), information indicating that the person is denied passage through the authentication gate G is assigned to the matching facial image in the registration list 15 (step S5). At the same time, an alert is output to a pre-set device indicating that a person exhibiting suspicious behavior has been present (step S6).

[0025] Meanwhile, at authentication gate G, authentication camera C2 captures a facial image of a person approaching the gate, analyzes the facial image, and authenticates whether the person has been registered in advance (step S11). At this time, control server 1 extracts features of the facial image captured by authentication camera C2 (step S12) and determines whether the facial image matches a facial image in registration list 15 to which gate passage denial information has been added (step S13). If the captured facial image matches a facial image to which gate passage denial information has been added (Yes in step S13), control server 1 locks authentication gate G without opening it to deny passage of the person (step S14).

[0026] Although the above example illustrates a case where a person behaving suspiciously is denied passage through authentication gate G, control may be exercised to deny any request made by a person behaving suspiciously. For example, when an elevator surveillance camera detects a person who has been set as denied in registration list 15, control may be exercised to disable the elevator operation buttons.

[0027] Furthermore, in the above, a face image is used to determine whether the person matches, but biometric information of other parts of the person may also be used to determine whether the person matches a person in the registration list.

[0028] As described above, according to the present invention, if a person behaving suspiciously is registered on a pre-prepared registration list, the system will deny the person from passing through the gate, thereby refusing any requests made by that person. This makes it possible to quickly deter the person who has behaved suspiciously from taking further action, thereby improving security and safety.

[0029] While the above description exemplifies a case in which the registration list 15 is a blacklist, the registration list 15 may also be a whitelist that stores biometric information such as facial images of persons for whom certain authorization is valid. For example, the authorization given to a person on the whitelist may be permission to pass through the authentication gate G. Even in this case, when the control unit 12 determines that the facial image of a person engaging in suspicious behavior is registered in the registration list 15, which is a whitelist, the control unit 12 sets a denial setting for the request of that person. In other words, the control unit 12 assigns gate-passing denial information to the facial image registered in the registration list 15, which is a whitelist, thereby invalidating the authorization to pass through the authentication gate G.

[0030] Then, if the facial image captured by the authentication camera C2 matches a facial image in the registration list 15, which is a whitelist, that has been assigned gate passage denial information, the authentication unit 13 keeps the authentication gate G closed to deny passage to that person. In other words, even if the person in the captured facial image is registered on the whitelist that has the authority to pass through the authentication gate G, the authentication unit 13 keeps the authentication gate G closed if the person is set as gate passage denial in the registration list 15. As a result, the person in question is denied entry into or exit from the predetermined area through the authentication gate G.

[0031] <Embodiment 2> Next, a second embodiment of the present invention will be described with reference to Fig. 5 to Fig. 8. Fig. 5 to Fig. 6 are diagrams for explaining the configuration of an information processing system. Fig. 7 and Fig. 8 are diagrams for explaining the operation of the information processing system.

[0032] [composition] The information processing system of the present invention has a configuration similar to that of the system of the above-described embodiment 1. However, in this embodiment, the configuration of the registration list, the processing performed on the registration list, and the authentication method by the authentication gate G are different. The following mainly describes the different configuration in detail.

[0033] 5, the information processing system includes a surveillance camera C1 that monitors a specific area A, a control server 1, an authentication gate G that is an entrance / exit to a predetermined area, and a card reader R that reads an authentication card D of a person passing through the authentication gate G. Each component will be described in detail below.

[0034] The control server 1 has a configuration similar to that of the first embodiment, and includes a detection unit 11 (detection means), a control unit 12 (control means), and an authentication unit 13, which are constructed by a computing device executing a program, as shown in Fig. 6. The control server 1 also includes a registration list 15 (second registration list) for registering blacklists and a registration list 16 (first registration list) for registering whitelists, which are formed in a storage device.

[0035] Among the above registration lists, the blacklist 15 registers facial image information of individuals previously identified as blacklists, along with the individual's identification information. Meanwhile, among the registration lists, the whitelist 16 registers facial image information of individuals who have the authority to pass through the authentication gate G or have some authority over other devices or systems, along with the individual's identification information. Here, the registered facial image information may be the facial image itself, or may be features extracted from the facial image. Note that the information of individuals registered in the registration lists 15 and 16 is not limited to facial image information, but may also be biometric information representing other physical features.

[0036] As in the first embodiment, the detection unit 11 detects a person engaging in a specific behavior from a captured image captured within the specific area A. When the detection unit 11 detects a person engaging in suspicious behavior, it passes the captured image to the control unit 12.

[0037] When the control unit 12 acquires a captured image of a person exhibiting suspicious behavior, it extracts a facial image of the person from the captured image and extracts features of the facial image. Then, the control unit 12 determines whether the extracted facial image matches a facial image in the blacklist 15 and whether it matches a facial image in the whitelist 16. The determination of whether the facial images match may be performed in the same manner as in the first embodiment, or in any other manner.

[0038] When the control unit 12 determines that the facial image of a person behaving suspiciously is registered in the whitelist 16, it deletes the facial image of the person from the whitelist 16 and registers it in the blacklist 15. At this time, at the same time as registering the person in the blacklist 15, a denial setting is made for requests made by the person. In this case, a setting is made to deny passage through the authentication gate G. For example, information indicating denial of passage through the gate is added to the facial image registered in the blacklist 15. When the control unit 12 determines that the facial image of a person behaving suspiciously is registered in the blacklist 15, the control unit 12 sets a denial setting for requests made by the person, as in the first embodiment.

[0039] The authentication unit 13 uses a card reader R installed at the authentication gate G to read an authentication card D presented by a person attempting to pass through the authentication gate G. The authentication card D stores the person's identification information in a storage device, and the authentication unit 13 authenticates whether the read identification information has been previously registered and opens or closes the authentication gate G. At this time, the authentication unit 13 determines whether the identification information read from the authentication card D matches identification information in the blacklist 15 that has been assigned information indicating denial of gate passage. If the identification information read from the authentication card D matches identification information assigned information indicating denial of gate passage, the authentication unit 13 keeps the authentication gate G closed to deny the person passage. In other words, even if the person identified by the identification information read from the authentication card D has the authority to pass through the authentication gate G, if the person is set as denied passage through the gate in the blacklist 15, the authentication unit 13 keeps the authentication gate G closed. As a result, the person is denied entry into or exit from the predetermined area after passing through the authentication gate G.

[0040] [Operation] Next, the operation of the control server 1 will be described with reference to the flowcharts of FIGS.

[0041] First, the control server 1 analyzes the image captured by the surveillance camera C1 to detect a person behaving suspiciously (step S21). If the control server 1 detects a person behaving suspiciously (Yes in step S22), it identifies the face area of ​​the person and extracts features of the face image (step S23).

[0042] The control server 1 determines whether the facial image of the person exhibiting suspicious behavior is registered in the blacklist 15 and the whitelist 16 based on the features of the facial image of the person (step S24). If the facial image of the person is registered in the whitelist 16 (Yes in step S24), the control server 1 deletes the information in the whitelist 16, registers the person in the blacklist 15, and assigns information to the authentication gate G that the person is denied passage through the gate (step S25). At the same time, an alert is output to a pre-set device indicating the presence of a person exhibiting suspicious behavior (step S26). If the person exhibiting suspicious behavior is registered in the blacklist 15, the same processing as in embodiment 1 is performed.

[0043] Meanwhile, at authentication gate G, card reader R reads identification information from authentication card D of a person approaching the gate, and performs authentication to determine whether or not the identification information has been registered in advance (step S31). If the read identification information is registered in whitelist 16 (whitelist in step S32), control server 1 opens the authentication gate (step S33). On the other hand, if the read identification information is registered in blacklist 15 and has been assigned information that denies passage through the gate (blacklist in step S32), control server 1 locks authentication gate G without opening it to deny passage of the person (step S34). If the read identification information is registered in blacklist 15 and has not been assigned information that denies passage through the gate, the authentication gate opens.

[0044] In the above example, registration on the whitelist 16 and the blacklist 15 is unrelated to the persons permitted to be authenticated at the authentication gate G, but persons permitted to pass through the authentication gate G may be registered on the whitelist 16, and persons denied authentication at the authentication gate G may be registered on the blacklist 15. In this case, as described above, there is no need to add gate passage denial information to the identification information, and it is sufficient to simply change the registration of the person who engaged in suspicious behavior from the whitelist 16 to the blacklist 15. As a result, the authentication unit 13 allows the person attempting to pass if they are registered on the whitelist 16, and denies them if they are registered on the blacklist 15.

[0045] Furthermore, although the above example illustrates a configuration including the blacklist 15 and the whitelist 16, only the whitelist 16 may be provided. For example, the authority given to a person on the whitelist 16 may be permission to pass through the authentication gate G. Even in this case, when the control unit 12 determines that the identification information of a person engaging in suspicious behavior is registered on the whitelist 16, it keeps the person registered on the whitelist 16 and sets a denial setting for requests made by that person. In other words, by adding gate denial information to the identification information registered on the whitelist 16, the authority to pass through the authentication gate G is invalidated, and as a result, the authentication card D (permit) is invalidated.

[0046] Then, if the identification information read from the authentication card D matches identification information in the whitelist 16 that has been assigned gate passage denial information, the authentication unit 13 keeps the authentication gate G closed without opening it to deny passage to that person. In other words, even if the person in the photographed face image is registered in the whitelist 16 as having the authority to pass through the authentication gate G, the authentication unit 13 keeps the authentication gate G closed if the gate passage denial setting is set. As a result, the person in question cannot pass through the authentication gate G, and is denied entry to or exit from the specified area.

[0047] As described above, according to the present invention, if a person behaving suspiciously is registered on a pre-prepared registration list, the system will deny the person from passing through the gate, thereby refusing any requests made by that person. This makes it possible to quickly deter the person who has behaved suspiciously from taking further action, thereby improving security and safety.

[0048] <Embodiment 3> Next, a third embodiment of the present invention will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining the operation of an information processing system.

[0049] [composition] The information processing system of the present invention has almost the same configuration as the system of the above-described embodiment 2. However, in this embodiment, the processing performed on the registration list is different. The following mainly describes the different configuration in detail.

[0050] First, as shown in Fig. 5, the information processing system of this embodiment, like the information processing system of embodiment 2, includes a surveillance camera C1 that monitors a specific area A, a control server 1, an authentication gate G that is an entrance / exit to a predetermined area, and a card reader R that reads an authentication card D of a person passing through the authentication gate G. Each component will be described in detail below.

[0051] The control server 1 has a configuration similar to that of the second embodiment, and includes a detection unit 11 (detection means), a control unit 12 (control means), and an authentication unit 13, which are constructed by a computing device executing a program, as shown in Fig. 6. The control server 1 also includes a registration list 15 (second registration list) for registering blacklists and a registration list 16 (first registration list) for registering whitelists, which are formed in a storage device.

[0052] Of the above registration lists, the blacklist 15 registers facial image information (first biometric information) of a person previously identified as a person requiring attention, along with the person's identification information. In addition, the blacklist 15 registers biometric information (second biometric information) different from the facial image information of a person who is later identified as a person requiring attention by engaging in suspicious behavior, such as biometric information about a person's physique. Note that the biometric information different from the facial image information registered in the blacklist 15 may be any information as long as it is biometric information of a person that can be extracted from a captured image.

[0053] On the other hand, the whitelist 16 of the registration lists, as in the second embodiment, registers facial image information of persons who have some authority over other devices or systems, such as the authority to pass through the authentication gate G, along with the person's identification information.

[0054] The facial image information registered in the blacklist 15 and the whitelist 16 may be the facial image itself or may be feature values ​​extracted from the facial image. However, the facial image information (first biometric information) registered in the registration lists 15 and 16 may be other biometric information of a person, and in this case, the other biometric information (second biometric information) registered in the blacklist 15 is yet another biometric information of the person.

[0055] The control unit 12 first extracts a facial image of a person appearing in a captured image from the captured image and extracts feature quantities of the facial image. At this time, the control unit 12 also extracts other biometric information, such as physique information, that is different from the feature quantities of the facial image of the person in the captured image. The control unit 12 then determines whether the extracted facial image or physique information matches a facial image or physique information in the blacklist 15. If the control unit 12 determines that the facial image or physique information in the captured image matches that in the blacklist 15, it performs processing to output an alert. For example, the position of a surveillance camera capturing the extracted person as described above may be reported to those around the person as the person's current location.

[0056] Furthermore, when the detection unit 11 detects a person behaving suspiciously, the control unit 12 acquires the captured image of the person, extracts a facial image of the person behaving suspiciously from the captured image, and extracts feature quantities of the facial image. At this time, the control unit 12 may not be able to extract the facial image of the person behaving suspiciously from the captured image of the person, or may not be able to extract feature quantities of the facial image. For example, there may be cases where the person's face cannot be captured because they are not facing the surveillance camera C1, or the facial image is unclear. In such cases, the control unit 12 extracts biometric information (second biometric information), such as physique information, which is different from the feature quantities of the facial image (first biometric information), from the captured image of the person behaving suspiciously.

[0057] Then, the control unit 12 registers the extracted physique information in the blacklist 15. As a result, even if the control unit 12 cannot extract a facial image of a person appearing in a captured image as described above, if the control unit 12 can extract physique information of a person who has engaged in suspicious behavior, the control unit 12 can output an alert as a person registered in the blacklist 15.

[0058] When the control unit 12 is able to extract the facial image information of a person who has behaved suspiciously, it determines whether or not the facial image of that person is registered in the blacklist 15 or the whitelist 16, as described in the first and second embodiments. If the facial image extracted from the person who has behaved suspiciously is registered in the blacklist 15 or the whitelist 16, it performs a refusal setting for requests made by that person, as described in the first and second embodiments.

[0059] Furthermore, even if the control unit 12 is able to extract facial image information of a person who has engaged in suspicious behavior, if the facial image of such person is not registered in the blacklist 15 or the whitelist 16, the control unit 12 extracts other biometric information different from the facial image information and registers it in the blacklist 15.

[0060] [Operation] Next, the operation of the control server 1 described above will be explained with reference to the flowchart of FIG.

[0061] First, the control server 1 analyzes the captured image taken by the monitoring camera C1 (step S41). At this time, if it is possible to extract biometric information such as facial features and other features such as physique information of the person appearing in the captured image (Yes in step S42), the control server 1 extracts the biometric features (step S43).

[0062] Then, the control server 1 determines whether the extracted face image or physique information matches a face image or physique information registered in the blacklist 15 (step S44). If the control server 1 determines that the face image or physique information of the person in the captured image matches that in the blacklist 15 (Yes in step S44), it performs processing to output an alert (step S45).

[0063] In addition to the above-described processing, the control server 1 also detects a person exhibiting suspicious behavior from the captured image (step S46). When the control server 1 detects a person exhibiting suspicious behavior (Yes in step S46), it identifies the face area of ​​the person and extracts features of the face image (step S47). If the control server 1 has successfully extracted features of the face image of the person exhibiting suspicious behavior (Yes in step S47), it determines whether the face image of the person exhibiting suspicious behavior is registered on the whitelist 16 (step S48). If the face image of the person exhibiting suspicious behavior is registered on the whitelist 16 (Yes in step S48), it performs processing to reject requests from the person, as in the second embodiment (step S49). At this time, the control server 1 may also check whether the person exhibiting suspicious behavior is registered on the blacklist 15, and if the person is registered on the blacklist 15, it may perform processing to reject requests from the person, as in the first embodiment.

[0064] On the other hand, if the control server 1 cannot extract features of the face image of the person who engaged in suspicious behavior, for example, because the face image of the person does not appear in the captured image (No in step S47), the control server 1 extracts other biometric information, such as physique information, from the captured image of the person engaging in suspicious behavior (step S50).Then, the control server 1 registers the extracted physique information in the blacklist 15 (step S51).Note that even if the control server 1 is able to extract the face image of the person who engaged in suspicious behavior, if the face image is not registered in the whitelist 16 or the blacklist 15, the control server 1 extracts other biometric information, such as physique information, of the person and registers it in the blacklist 15, as described above.

[0065] As described above, by registering biometric information different from the feature amount of the facial image of a person who has behaved suspiciously in the blacklist 15, even if the facial image of the person cannot be extracted from the captured image, if other biometric information of the person who has behaved suspiciously can be extracted, an alert can be output as the person is registered in the blacklist 15. This can improve security and safety.

[0066] <Embodiment 4> Next, a fourth embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of an information processing system according to the fourth embodiment. Note that this embodiment shows an outline of the configuration of the information processing system described in the third embodiment.

[0067] As shown in FIG. 10, the information processing system 100 according to this embodiment includes: a registration list 110 storing biometric information of a person; detection means 120 for detecting a person engaging in a specific behavior from an image captured by a camera; a control means 130 for extracting biometric information of the person exhibiting the detected specific behavior from the captured image, and extracting other biometric information from the person exhibiting the detected specific behavior and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; Equipped with.

[0068] According to the above invention, first, when a person exhibiting specific behavior that is deemed suspicious in advance is detected from an image captured by a camera, a check is made to see if the person's biometric information, such as a facial image, is registered in a registration list. In this case, there are cases where the person's facial image cannot be extracted from the image of the person exhibiting suspicious behavior, resulting in the person not being registered in the registration list, or where a facial image can be extracted but not registered in the registration list. In such cases, a process is performed to extract other biometric information from the image of the person exhibiting suspicious behavior that is different from the biometric information to be compared with the registration list, and register the extracted information in the registration list. This allows the other biometric information of the person exhibiting suspicious behavior to be compared with the registration list later, thereby deterring suspicious behavior and improving security and safety.

[0069] This embodiment may also have the following configuration, and the information processing system 100 shows an outline of the configuration of the information processing system described in the first and second embodiments. As shown in FIG. 10, the information processing system 100 according to this embodiment includes: a registration list 110 storing biometric information of a person; detection means 120 for detecting a person engaging in a specific behavior from an image captured by a camera; a control means 130 for extracting biometric information of the detected person from the photographed image and performing predetermined control to reject a request made by the detected person according to the registration status of the extracted biometric information in the registration list; Equipped with.

[0070] According to the above invention, when a person exhibiting a specific behavior that is deemed suspicious in advance is detected from an image captured by a camera, the system checks whether the person's biometric information, such as a facial image, is registered in a registration list. Then, depending on the registration status of the detected person in the registration list, a predetermined control is performed to deny the person's request. For example, if the registration list is a blacklist, when the detected person attempts to enter or exit a predetermined area through an authentication gate opened and closed by an authentication system using biometric information, the authentication process is deemed to have failed, and the authentication gate is controlled not to open, thereby denying entry or exit to the predetermined area. This allows for the suspicious person's behavior to be quickly deterred, thereby improving security and safety.

[0071] <Additional Notes> A part or all of the above-described embodiments can be described as follows: The following provides an overview of the configurations of the information processing system, program, and control method according to the present invention. However, the present invention is not limited to the following configurations.

[0072] (Appendix 1) a registration list storing biometric information of people; a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; An information processing system comprising:

[0073] (Appendix 2) 10. The information processing system of claim 1, when first biometric information for comparison with the registration list cannot be extracted from the detected person exhibiting specific behavior, the control means extracts second biometric information different from the first biometric information from the detected person exhibiting specific behavior and registers the second biometric information in the registration list; Information processing system.

[0074] (Appendix 3) 3. The information processing system according to claim 1, the registration list includes a first registration list and a second registration list; The control means if the first biometric information extracted from the detected person exhibiting specific behavior is not registered in the first registration list, extracting second biometric information different from the first biometric information from the detected person exhibiting specific behavior and registering the second biometric information in the second registration list; Information processing system.

[0075] (Appendix 4) 4. The information processing system according to claim 3, the control means extracts the first biometric information and the second biometric information of the person from the captured image, and performs a predetermined alert process when the first biometric information or the second biometric information is registered in the second registration list. Information processing system.

[0076] (Appendix 5) 5. The information processing system according to claim 3, the control means extracts, as the first biometric information, a feature amount based on a face image of the person, and extracts, as the second biometric information, biometric information different from the feature amount based on the face image of the person; Information processing system.

[0077] (Appendix 6) An information processing system according to any one of Supplementary Notes 3 to 5, the control means performs predetermined control to reject a request from a person who has detected a specific behavior when the biometric information extracted from the person is registered in the first registration list; Information processing system.

[0078] (Appendix 6.1) a registration list storing biometric information of people; a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and performing predetermined control to reject a request made by the detected person in accordance with the registration status of the extracted biometric information in the registration list; An information processing system comprising:

[0079] (Appendix 6.2) 1. An information processing system according to claim 6 or 6.1, the control means performs predetermined control to deny entry and / or exit of a person to a predetermined area when the biometric information extracted from the person exhibiting the detected specific behavior is registered in the first registration list. Information processing system.

[0080] (Appendix 6.3) An information processing system according to any one of Supplementary Notes 6 to 6.2, When the biometric information extracted from the detected person exhibiting the specific behavior is registered in the first registration list, the control means controls the authentication system that performs authentication processing using biometric information so that the authentication processing using the biometric information extracted from the detected person exhibiting the specific behavior fails. Information processing system.

[0081] (Appendix 6.4) 6. An information processing system according to claim 6.3, comprising: When the biometric information extracted from the detected person exhibiting the specific behavior is registered in the first registration list, the control means controls the authentication system so that the authentication process using the biometric information extracted from the detected person exhibiting the specific behavior fails, and controls an authentication gate that opens and closes depending on the result of the authentication process by the authentication system so as not to open. Information processing system.

[0082] (Appendix 7) An information processing system according to any one of Supplementary Notes 6 to 6.4, When the biometric information extracted from the person exhibiting the detected specific behavior is registered in the first registration list, the control means deletes the biometric information of the person from the first registration list and registers it in the second registration list. Information processing system.

[0083] (Appendix 8) 8. An information processing system according to any one of Supplementary Notes 6 to 7, the control means performs control to invalidate the authority associated with the person when the biometric information extracted from the person exhibiting the detected specific behavior is registered in the first registration list. Information processing system.

[0084] (Appendix 8.1) 9. The information processing system according to claim 8, the control means performs control to invalidate a permit issued in association with the detected person when the biometric information extracted from the detected person is registered in the first registration list; Information processing system.

[0085] (Appendix 9) In the information processing device, a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; A program to achieve this.

[0086] (Appendix 10) 10. The program of claim 9, when first biometric information for comparison with the registration list cannot be extracted from the detected person exhibiting specific behavior, the control means extracts second biometric information different from the first biometric information from the detected person exhibiting specific behavior and registers the second biometric information in the registration list; program.

[0087] (Appendix 11) an information processing device having a registration list storing biometric information of a human body, Detects people engaging in specific behavior from images captured by a camera, extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and performing a process of extracting other biometric information from the person exhibiting the specific behavior detected and registering it in the registration list according to the registration status of the extracted biometric information in the registration list; Control method.

[0088] (Appendix 12) 12. The control method of claim 11, further comprising: when first biometric information for matching with the registration list cannot be extracted from the detected person exhibiting specific behavior, second biometric information different from the first biometric information is extracted from the detected person exhibiting specific behavior, and the second biometric information is registered in the registration list; Control method.

[0089] (Appendix 13) 13. The control method according to claim 11 or 12, the registration list includes a first registration list and a second registration list; if the first biometric information extracted from the detected person exhibiting specific behavior is not registered in the first registration list, extracting second biometric information different from the first biometric information from the detected person exhibiting specific behavior and registering the second biometric information in the second registration list; Control method.

[0090] (Appendix 14) 14. The control method of claim 13, further comprising: extracting the first biometric information and the second biometric information of the person from the captured image, and performing a predetermined alert process if the first biometric information or the second biometric information is registered in the second registration list; Control method.

[0091] (Appendix 15) 15. The control method according to claim 13 or 14, extracting, as the first biometric information, a feature amount based on a face image of a person, and extracting, as the second biometric information, biometric information different from the feature amount based on the face image of the person; Information processing system.

[0092] (Appendix 16) 16. The control method according to any one of Supplementary Notes 13 to 15, When the biometric information extracted from the person exhibiting the specific behavior detected is registered in the first registration list, a predetermined control is performed to reject the request action of the person. Control method.

[0093] (Appendix 17) an information processing device having a registration list storing biometric information of a person, Detects people engaging in specific behavior from images captured by a camera, extracting biometric information of the person exhibiting the specific behavior detected from the captured image, and performing predetermined control to reject a request made by the detected person depending on the registration status of the extracted biometric information in the registration list; Control method.

[0094] (Appendix 18) 18. The control method according to any one of Supplementary Notes 13 to 17, When biometric information extracted from the person exhibiting the detected specific behavior is registered in the first registration list, the biometric information of the person is deleted from the first registration list and registered in the second registration list. Control method.

[0095] (Appendix 19) 19. The control method according to any one of Supplementary Notes 16 to 18, When biometric information extracted from the person exhibiting the detected specific behavior is registered in the first registration list, control is performed to invalidate the authority associated with the person. Control method.

[0096] (Appendix 20) 19. The control method of claim 18, further comprising: When the biometric information extracted from the detected person is registered in the first registration list, a control is performed to invalidate a permit issued in association with the person. Control method.

[0097] (Appendix A1) a registration list storing biometric information of people; a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the detected person from the captured image and performing predetermined control to reject a request made by the detected person depending on the registration status of the extracted biometric information in the registration list; An information processing system comprising:

[0098] (Appendix A2) An information processing system according to claim A1, the control means performs predetermined control to deny the person from entering and / or exiting a predetermined area when the biometric information extracted from the detected person is registered in the registration list. Information processing system.

[0099] (Appendix A3) An information processing system according to appendix A1 or A2, the control means controls an authentication system that performs authentication processing using biometric information so that, when the biometric information extracted from the detected person is registered in the registration list, the authentication system determines that the authentication processing using the biometric information extracted from the detected person has failed. Information processing system.

[0100] (Appendix A4) 1. An information processing system according to claim 1, further comprising: When the biometric information extracted from the detected person is registered in the registration list, the control means controls the authentication system so that the authentication process using the biometric information extracted from the detected person fails, and controls an authentication gate that opens and closes depending on the result of the authentication process by the authentication system so that it does not open. Information processing system.

[0101] According to the above invention, when a person exhibiting a specific behavior that is deemed suspicious in advance is detected from an image captured by a camera, the system checks whether the person's biometric information, such as a facial image, is registered in a registration list. Then, depending on the registration status of the detected person in the registration list, a predetermined control is performed to deny the person's request. For example, if the registration list is a blacklist, when the detected person attempts to enter or exit a predetermined area through an authentication gate opened and closed by an authentication system using biometric information, the authentication process is deemed to have failed, and the authentication gate is controlled not to open, thereby denying entry or exit to the predetermined area. This allows for the suspicious person's behavior to be quickly deterred, thereby improving security and safety.

[0102] (Appendix A5) An information processing system according to any one of appendices A1 to A4, the registration list includes a first registration list and a second registration list; When the biometric information extracted from the detected person is registered in the first registration list, the control means deletes the biometric information of the person from the first registration list and registers it in the second registration list, and performs predetermined control to reject a request action from the person registered in the second registration list. Information processing system.

[0103] (Appendix A6) An information processing system according to any one of appendices A1 to A5, The registration list includes a whitelist storing biometric information of persons for whom associated authorization is valid; the control means performs control to invalidate the authority associated with the detected person when the biometric information extracted from the person is registered in the whitelist; Information processing system.

[0104] (Appendix A7) An information processing system according to Appendix A6, the control means performs control to invalidate a permit issued in association with a detected person when the biometric information extracted from the person is registered in the whitelist; Information processing system.

[0105] According to the above invention, first, when a person exhibiting specific behavior that is deemed suspicious in advance is detected from an image captured by a camera, the biometric information of the person, such as a facial image, is checked to see if it is registered on a whitelist. If the detected person is registered on the whitelist, the person is removed from the whitelist and registered on a blacklist, or the person's license is invalidated, thereby invalidating the person's granted authority. As a result, even if a person registered on the whitelist exhibits suspicious behavior, predetermined control is performed to deny the person's requests. As a result, further suspicious behavior by the suspicious person can be quickly deterred, thereby improving security and safety.

[0106] (Appendix A8) In the information processing device, a detection means for detecting a person engaging in a specific behavior from an image captured by a camera; a control means for extracting biometric information of the detected person from the captured image and performing predetermined control to reject a request made by the detected person according to the registration status of the extracted biometric information in a registration list; A program to achieve this.

[0107] (Appendix A8.1) A program according to Appendix A8, the control means performs predetermined control to deny the person from entering and / or exiting a predetermined area when the biometric information extracted from the detected person is registered in the registration list. program.

[0108] (Appendix A8.2) 1. A program according to claim A8 or A8.1, the control means controls an authentication system that performs authentication processing using biometric information so that, when the biometric information extracted from the detected person is registered in the registration list, the authentication system determines that the authentication processing using the biometric information extracted from the detected person has failed. program.

[0109] (Appendix A8.3) 1. The program according to claim 1, further comprising: When the biometric information extracted from the detected person is registered in the registration list, the control means controls the authentication system so that the authentication process using the biometric information extracted from the detected person fails, and controls an authentication gate that opens and closes depending on the result of the authentication process by the authentication system so that it does not open. program.

[0110] (Appendix A8.4) A program according to any one of appendices A8 to A8.3, the registration list includes a first registration list and a second registration list; When the biometric information extracted from the detected person is registered in the first registration list, the control means deletes the biometric information of the person from the first registration list and registers it in the second registration list, and performs predetermined control to reject a request action from the person registered in the second registration list. program.

[0111] (Appendix A8.5) A program according to any one of appendices A8 to A8.4, The registration list includes a whitelist storing biometric information of persons for whom associated authorization is valid; the control means performs control to invalidate the authority associated with the detected person when the biometric information extracted from the person is registered in the whitelist; program.

[0112] (Appendix A8.6) 1. The program of claim 1, further comprising: the control means performs control to invalidate a permit issued in association with a detected person when the biometric information extracted from the person is registered in the whitelist; program.

[0113] (Appendix A9) Detects people engaging in specific behavior from images captured by a camera, extracting biometric information of the detected person from the captured image, and performing predetermined control to reject a request from the detected person depending on the registration status of the extracted biometric information in a registration list; Control method.

[0114] (Appendix A9.1) A control method according to Appendix A9, comprising: If the biometric information extracted from the detected person is registered in the registration list, a predetermined control is performed to deny the person from entering and / or exiting a predetermined area. Control method.

[0115] (Appendix A9.2) 1. A control method according to claim A9 or A9.1, comprising: When the biometric information extracted from the detected person is registered in the registration list, an authentication system that performs authentication processing using biometric information controls the authentication system so that the authentication processing using the biometric information extracted from the detected person fails. Control method.

[0116] (Appendix A9.3) 1. A control method according to claim A9.2, comprising: When the biometric information extracted from the detected person is registered in the registration list, the authentication system controls the authentication system so that the authentication process using the biometric information extracted from the detected person fails, and also controls an authentication gate that opens and closes depending on the result of the authentication process by the authentication system so that it does not open. Control method.

[0117] (Appendix A9.4) 1. A control method according to any one of appendices A9 to A9.3, comprising: the registration list includes a first registration list and a second registration list; If the biometric information extracted from the detected person is registered in the first registration list, the biometric information of the person is deleted from the first registration list and registered in the second registration list, and a predetermined control is performed to reject requests from the person registered in the second registration list. Control method.

[0118] (Appendix A9.5) 1. A control method according to any one of appendices A9 to A9.4, comprising: The registration list includes a whitelist storing biometric information of persons for whom associated authorization is valid; If the biometric information extracted from the detected person is registered in the whitelist, the authority associated with the person is invalidated. Control method.

[0119] (Appendix A9.6) 1. A control method according to claim A9.5, comprising: The control means performs control to invalidate an ID card issued in association with a detected person when the biometric information extracted from the person is registered in the whitelist. Control method.

[0120] The above-mentioned program is stored in a storage device or a computer-readable recording medium, such as a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0121] Although the present invention has been described above with reference to the above-mentioned embodiments, the present invention is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention. [Explanation of symbols]

[0122] 1 Control Server 11 Detection unit 12 Control Unit 13 Authentication Section 15 Registration list (blacklist) 16 Registration list (whitelist) 100 Information Processing Systems 110 Registration List 120 Detection means 130 Control Means A Specific Area C1 Surveillance Camera C2 Authentication Camera D Authentication Card G Authentication Gate R card reader

Claims

1. An extraction means for extracting a movement trajectory of a person from an image captured by a camera; A detection means for detecting a person engaging in a specific behavior based on the movement trajectory; a control means for performing a process of extracting second biometric information different from the first biometric information from the person engaging in specific behavior and registering the second biometric information in the registration list when first biometric information for matching with biometric information of a specific person registered in advance in a registration list cannot be extracted from the person engaging in specific behavior; An information processing system comprising:

2. The control means and when the first biometric information extracted from the detected person engaging in the specific behavior is not registered in the registration list, extracting the second biometric information different from the first biometric information from the detected person engaging in the specific behavior and registering the second biometric information in the registration list. The information processing system according to claim 1 .

3. the control means extracts the first biometric information and the second biometric information of the person from the captured image, and performs a predetermined alert process when the first biometric information or the second biometric information is registered in the registration list. The information processing system according to claim 2 .

4. the control means extracts, as the first biometric information, a feature amount based on a face image of the person, and extracts, as the second biometric information, biometric information different from the feature amount based on the face image of the person.

4. The information processing system according to claim 2 or 3.

5. the control means performs a predetermined control so as to reject a request action of a person when the first biometric information or the second biometric information extracted from the person who has detected the specific behavior is registered in the registration list.

5. An information processing system according to claim 2.

6. having another registration list different from the registration list; when the first biometric information or the second biometric information extracted from the detected person exhibiting the specific behavior is registered in the other registration list, the control means deletes the first biometric information or the second biometric information of the person from the other registration list and registers it in the registration list.

6. The information processing system according to claim 5.

7. the control means performs control to invalidate an authority associated with a person when the first biometric information or the second biometric information extracted from the person performing the specific behavior detected is registered in the registration list.

7. The information processing system according to claim 5 or 6.

8. On the computer, A process of extracting a person's movement trajectory from an image captured by a camera; A process of detecting a person engaging in a specific behavior based on the movement trajectory; a process of extracting second biometric information different from the first biometric information from the person engaging in specific behavior and registering the second biometric information in the registration list when first biometric information for matching with biometric information of a specific person registered in advance in a registration list cannot be extracted from the person engaging in specific behavior; A program for executing the above.

9. The computer Extract the movement trajectory of a person from the image captured by the camera, Detecting a person engaging in a specific action based on the movement trajectory; when it is not possible to extract, from the person taking the specific behavior, first biometric information for matching with biometric information of a specific person pre-registered in a registration list, extracting, from the person taking the specific behavior, second biometric information different from the first biometric information, and registering the second biometric information in the registration list; Information processing methods.

Images