Information processing system, behavior analysis device, behavior analysis method, and behavior analysis program

Abstract

To provide an information processing system, a behavior analysis device, a behavior analysis method, and a behavior analysis program that have a simple configuration and can detect suspicious behavior with high accuracy.SOLUTION: An imaging device images a monitoring area. A behavior analysis device comprises: a specifying unit that specifies a dead angle area where a monitoring target is present based on detection information for detecting presence or absence of the monitoring target in time series based upon imaging information, and dead angle information which includes position information of the dead angle area where the monitoring target cannot be imaged; a storage unit that stores the detection information and analysis result information obtained by associating the monitoring target with the identified dead angle area; an evaluation unit that increases an evaluation level for indicating a high possibility of a suspicious behavior for the dead angle area by determining that there is a possibility of the suspicious behavior if it is determined that a plurality of monitoring targets is present in the same dead angle area at the same point in time; and a communication unit that transmits evaluation results associated with the analysis result information to a computer device. The computer device receives and outputs the analysis result information and the evaluation result from the behavior analysis device.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing system, a behavior analysis apparatus, a behavior analysis method, and a behavior analysis program. [Background Art]

[0002] Conventionally, systems for detecting suspicious behavior of persons have been widely used to prevent the occurrence of harassment in facilities and the like.

[0003] In such a system, for example, in order to associate video of a person captured by a surveillance camera with the person's position information, it is possible to recognize where and what kind of behavior the person is taking. However, when a person enters a blind spot of the surveillance camera, video of the person cannot be obtained, so even if harassment were occurring in the blind spot, it could not be detected.

[0004] For this reason, in the prior art, there is disclosed a technology in which, when a person takes an exceptional behavior in an area recognizable by a surveillance camera, the person's position information is compared with the position of a blind spot of the surveillance camera, and when the matching degree of the information is high, it is recognized that the person has taken a suspicious behavior. [Summary of the Invention] [Problem to be Solved by the Invention]

[0005] However, the aforementioned technology obtains a person's location information by having the person being monitored carry a device, so detecting suspicious behavior such as harassment in blind spots requires one or more devices in addition to the surveillance camera, which poses a problem in terms of management and operation. Furthermore, since the detection of suspicious behavior is triggered by an exceptional action taken by a person, it is difficult to detect suspicious behavior if, for example, a person is intentionally behaving in a standard manner within the area detectable by the surveillance camera. Because those who intend to commit suspicious acts such as harassment are often aware of the presence of surveillance cameras and refrain from suspicious behavior in the area detectable by the cameras, such conventional technologies could not detect a person's suspicious behavior with sufficient accuracy.

[0006] The present invention has been made in view of the above, and aims to provide an information processing system, a behavioral analysis device, a behavioral analysis method, and a behavioral analysis program that can detect suspicious behavior with a simple configuration and high accuracy. [Means for solving the problem]

[0007] To solve the above-mentioned problems and achieve the objective, the present invention provides an information processing system comprising: a plurality of imaging devices installed in a monitoring area where one or more objects to be monitored exist; a behavior analysis device that analyzes the behavior of the objects to be monitored based on imaging information, which includes at least time information acquired by the imaging devices; and a computer device connected to the behavior analysis device via a network, wherein each of the plurality of imaging devices images the monitoring area, and the behavior analysis device detects the presence or absence of the objects to be monitored in a time series based on the imaging information acquired from the imaging devices, and detects the detection result, and detects the presence or absence of the objects to be monitored based on blind spot information, which includes location information of blind spot areas where the objects to be monitored cannot be imaged by the imaging devices. The device comprises: an identification unit that identifies the blind spot area; a storage unit that stores analysis result information linking the time information of the detection information, the monitored target, and the identified blind spot area in the storage unit; an evaluation unit that, based on the analysis result information, determines that there is a possibility of suspicious behavior by the monitored target when it is determined that multiple monitored targets exist in the same blind spot area at the same time, and performs an evaluation that increases the evaluation level indicating the high probability of suspicious behavior for the blind spot area; and a communication unit that transmits the evaluation result associated with the analysis result information to the computer device. The computer device receives and outputs the analysis result information and the evaluation result associated with the analysis result information from the behavior analysis device. [Effects of the Invention]

[0008] The present invention provides a system that can detect suspicious behavior with a simple configuration and high accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a diagram showing the configuration of an information processing system according to an embodiment. [Figure 2] Figure 2 shows an example of the server hardware configuration in the information processing system according to the embodiment. [Figure 3]Figure 3 shows an example of the functional configuration of a server in the information processing system according to the embodiment. [Figure 4] Figure 4 shows an example of the data structure of the first analysis results DB according to the embodiment. [Figure 5] Figure 5 shows an example of the data structure of the second analysis results DB according to the embodiment. [Figure 6] Figure 6 illustrates the blind spot region in the embodiment. [Figure 7] Figure 7 illustrates the method for identifying blind spots in the embodiment. [Figure 8] Figure 8 illustrates the identification of blind spot lines in the embodiment. [Figure 9] Figure 9 illustrates a new method for identifying blind spots in an embodiment. [Figure 10] Figure 10 is a flowchart showing an example of the suspicious behavior analysis process according to the embodiment. [Figure 11] Figure 11 is a diagram illustrating the authentication method for the monitored target in the embodiment. [Figure 12] Figure 12 shows an example of the first analysis result DB of the embodiment. [Figure 13] Figure 13 shows an example of a second analysis results database according to the embodiment. [Figure 14] Figure 14 shows an example of a second analysis results database according to the embodiment. [Modes for carrying out the invention]

[0010] The embodiments of the information processing system will be described in detail below with reference to the attached drawings. To facilitate understanding of the explanation, the same reference numerals are used for identical components in each drawing whenever possible, and redundant explanations are omitted.

[0011] Figure 1 is a diagram showing the configuration of an information processing system according to an embodiment.

[0012] The information processing system shown in FIG. 1 is configured to include one or a plurality of imaging devices 1-1 to 1-n, that is, n imaging devices (n is any integer value of 1 or more), a behavior analysis device 5 connected to the imaging devices 1-1 to 1-n via a communication network 100, and a computer device 7 connected to the behavior analysis device 5 via a network.

[0013] The imaging devices 1-1 to 1-n are installed in a monitoring area R where one or more monitoring targets M exist, and capture images of the monitoring area R. Specifically, each of the imaging devices 1-1 to 1-n is, for example, a camera, installed at each location in the monitoring area R, and captures images of the monitoring area R for 24 hours. The imaging devices 1-1 to 1-n transmit k pieces (k is any integer value of 1 or more) of imaging information A-1 to A-k including captured images to the behavior analysis device 5. Note that the monitoring target M is, for example, a person.

[0014] The behavior analysis device 5 analyzes the behavior of the monitoring target M based on imaging information A-1 to A-k including at least time information acquired by the imaging devices 1-1 to 1-n, and transmits an analysis result C and an evaluation result E associated with the analysis result C to the computer device 7.

[0015] The computer device 7 is a terminal managed by an administrator or the like who manages the monitoring area R. The computer device 7 receives the analysis result C of the imaging information A-1 to A-k and the evaluation result E from the behavior analysis device 5, and outputs the same.

[0016] Note that when there is no need to individually distinguish the imaging devices 1-1 to 1-n, they may be collectively referred to as the imaging device 1. Also, when there is no need to individually distinguish the imaging information A-1 to A-k, they may be collectively referred to as the imaging information A.

[0017] Next, an example of the hardware configuration of the behavior analysis device 5 configuring the above-described information processing system will be described.

[0018] FIG. 2 is a hardware configuration diagram of the behavior analysis device 5.

[0019] As shown in Figure 2, the behavioral analysis device 5 is built by a computer and includes a CPU 501, ROM 502, RAM 503, HD 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, data bus 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, and media I / F 516.

[0020] Of these, the CPU 501 controls the operation of the entire behavioral analysis device 5. For example, the CPU 501 comprehensively controls the operation of the entire behavioral analysis device 5 by executing a control program related to the suspicious behavior analysis process, which will be described later.

[0021] ROM502 stores programs used to drive CPU501, such as IPL.

[0022] RAM503 is used as the work area for CPU501. HD504 stores various data such as programs.

[0023] The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501.

[0024] Display 506 displays various information such as cursors, menus, windows, text, or images.

[0025] The External Device Connection I / F508 is an interface for connecting various external devices. These external devices include, for example, USB (Universal Serial Bus) memory devices and printers.

[0026] Network I / F 509 is an interface for data communication using communication network 100.

[0027] The data bus 510 is an address bus and data bus used to electrically connect various components such as the CPU 501.

[0028] Furthermore, the keyboard 511 is a type of input device equipped with multiple keys for inputting characters, numbers, various instructions, and so on.

[0029] The pointing device 512 is a type of input means used for selecting and executing various instructions, selecting the object to be processed, moving the cursor, and so on.

[0030] The DVD-RW drive 514 controls the reading or writing of various types of data to the DVD-RW 513, which is an example of a removable recording medium. Note that it is not limited to DVD-RW; DVD-R or other types may also be used.

[0031] The media interface 516 controls the reading or writing (storage) of data to or from the recording medium 515, such as flash memory.

[0032] Next, using Figure 3, we will explain the functional block diagrams of each function realized when the CPU 501 in Figure 2 executes the suspicious behavior analysis program described later.

[0033] Figure 3 shows an example of the functional configuration of the behavioral analysis device 5 in the information processing system according to the embodiment.

[0034] As shown in Figure 3, the behavioral analysis device 5 according to this embodiment has a functional configuration comprising a storage unit 504, a location information management unit 51, an authentication unit 52, a detection unit 53, a identification unit 54, an evaluation unit 55, a storage unit 56, and a communication unit 57.

[0035] The storage unit 504 is a storage medium such as an HDD or SSD (Solid State Drive). The storage unit 504 stores a location information database 100 (hereinafter referred to as the location information DB100), a feature information database 200 (hereinafter referred to as the feature information DB200), and an analysis result database 300 (hereinafter referred to as the analysis result DB300).

[0036] The location information DB100 contains blind spot information, including the location information of blind spot areas DR where imaging of the target M is impossible by the imaging device 1. Specifically, the location information DB100 contains the location information of blind spot areas DR, which are identified by the location information management unit 51 (described later), and blind spot line DL, which is the location where the presence or absence of the target M changes in the imaging information A acquired over time. These are associated with a blind spot area ID, which is unique identification information, and registered as blind spot information. Note that the location information DB100 is not shown in the diagram.

[0037] The Feature Information DB200 contains information about characteristics that identify a person, as well as information about a person's behavioral patterns, as an example of a monitored target M. Specifically, the Feature Information DB200 associates a person's name with their facial photograph and identification information. Furthermore, the Feature Information DB200 associates a person's state with information about the behavioral patterns that characterize that state. Note that the Feature Information DB200 is not shown in the diagram.

[0038] The analysis results DB300 contains analysis result C, which is the result of the suspicious behavior analysis process described later, and evaluation result E, which is an evaluation of analysis result C.

[0039] Figures 4 and 5 show examples of the data structure of the analysis results database. Figure 4 shows the data structure of the first analysis results database 300a, and Figure 5 shows the data structure of the second analysis results database 300b.

[0040] The first analysis result DB300a shown in Figure 4 contains, for example, an analysis result C that associates whether the monitored subject M is captured by one of the imaging devices 1 (camera) at a predetermined time (Yes / No), and if the monitored subject M is not captured by the camera, the predicted blind spot area DR and the time when the subject entered or left the blind spot area DR, and is registered in association with a person ID that uniquely identifies the monitored subject M.

[0041] Furthermore, in the second analysis result DB300b shown in Figure 5, analysis result C is registered, which associates a blind spot area ID with a combination of person IDs of multiple monitored objects M present in the same blind spot area DR at the same time, and the time when these monitored objects M entered and exited the blind spot area DR. This C is associated with a pattern ID that uniquely identifies the combination of person IDs of the monitored objects M. In addition, in the second analysis result DB300b shown in Figure 5, if the same combination of monitored objects M have previously been present together in the same blind spot area DR at the same time, evaluation result E indicating the number of times this has happened is registered, associated with analysis result C.

[0042] The location information management unit 51 manages blind spot information, including the location information of blind spot areas DR where imaging of the monitoring target M is impossible by the imaging device 1. Figure 6 shows an example of imaging information A acquired by the imaging device 1 installed in the monitoring area R. The location information management unit 51 analyzes the imaging information A and recognizes, for example, that the area after turning left at the end of the corridor is blind spot area DR1, the area after turning right at the end of the corridor is blind spot area DR2, the area directly below the imaging device 1 is blind spot area DR3, the area after entering the room on the left is blind spot area DR4, and the area after entering the room on the right is blind spot area DR5. The location information management unit 51 manages the location information of such blind spot areas DR as blind spot information.

[0043] Specifically, the location information management unit 51 acquires map information of the monitoring area R and location information of imaging devices 1-1 to 1-n installed in the monitoring area R, and identifies the blind spot area DR based on the map information and the location information of imaging devices 1-1 to 1-n. That is, the location information management unit 51 analyzes the floor plan of the building in the monitoring area R and the location information of the installation position of imaging device 1 to extract candidate location information that could be the blind spot area DR, and identifies the blind spot area DR based on the candidate location information and imaging information A.

[0044] Figure 7 shows the floor plan of the building in the monitoring area R, and the installation locations of imaging devices 1-1 and 1-2. If we were to attempt to identify the blind spot area DR based on imaging information A acquired from imaging device 1 without considering the building floor plan, blind spot areas DR21 to DR22 would be recognized from imaging information A acquired by imaging device 1-1, and blind spot areas DR23 to DR24 would be recognized from imaging information A acquired by imaging device 1-2. In other words, blind spot areas DR22 and DR23 would be recognized as overlapping. In contrast, the location information management unit 51 determines that blind spot areas DR22 and DR23 represent the same area based on the floor plan of the building in the monitoring area R and the installation location of imaging device 1, thus preventing the same location from being recognized as a blind spot area DR multiple times.

[0045] The location information management unit 51 stores the location information of the blind spot area DR, which has been identified considering the floor plan and the installation position of the imaging device 1, as blind spot information in the location information DB 100.

[0046] Furthermore, the location information management unit 51 identifies the location where a change in the presence or absence of the monitored object M occurs as a blind spot line DL, based on detection information that shows the results of detecting the presence or absence of the monitored object M in a time series based on the imaging information A. The location information of the identified blind spot line DL is associated with the blind spot area DR and stored in the location information DB 100. Specifically, the location information management unit 51 tracks the monitored object M in a time series based on the detection information. For example, as shown in Figure 8, if the monitored object M can no longer be tracked after turning right at the end of a passageway, the location information management unit 51 identifies the location where tracking can no longer be done, i.e., the location where a change in the presence or absence of the monitored object M occurs, as a blind spot line DL2. The location information management unit 51 stores the blind spot line DL2 identified in this way in the location information DB 100, associated with the blind spot area DR2.

[0047] Furthermore, if the blind spot area DR where the monitored target M is located is not identified by the identification unit 54 (described later), the location information management unit 51 recognizes a new blind spot line DL based on the detection information, associates the recognized blind spot line DL with the blind spot area DR, and stores it in the location information DB 100. Here, Figure 9 is a diagram illustrating the registration of a new blind spot area DR. For example, consider a case where a new obstacle, as shown in Figure 9(b), is installed in a location where there was initially no blind spot area DR, as shown in Figure 9(a). Since the blind spot information of the blind spot area DR 40 formed by the obstacle is not registered in the location information DB 100, even if the presence of the monitored target M is not detected, it is not possible to identify the blind spot area DR where the monitored target M is located.

[0048] In response, the location information management unit 51 identifies blind spot lines DL41 to DL42 based on detection information acquired in chronological order, associates them with blind spot areas DR40, and saves them in the location information DB 100 as new blind spot information. In this way, the newly formed blind spot areas DR are automatically identified and registered as blind spot information, making it easy to identify the blind spot areas DR where the monitored target M is located.

[0049] The authentication unit 52 identifies the monitored subject M based on the detection information. Specifically, if the authentication unit 52 recognizes the presence of the monitored subject M from the detection information described later, it identifies the person of the monitored subject M using an image recognition method or the like based on the feature information DB 200, and associates it with a person ID, which is unique identification information that can identify the monitored subject M. The authentication unit 52 performs this identification before the suspicious behavior analysis process described later.

[0050] The authentication unit 52 acquires information about the behavioral patterns of the identified monitored subject M and determines the state of the monitored subject M. Specifically, the authentication unit 52 attempts to determine the state of the monitored subject M using image recognition methods or the like, based on the feature information DB 200. For example, if the monitored subject M exhibits a behavioral pattern such as "going back and forth to the same place," the authentication unit 52 will determine that the monitored subject M is "restless." The authentication unit 52 performs this determination before the suspicious behavior analysis process described later.

[0051] The detection unit 53 detects the presence or absence of the monitored object M in a time series based on imaging information A which includes at least time information acquired from multiple imaging devices 1, and displays the detection result as detection information. Specifically, for example, at a predetermined time, the detection unit 53 determines whether or not the monitored object M is included in the captured image from the imaging information A acquired by the imaging device 1, and stores the determination result as detection information linked to the imaging information A.

[0052] The identification unit 54 identifies the blind spot area DR where the monitored object M is located based on the detection information and the blind spot information. That is, the identification unit 54 analyzes the detection information and identifies the blind spot area DR where the monitored object M is located based on the location information DB 100.

[0053] Specifically, when the identification unit 54 detects the presence or absence of the monitored object M based on the detection information, it identifies the blind spot line DL where the change in presence or absence occurred, and identifies that the monitored object M is present in the blind spot region DR associated with the blind spot line DL. For example, if the identified unit 54 includes the monitored object M in the imaging information A at time t1, but does not include the monitored object M at time t2, it identifies that the monitored object M has entered the blind spot region DR. The identified unit 54 then analyzes the detection information at time t1 and time t2 to identify the blind spot line DL where the change in the presence or absence of the monitored object M occurred. Based on the location information DB 100, the identified unit 54 uses an image recognition method or the like to identify the blind spot region DR where the monitored object M is present.

[0054] On the other hand, for example, if the monitoring target M is not included in the imaging information A at time t3, but is included in the imaging information A at time t4, the identification unit 54 identifies that the monitoring target M has emerged from the blind spot area DR. The identification unit 54 then analyzes the detection information at times t3 and t4 to identify the blind spot line DL where the change in the presence or absence of the monitoring target M occurred. Based on the location information DB 100, the identification unit 54 uses an image recognition method or the like to identify the blind spot area DR from which the monitoring target M emerged.

[0055] Furthermore, the identification unit 54 identifies the blind spot area DR where a specific target M is located. That is, for example, the identification unit 54 identifies the name of the person, etc., of the target M located in the blind spot area DR based on the detection information and the results identified by the authentication unit 52.

[0056] Furthermore, the identification unit 54 identifies the blind spot area DR in which a monitored object M having a predetermined state exists. That is, for example, based on the results identified by the authentication unit 52, the identification unit 54 identifies the state of the monitored object M in the blind spot area DR as "restless" or the like.

[0057] The storage unit 56 stores analysis result C in the analysis result DB 300a as analysis result information that links the time information included in the detection information, the monitored object M, and the identified blind spot area DR. Specifically, for example, the storage unit 56 registers analysis result C in the analysis result DB 300a that associates the person ID of the monitored object M with whether or not the monitored object M is included in the imaging information A acquired by the imaging device 1, the blind spot area DR where the monitored object M is located, the time t1 when the monitored object M entered the blind spot area DR, and the time t2 when it left the blind spot area DR. Alternatively, for example, the storage unit 56 registers analysis result C in the analysis result DB 300b that associates the blind spot area DR with a combination of person IDs of multiple monitored objects M that are present in the monitoring area DR at the same time, the time t3 when the monitored object M entered the monitoring area DR and the time t4 when it left, and a pattern ID indicating the combination of monitored objects M.

[0058] Based on the analysis result C, the evaluation unit 55 determines that there is a possibility of suspicious behavior by the monitored subjects M if it determines that multiple monitored subjects M are present in the same blind spot area DR at the same time, and performs an evaluation that increases the evaluation level indicating the likelihood of suspicious behavior for the blind spot area DR. Here, the evaluation level is the number of times that suspicious behavior was determined. That is, based on the analysis result C, the evaluation unit 55 determines that there is a possibility of suspicious behavior by the monitored subjects M if it determines that multiple monitored subjects M are present in the same blind spot area DR at the same time, and increments the count as an evaluation level for the blind spot area DR. Specifically, for example, if between time t5 and time t6, person A and person B, who are monitored subjects M, are present in the same blind spot area DR1, the evaluation unit 55 determines that there is a possibility of suspicious behavior and counts it as 1 for that blind spot area DR1.

[0059] Furthermore, if the evaluation unit 55 finds that person A and person B are present in the same blind spot area DR1 between time t7 and time t8, it determines that there is a possibility of suspicious behavior and counts up the blind spot area DR1 one more time. In this way, the evaluation unit 55 repeatedly counts up the blind spot area DR each time the same combination of monitored objects M are present in the same blind spot area DR.

[0060] Furthermore, based on the analysis result C, the evaluation unit 55 determines that if multiple monitoring targets M are present in the same blind spot area DR for longer than a predetermined period, it counts them up with a difference in the count. For example, if multiple monitoring targets M are present in the same blind spot area DR for longer than a predetermined time, the evaluation unit 55 determines that there is a high possibility of suspicious behavior and counts up the blind spot area DR by assigning a number greater than 1 to the count.

[0061] Furthermore, based on the analysis result C, the evaluation unit 55 also counts up the count with a difference in the number of occurrences if it determines that a specific target M is present in the same blind spot area DR at the same time, or if it determines that multiple targets M are present in the same blind spot area DR at the same time. For example, if a predetermined target M identified by the authentication unit 52 is present in the same blind spot area DR at the same time as other targets M, the evaluation unit 55 determines that there is a high possibility of suspicious behavior and counts up the count with a value greater than 1. Also, if multiple targets M are present at the same time in a specific blind spot area DR that is not usually accessed, the evaluation unit 55 determines that there is a high possibility of suspicious behavior and counts up the count with a value greater than 1.

[0062] Furthermore, based on the analysis result C, the evaluation unit 55 determines that the number of monitored subjects M present in the same blind spot area DR at the same time meets predetermined conditions, and counts them up with a difference in the count. For example, if 5 to 6 monitored subjects M are present in the same blind spot area DR, the evaluation unit 55 determines that the possibility of suspicious behavior is low and counts them up with a number less than 1.

[0063] The storage unit 56 registers the number of times the evaluation result E, based on the analysis result C, occurs in the analysis result DB 300.

[0064] Furthermore, the aforementioned suspicious behavior includes acts of harassment.

[0065] The communication unit 57 outputs an evaluation result E corresponding to the analysis result C. Specifically, for example, if the number of times an evaluation result E occurs exceeds a predetermined value, the communication unit 57 determines that there is a high risk of harassment and transmits the analysis result C and the evaluation result E corresponding to the analysis result C to the computer device 7. This allows an alert to be issued to the administrator or other relevant personnel.

[0066] The suspicious behavior analysis program executed by the behavior analysis device 5 of this embodiment is provided as a file in an installable or executable format, recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0067] Furthermore, the suspicious behavior analysis program executed by the behavior analysis device 5 of this embodiment may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Alternatively, the suspicious behavior analysis program executed by the behavior analysis device 5 of this embodiment may be provided or distributed via a network such as the Internet.

[0068] Alternatively, the suspicious behavior analysis program executed by the behavior analysis device 5 of this embodiment may be pre-installed and provided in a ROM or the like.

[0069] The suspicious behavior analysis program executed by the behavior analysis device 5 of this embodiment has a modular configuration that includes the above-mentioned parts (location information management unit 51, authentication unit 52, detection unit 53, identification unit 54, evaluation unit 55, storage unit 56, and communication unit 57). In actual hardware, the CPU (processor) reads the suspicious behavior analysis program from the storage medium and executes it, loading the above-mentioned parts onto the main memory, and generating the location information management unit 51, authentication unit 52, detection unit 53, identification unit 54, evaluation unit 55, storage unit 56, and communication unit 57 on the main memory.

[0070] Next, the suspicious behavior analysis process performed by the behavior analysis device 5 of the information processing system according to this embodiment will be described.

[0071] Figure 10 is a flowchart illustrating an example of the suspicious behavior analysis process according to the embodiment. Figure 11 is a diagram illustrating the authentication method of the monitored target M according to the embodiment, and Figures 12 to 14 are diagrams illustrating an example of the analysis result DB300 according to the embodiment.

[0072] The flowchart in Figure 10 will be explained below with reference to Figures 11-14.

[0073] The authentication unit 52 identifies the monitored object M based on the detection information. Specifically, for example, the authentication unit 52 identifies the monitored object M using various recognition methods such as facial recognition using the facial image F of the monitored object M as shown in Figure 11(a), image recognition using QR codes (registered trademarks) C1 and C2 attached to the monitored object M as shown in Figure 11(b), and character recognition using a tag T on which a person's name is written as shown in Figure 11(c) (step S11).

[0074] The identification unit 54 analyzes the detection information to determine whether or not the monitored object M exists (step S12). Specifically, for example, the identification unit 54 currently determines whether or not the monitored object M is included in the captured image from the imaging information A.

[0075] If it is determined in step S12 that the monitored object M is included in the captured image, the process returns to step S11.

[0076] If, in step S12, it is determined that the monitored object M is not included in the captured image, the identification unit 54 identifies that the monitored object M is in the blind spot area DR (step S13).

[0077] The identification unit 54 identifies the blind spot area DR where the monitored object M is located, and the time when the monitored object M entered or left the blind spot area DR, based on the detection information and the blind spot information (step S14).

[0078] Figure 12 shows an example of the analysis results DB300a. As shown in Figure 12, for example, if it is determined that a monitored subject M with person ID "C" is not visible on camera at a predetermined time, the identification unit 54 registers an analysis result C in the analysis results DB300a, associating "C" with the determination result "No", the blind spot ID "aaa" of the blind spot area DR where the monitored subject M is predicted to be located, and the time "2021 / 11 / 13 / 15:14:01" when "C" entered the blind spot area. On the other hand, for example, if it is determined that a monitored subject M with person ID "A" is visible on camera at a predetermined time, the identification unit 54 associates "A" with the determination result "Yes" and registers it in the analysis results DB300a.

[0079] Based on the analysis result C, the evaluation unit 55 determines whether multiple monitoring targets M are in the same blind spot area DR at the same time (step S15).

[0080] If, in step S15, it is not determined that multiple monitored targets M are in the same blind spot area DR at the same time, the process returns to step S12.

[0081] In step S15, if it is determined that multiple monitored objects M are in the same blind spot area DR at the same time, the evaluation unit 55 determines that there is a possibility of suspicious behavior by the monitored objects M. Furthermore, based on the analysis result C, the evaluation unit 55 determines whether the multiple monitored objects M and the monitoring area DR that have entered the blind spot area DR satisfy conditions (1) to (4).

[0082] Specifically, for example, the evaluation unit 55 determines whether (1) multiple monitoring targets M are present together in the same blind spot area DR for a predetermined period of time, (2) a specific combination of monitoring targets M are present together in the same blind spot area DR at the same time, (3) multiple monitoring targets M are present in a specific blind spot area DR at the same time, or (4) the number of monitoring targets M present in the same blind spot area DR at the same time satisfies a predetermined condition (step S16).

[0083] In step S16, if it is determined that multiple monitoring targets M and monitoring areas DR that have entered the same blind spot area DR do not satisfy any of the conditions (1) to (4), the evaluation unit 55 counts one instance as evaluation result E for that blind spot area DR (step S17).

[0084] Figure 13 shows an example of the analysis results DB300b in which such evaluation result E and analysis result C are registered. As shown in Figure 13, for example, if "G" and "H" were both present as monitored targets M in the blind spot area ID "ccc" from "2021 / 11 / 13 / 14:52:04" to "2021 / 11 / 13 / 14:52:15", the evaluation unit 55 determines that there is a possibility of suspicious behavior between "G" and "H" in "ccc" and counts it as 1 for "ccc". The evaluation unit 55 also registers "34" as a precedent, which is the number of times it has been determined that there is a possibility of suspicious behavior for the combination of "G" and "H" in the blind spot area ID "ccc", in the analysis results DB300b by associating it with the pattern ID "6496fjkr3649" which represents the combination of "G" and "H", and the blind spot area ID "ccc".

[0085] In step S16, if the evaluation unit 55 determines that multiple monitoring targets M that have entered the same blind spot area DR satisfy the conditions (1) to (4) described above, it counts the number of times that the blind spot area DR has been affected by a value other than "1", i.e., a number other than "1" (step S18).

[0086] Figure 14 shows an example of the analysis results DB300b in which such evaluation result E and analysis result C are registered. As shown in Figure 14, for example, if "G" and "H" as monitored targets M are both present in the blind spot area ID "ccc" for 11 seconds from "2021 / 11 / 13 / 14:52:04" to "2021 / 11 / 13 / 14:52:15", the evaluation unit 55 determines that a predetermined time, for example 10 seconds, has been exceeded and counts 1.8 times for the blind spot area ID "ccc". The evaluation unit 55 then registers "35.8" as evaluation result E, which is a count of "1.8" up from the previous example's "34", in the analysis results DB300b as a risk of harassment, associating it with "ccc".

[0087] The communication unit 57 outputs an evaluation result E corresponding to the analysis result C described above, and the suspicious behavior analysis process ends (step S19).

[0088] As described above, the information processing system of this embodiment is an information processing system having a plurality of imaging devices 1 installed in a monitoring area R where one or more monitored targets M exist, an action analysis device 5 that analyzes the behavior of monitored targets M based on imaging information A which includes at least time information acquired by the imaging devices 1, and a computer device 7 connected to the action analysis device 5 via a network, wherein each of the plurality of imaging devices 1 images the monitoring area R, and the action analysis device 5 includes a storage unit 504, detection information which indicates the presence or absence of monitored targets M in a time series based on the imaging information A acquired from the imaging devices 1, and blind spot information which includes location information of a blind spot area DR where it is impossible to image monitored targets M by the imaging devices 1, and based on this, the monitoring The system includes an identification unit 54 that identifies the blind spot area DR where the elephant M is located, a storage unit 56 that stores an analysis result C in the analysis result DB 300, which links the time information of the detection information, the monitored target M, and the identified blind spot area DR, an evaluation unit 55 that, based on the analysis result C, determines that there is a possibility of suspicious behavior by the monitored target M and performs an evaluation that raises the evaluation level indicating the high probability of suspicious behavior in the blind spot area DR, if it is determined that multiple monitored targets M are present in the same blind spot area DR at the same time, and a communication unit 57 that transmits an evaluation result E corresponding to the analysis result C to the computer device 7. The computer device 7 receives and outputs the analysis result C and the evaluation result E corresponding to the analysis result C from the behavior analysis device 5.

[0089] Furthermore, the behavioral analysis device 5 of the embodiment includes a storage unit 504, an identification unit 54 that detects the presence or absence of a monitored target M in a time series based on imaging information A which includes at least time information acquired from a plurality of imaging devices 1 installed in a monitoring area R where one or more monitored targets M exist, and identifies a blind spot area DR where a monitored target M exists based on detection information indicating the detection result and blind spot information which includes location information of a blind spot area DR where the monitored target M cannot be imaged by the imaging device 1, a storage unit 56 that stores an analysis result C which links the time information of the detection information, the monitored target M, and the identified blind spot area DR in an analysis result DB 300, an evaluation unit 55 that, based on the analysis result C, determines that there is a possibility of suspicious behavior by the monitored target M and performs an evaluation that raises the evaluation level indicating the high probability of suspicious behavior for the blind spot area DR, if it is determined that multiple monitored targets M exist in the same blind spot area DR at the same time, and a communication unit 57 that outputs an evaluation result E corresponding to the analysis result C.

[0090] Therefore, in the information processing system and behavioral analysis device 5 according to this embodiment, the same imaging device 1 can be used to detect suspicious behavior in blind spots and to detect the location information of blind spots. There is no need to provide terminals or other sensors in addition to the imaging device, such as having the monitored subject M possess a location sensor, and the blind spot area DR where the monitored subject M is located can be identified. Accordingly, the information processing system and behavioral analysis device 5 of this embodiment can detect suspicious behavior with a simple configuration.

[0091] Furthermore, in the information processing system and behavioral analysis device 5 according to this embodiment, since the blind spot area DR where the monitored target M is located can be identified based on the imaging information A acquired by the imaging device 1, it is possible to identify when multiple monitored targets M are in the same blind spot area DR at the same time, and in such cases, a predetermined evaluation can be performed as there is a possibility of suspicious behavior. As a result, for example, even if a suspicious person with the intention of harassment deliberately takes inconspicuous actions within the imaging area of ​​the imaging device 1, the possibility of suspicious behavior occurring can be detected with high accuracy.

[0092] As described above, the information processing system and behavioral analysis device 5 according to this embodiment make it possible to detect suspicious human behavior with a simple configuration and high accuracy.

[0093] Furthermore, the behavioral analysis device 5 of the embodiment includes a location information management unit 51 that acquires map information of the monitoring area R and location information of a plurality of imaging devices 1 installed in the monitoring area R, identifies a blind spot area DR based on the map information and the location information of the plurality of imaging devices 1, and stores the identified blind spot area DR as blind spot information in the location information DB 100. The identification unit 54 identifies the blind spot area DR where the monitored target M is located based on the detection information and the blind spot information.

[0094] As a result, in the information processing system and behavioral analysis device 5 of this embodiment, even if the same location is recognized as separate blind spot areas (DRs) by the imaging device 1, the duplicated information is eliminated and it is recognized as a single blind spot area (DR). This allows for the accurate identification of the blind spot area (DR) where the monitored target M is located, enabling high-precision detection of the possibility of suspicious behavior occurring.

[0095] Furthermore, the location information management unit 51 of the embodiment identifies a location where a change in the presence or absence of the monitored object M occurs as a blind spot line DL based on the detection information, and stores the location information of the identified blind spot line DL and the blind spot area DR in the location information DB 100 in association with each other. The identification unit 54, when it detects the presence or absence of the monitored object M based on the detection information, identifies the blind spot line DL where the change in presence or absence occurred, and identifies that the monitored object M is present in the blind spot area DR associated with the blind spot line DL.

[0096] As a result, the presence or absence of the monitored object M can be accurately detected based on the imaging information A acquired from the imaging device 1, making it possible to accurately identify the blind spot area DR where the monitored object M is located.

[0097] Furthermore, in the embodiment, if the identification unit 54 fails to identify the blind spot area DR where the monitored object M is located, the location information management unit 51 detects a new blind spot line DL based on the detection information, and stores the location information of the detected blind spot line DL in association with the blind spot area DR in the storage unit 504.

[0098] This allows the blind spot information to be automatically updated based on the imaging information A, even if a new blind spot area DR is created in the monitoring area R due to rearrangement or other reasons. This makes it possible to accurately identify the blind spot area DR where the monitored target M is located.

[0099] Furthermore, the behavioral analysis device 5 of the embodiment further includes an authentication unit 52 that identifies a target M based on detection information, and the identification unit 54 identifies a blind spot area DR where a specific target M exists.

[0100] This allows for the extraction of specific surveillance targets M that are likely to be the target of suspicious behavior, and the identification of the location where these surveillance targets M are located, thereby enabling the detection of the possibility of suspicious behavior with even greater accuracy.

[0101] Furthermore, the authentication unit 52 of the embodiment acquires information regarding the behavioral patterns of the identified monitored object M to determine the state of the monitored object M, and the identification unit 54 identifies the blind spot region DR in which the monitored object M having a predetermined state exists.

[0102] This allows for, for example, the extraction of only those individuals M who are highly likely to exhibit suspicious behavior, and the identification of their locations, thereby enabling the detection of suspicious behavior with even greater accuracy.

[0103] Furthermore, the evaluation level in this embodiment is the number of times suspicious behavior is detected. Based on the analysis result C, the evaluation unit 55 determines that there is a possibility of suspicious behavior by the monitored object M if multiple monitored objects M are present in the same blind spot area DR at the same time, and increments the count as an evaluation level for the blind spot area DR.

[0104] In this way, the likelihood of suspicious behavior is evaluated as a number of occurrences, allowing administrators and others to grasp the possibility of suspicious behavior at a glance.

[0105] Furthermore, the suspicious behavior in this embodiment includes acts of harassment.

[0106] This makes it possible to suppress the possibility of harassment occurring outside the target area of ​​the imaging device 1.

[0107] Furthermore, the evaluation unit 55 of the embodiment counts up the number of times in each of the following cases based on the analysis result C: when it is determined that multiple monitoring targets M are present in the same blind spot area DR in excess of a predetermined condition; when it is determined that a specific monitoring target M is present in the same blind spot area DR at the same time; when it is determined that multiple monitoring targets M are present in a specific blind spot area DR at the same time; and when it is determined that the number of monitoring targets M present in the same blind spot area DR at the same time satisfies a predetermined condition.

[0108] In this way, for example, based on past cases, we can identify the conditions under which suspicious behavior is likely to occur, assign weights to each condition, and count the occurrences accordingly. This allows for the detection of the possibility of suspicious behavior with high accuracy.

[0109] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and any modifications, improvements, etc., that can achieve the objectives of the present invention are included in the present invention.

[0110] In the above-described embodiment, it was explained that the imaging information A acquired by the imaging device 1 is transmitted from the imaging device 1 to the behavior analysis device 5, but this is not limited to this. That is, the imaging information A may be transmitted to a predetermined receiving device installed in the monitoring area R, and then transmitted from the receiving device to the behavior analysis device 5.

[0111] Furthermore, in the above-described embodiment, harassment is just one example of suspicious behavior. In other words, suspicious behavior includes behaviors other than harassment.

[0112] Furthermore, in the above-described embodiment, the evaluation level indicating the likelihood of suspicious behavior was explained as the number of occurrences, but the number of occurrences is just one example of an evaluation level. That is, for example, the likelihood of suspicious behavior may be evaluated using a ranking system that groups the number of occurrences into stages, for example, rank C for 1 to 3 times, rank B for 3 to 5 times, and rank D for 6 or more times.

[0113] Furthermore, in the above-described embodiment, the same time includes not only the exact same time, but also a time that falls within a certain range of time, such as one minute. [Explanation of Symbols]

[0114] 1. Imaging device 5 Behavior analysis device 7. Computer equipment 51 Location information management department 52 Certification Department 53 Detection unit 54 Specific part 55 Evaluation Department 56 Preservation Department 57 Communications Department [Prior art documents] [Patent Documents]

[0115] [Patent Document 1] Japanese Patent Publication No. 2011-215829

Claims

1. An information processing system comprising: a plurality of imaging devices installed in a monitoring area where one or more objects to be monitored exist; a behavior analysis device that analyzes the behavior of the objects to be monitored based on imaging information, which includes at least time information, acquired by the imaging devices; and a computer device connected to the behavior analysis device via a network, Each of the plurality of imaging devices captures the monitoring area, The aforementioned behavioral analysis device is Memory unit and, A unit that detects the presence or absence of the monitored target in a time series based on imaging information acquired from the imaging device, and identifies the blind spot area where the monitored target exists based on detection information indicating the detection result and blind spot information including location information of a blind spot area where the monitored target cannot be imaged by the imaging device, A storage unit stores analysis result information in the storage unit, which links the time information of the detection information, the monitored target, and the identified blind spot area. Based on the analysis results information, if it is determined that multiple monitored targets exist in the same blind spot area at the same time, the evaluation unit determines that there is a possibility of suspicious behavior by the monitored targets and performs an evaluation that increases the evaluation level indicating the high probability of such suspicious behavior in the blind spot area. A communication unit that transmits evaluation results corresponding to the analysis result information to the computer device, Equipped with, The aforementioned computer device, The behavioral analysis device receives and outputs the analysis result information and the evaluation result associated with the analysis result information. Information processing system.

2. Memory unit and, A unit that detects the presence or absence of a monitored object in a time series based on imaging information which includes at least time information obtained from multiple imaging devices installed in a monitoring area where one or more monitored objects exist, and identifies the blind spot area where the monitored object exists based on detection information which includes location information of a blind spot area where the monitored object cannot be imaged by the imaging device, A storage unit stores analysis result information in the storage unit, which links the time information of the detection information, the monitored target, and the identified blind spot area. Based on the analysis results information, if it is determined that multiple monitored targets exist in the same blind spot area at the same time, the evaluation unit determines that there is a possibility of suspicious behavior by the monitored targets and performs an evaluation that increases the evaluation level indicating the high probability of such suspicious behavior in the blind spot area. A communication unit that outputs evaluation results corresponding to the aforementioned analysis result information, Equipped with, Behavior analysis device.

3. A location information management unit acquires map information of the monitoring area and location information of the multiple imaging devices installed in the monitoring area, identifies the blind spot area based on the map information and the location information of the multiple imaging devices, and stores the identified blind spot area as blind spot information in the storage unit. Furthermore, The specified part is, Based on the detection information and the blind spot information, the blind spot region in which the monitored object exists is identified. The behavioral analysis device according to claim 2.

4. The aforementioned location information management unit, Based on the detection information, the location where the change in the presence or absence of the monitored target occurs is identified as a blind spot line, and the location information of the identified blind spot line is stored in the storage unit in association with the blind spot area. When the identification unit detects the presence or absence of the monitored target based on the detection information, it identifies the blind spot line where the change in presence or absence occurred, and identifies that the monitored target exists in the blind spot region associated with the blind spot line. The behavioral analysis device according to claim 3.

5. The aforementioned location information management unit, If the blind spot area where the monitored object is located is not identified in the identification unit, a new blind spot line is detected based on the detection information, and the location information of the detected blind spot line is stored in the storage unit in association with the blind spot area. The behavioral analysis device according to claim 4.

6. An authentication unit that identifies the target to be monitored based on the detection information, Furthermore, The specified part is, A specific target of monitoring exists and the blind spot area is identified. The behavioral analysis device according to claim 2.

7. The authentication unit, Obtain information regarding the behavioral patterns of the identified monitored subject to determine the state of the monitored subject. The specified part is, The monitoring target having a predetermined state exists and the blind spot area is identified. The behavioral analysis device according to claim 6.

8. The aforementioned evaluation level is the number of times the behavior was judged to be suspicious. Based on the analysis results, the evaluation unit determines that if multiple monitored targets exist in the same blind spot area at the same time, it determines that there is a possibility of suspicious behavior by the monitored targets and increments the count as the evaluation level for the blind spot area. The behavioral analysis device according to claim 2.

9. The aforementioned suspicious behavior includes, The behavioral analysis device according to claim 2.

10. Based on the analysis result information, the evaluation unit counts up the number of times in each of the following cases: when it is determined that multiple monitoring targets exist in the same blind spot area in excess of a predetermined condition; when it is determined that a specific monitoring target exists in the same blind spot area at the same time; when it is determined that multiple monitoring targets exist in a specific blind spot area at the same time; and when it is determined that the number of monitoring targets present in the same blind spot area at the same time satisfies a predetermined condition. The behavioral analysis device according to claim 8.

11. A behavior analysis method performed on an information processing system comprising: a plurality of imaging devices installed in a monitoring area where one or more objects to be monitored exist; a behavior analysis device that analyzes the behavior of the objects to be monitored based on imaging information, which includes at least time information, acquired by the imaging devices; and a computer device connected to the behavior analysis device via a network, wherein the method is performed on an information processing system. The behavioral analysis device includes a memory unit, Each of the plurality of imaging devices captures the monitoring area, The behavioral analysis device detects the presence or absence of the monitored object in a time series based on imaging information which includes at least the time information obtained from the plurality of imaging devices installed in the monitoring area where one or more monitored objects exist, and identifies the blind spot area where the monitored object exists based on detection information which includes location information of a blind spot area where the monitored object cannot be imaged by the imaging device, The behavioral analysis device stores analysis result information in the storage unit, which links the time information of the detection information, the monitored target, and the identified blind spot area. Based on the analysis results, if it is determined that multiple monitored targets exist in the same blind spot area at the same time, an evaluation step is made to determine that there is a possibility of suspicious behavior by the monitored targets and to raise the evaluation level indicating the high probability of such suspicious behavior in the blind spot area. The behavioral analysis device transmits the evaluation results corresponding to the analysis result information to the computer device. The computer device receives and outputs the analysis result information and the evaluation result associated with the analysis result information from the behavioral analysis device. Behavioral analysis methods including

12. A behavioral analysis program to be executed by the computer of a behavioral analysis device, The behavioral analysis device includes a memory unit, Steps include: detecting the presence or absence of a monitored object in a time series based on imaging information that includes at least time information acquired from multiple imaging devices installed in a monitoring area where one or more monitored objects exist; identifying the blind area where the monitored object exists based on detection information indicating the detection result and blind spot information that includes location information of a blind spot area where the monitored object cannot be imaged by the imaging device; The steps include storing analysis result information in the storage unit, which links the time information of the detection information, the monitored target, and the identified blind spot area; Based on the analysis results, if it is determined that multiple monitored targets exist in the same blind spot area at the same time, an evaluation step is made to determine that there is a possibility of suspicious behavior by the monitored targets and to raise the evaluation level indicating the high probability of such suspicious behavior in the blind spot area. The steps include: transmitting evaluation results corresponding to the aforementioned analysis result information; A behavioral analysis program to cause the aforementioned computer to execute.

Images