Information processing system, information processing device, information processing method, and program

Abstract

An information processing system (100) comprises an analysis result acquisition unit (131), a candidate detection unit (132), and a missing child detection unit (134). The analysis result acquisition unit (131) acquires an analysis result of video captured by a plurality of video capturing devices (101). The candidate detection unit (132) uses a person attribute included in the analysis result and a candidate condition to detect missing child candidates from among persons captured in the video. The missing child detection unit (134) detects, when the missing child candidate has an accompanying person at a first time point, on the basis of a result of comparison between the accompanying person of the missing child candidate between this first time point and an accompanying person at a second time point earlier than this first time point, a missing child from among the missing child candidates.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing apparatus, an information processing method, and program and.

Background Art

[0002] For example, Patent Document 1 discloses a technique for detecting a lost child.

[0003] The lost child identification unit described in Patent Document 1 extracts only persons of an age who are particularly likely to become lost children based on person information.

[0004] This person information is the result of performing feature extraction such as contours by a feature extraction unit, a person extraction unit, and an individual feature analysis unit from an image of a surveillance camera installed at a certain location, automatically grasping persons, and performing individual feature analysis of the age, clothing, build, etc. of each person.

[0005] The lost child identification unit described in Patent Document 1 identifies a person as a lost child when it is determined that there is a possibility of a lost child based on information such as a worried expression or behavior, whether the person is acting alone, etc., which is the result of the behavior analysis of a person performed by the behavior analysis unit in parallel.

[0006] Note that Patent Document 2 describes a technique for calculating the feature amount of each of a plurality of key points of a human body included in an image, searching for an image including a human body with a similar posture or a human body with a similar movement based on the calculated feature amount, or classifying together those with similar postures or movements.

[0007] Non-Patent Document 1 describes a technique related to human skeleton estimation.

Prior Art Documents

Patent Documents

[0008]

Patent Document 1

Patent Document 2

[0009] [Non-Patent Document 1] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, P. 7291-7299 [Overview of the project] [Problems that the invention aims to solve]

[0010] However, the technology described in Patent Document 1 detects lost children based on information such as anxious facial expressions and behavior, and whether they are acting alone. Therefore, it is difficult to accurately detect lost children who have been taken away by a complete stranger.

[0011] For example, it is generally difficult to detect a person's facial expressions and actions in an image with good accuracy. Even if it were possible, poor image quality could prevent accurate detection of a person's facial expressions and actions. With such low accuracy in detecting anxious facial expressions and actions, the technology described in Patent Document 1 may not be able to accurately detect a lost child who has been abducted.

[0012] Furthermore, for example, a person of an age where they could easily get lost may exhibit anxious facial expressions or anxious behavior even when accompanied by a guardian. In such cases, the technology described in Patent Document 1 may detect the person as lost even when accompanied by a guardian.

[0013] Furthermore, for example, a child who has been abducted is highly likely to be acting with an unfamiliar third party and not acting alone. Therefore, it is difficult to detect a child who has been abducted using the technique described in Patent Document 1, which relies on the fact that the child is acting alone.

[0014] Abduction is highly likely to be a dangerous situation for the lost child, and detecting it is extremely important for the child's safety.

[0015] Furthermore, Patent Document 2 and Non-Patent Document 1 do not disclose any technology for detecting lost children.

[0016] One example of the object of the present invention is to provide an information processing system, information processing device, information processing method, and recording medium that solve the problem of ensuring the safety of lost children, in view of the above-mentioned problems. [Means for solving the problem]

[0017] According to one aspect of the present invention, An analysis result acquisition means for acquiring the analysis results of images captured by multiple shooting means, A candidate detection means for detecting a lost child candidate from the person shown in the video, using the person attributes and candidate conditions included in the analysis results, The system includes a lost child detection means that, if there is an accompanying person for the lost child at a first time point, detects the lost child from among the lost child candidates based on a comparison of the accompanying person at the first time point with a second time point prior to the first time point. An information processing system is provided.

[0018] According to one aspect of the present invention, One or more computers, We obtained the analysis results of videos captured using multiple shooting methods. Using the person attributes and candidate conditions included in the analysis results, a lost child candidate is detected from the person shown in the video. When there is a companion of the missing child candidate at the first time point, based on the result of comparing the companions of the missing child candidate at the first time point and the second time point before the first time point, a missing child is detected from the missing child candidates. An information processing method is provided.

[0019] According to one aspect of the present invention, one or more computers acquire analysis results of images captured by a plurality of imaging means, detect missing child candidates from the people shown in the images using the person attributes and candidate conditions included in the analysis results, when there is a companion of the missing child candidate at the first time point, detect a missing child from the missing child candidates based on the result of comparing the companions of the missing child candidate at the first time point and the second time point before the first time point A recording medium on which a program for executing the above is recorded is provided.

Effect of the Invention

[0020] According to one aspect of the present invention, it becomes possible to ensure the safety of a missing child.

Brief Description of the Drawings

[0021] [Figure 1] It is a diagram showing an overview of the information processing system according to Embodiment 1. [Figure 2] It is a diagram showing an overview of the information processing apparatus according to Embodiment 1. [Figure 3] It is a flowchart showing an overview of the information processing according to Embodiment 1. [Figure 4] It is a diagram showing a configuration example of the information processing system. [Figure 5] It is a diagram showing a functional configuration example of the information processing apparatus according to Embodiment 1. [Figure 6] It is a diagram showing a functional configuration example of the missing child detection unit according to Embodiment 1. [Figure 7] It is a diagram showing a functional configuration example of the terminal according to Embodiment 1. [Figure 8]This figure shows an example of the physical configuration of the imaging device according to Embodiment 1. [Figure 9] This figure shows an example of the physical configuration of the analytical apparatus according to Embodiment 1. [Figure 10] This flowchart shows an example of the imaging process according to Embodiment 1. [Figure 11] This figure shows an example of a floor map of the target area. [Figure 12] This figure shows an example of frame information. [Figure 13] This flowchart shows an example of the analysis process according to Embodiment 1. [Figure 14] This is a flowchart showing an example of the lost child detection process according to Embodiment 1. [Figure 15] This flowchart shows an example of the detection process according to Embodiment 1. [Figure 16] This is a diagram illustrating the comparison process according to Embodiment 1. [Figure 17] This flowchart shows an example of the display process according to Embodiment 1. [Figure 18] This figure shows an example of the functional configuration of the information processing device according to Embodiment 2. [Figure 19] This flowchart shows an example of a lost child detection process according to Embodiment 2. [Figure 20] This figure shows an example of the functional configuration of the information processing device according to Embodiment 3. [Figure 21] This figure shows an example of the functional configuration of the lost child detection unit according to Embodiment 3. [Figure 22] This is a flowchart showing an example of the lost child detection process according to Embodiment 3. [Figure 23] This flowchart shows an example of the detection process according to Embodiment 3. [Figure 24] This is a flowchart showing an example of the comparison process according to Embodiment 3. [Figure 25] This is a flowchart showing an example of the comparison process according to Embodiment 3. [Modes for carrying out the invention]

[0022] Hereinafter, one embodiment of the present invention will be described with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0023] <Embodiment 1> Figure 1 is a diagram showing an overview of the information processing system 100 according to Embodiment 1. The information processing system 100 includes an analysis result acquisition unit 131, a candidate detection unit 132, and a lost child detection unit 134.

[0024] The analysis result acquisition unit 131 acquires the analysis results of images captured by multiple imaging devices 101.

[0025] The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results to detect potential lost persons from the people shown in the video.

[0026] The lost child detection unit 134 detects a lost child from among the potential lost children based on the results of comparing the companions of the potential lost child at the first time point with those at a second time point prior to the first time point, if there are companions of the potential lost child at the first time point.

[0027] This information processing system 100 makes it possible to ensure the safety of lost children.

[0028] Figure 2 is a diagram showing an overview of the information processing device 103 according to Embodiment 1.

[0029] The information processing device 103 includes an analysis result acquisition unit 131, a candidate detection unit 132, and a lost child detection unit 134.

[0030] The analysis result acquisition unit 131 acquires the analysis results of images captured by multiple imaging devices 101.

[0031] The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results to detect potential lost persons from the people shown in the video.

[0032] The lost child detection unit 134 detects a lost child from among the potential lost children based on the results of comparing the companions of the potential lost child at the first time point with those at a second time point prior to the first time point, if there are companions of the potential lost child at the first time point.

[0033] This information processing device 103 makes it possible to ensure the safety of lost children.

[0034] Figure 3 is a flowchart showing an overview of the information processing according to Embodiment 1.

[0035] The analysis result acquisition unit 131 acquires the analysis results of the images captured by the multiple imaging devices 101 (step S301).

[0036] The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results to detect potential lost persons from the people shown in the video (step S302).

[0037] If a person accompanying a potential lost child is present at the first time point, the lost child detection unit 134 detects the lost child from among the potential lost children based on the results of comparing the person accompanying the potential lost child at the first time point with a second time point prior to the first time point (step S304).

[0038] This information processing makes it possible to ensure the safety of lost children.

[0039] The following describes a detailed example of the information processing system 100 according to Embodiment 1.

[0040] (detail) (Example configuration of information processing system 100) Figure 4 shows an example configuration of the information processing system 100.

[0041] The information processing system 100 is a system for detecting a lost child who has been abducted. A lost child who has been abducted is someone who has been taken away by a third party. This third party is, for example, someone other than the lost child's guardian. The lost child is not limited to children; it may also be, for example, an elderly person.

[0042] In this embodiment, the area where the information processing system 100 detects lost children is a shopping mall. The target area can be predetermined as appropriate, and may be, for example, various facilities or landmarks, part or all of a building, or a designated area on a public road.

[0043] The information processing system 100 comprises first to M imaging devices 101_1 to 101_M1, an analysis device 102, an information processing device 103, and first to N terminals 104_1 to 104_M2.

[0044] M1 is an integer greater than or equal to 2. M2 is an integer greater than or equal to 1. Note that M1 may also be 1.

[0045] Each of the first to M imaging devices 101_1 to 101_M1 may be configured similarly. Therefore, in the following, any one of the first to M imaging devices 101_1 to 101_M1 will also be referred to as "imaging device 101".

[0046] Furthermore, each of terminals 104_1 to 104_M2 may be configured similarly. Therefore, in the following, any one of terminals 104_1 to 104_M2 will also be referred to as "terminal 104".

[0047] Each of the multiple imaging devices 101, the analysis device 102, the information processing device 103, and one or more terminals 104 are connected to each other via a communication network, and can send and receive information from each other via the communication network.

[0048] (Example of functional configuration of imaging device 101) The imaging device 101 captures a predetermined imaging area and generates an image. The image consists of, for example, a series of frame images showing the imaging area. The imaging device 101 transmits the image to the analysis device 102. The imaging area is part or all of the target area.

[0049] The imaging area is predetermined for each of the imaging devices 101_1 to 101_M1, from the first to the Mth. Therefore, there are multiple imaging areas in the information processing system 100.

[0050] Multiple shooting areas can be different areas within the target area. For example, multiple shooting areas are areas that do not overlap with each other. Note that multiple shooting areas are... photograph A portion or all of the region is other photograph It may be a part of the region or a region that overlaps with the entire region. If the entire regions of the imaging area overlap each other, these imaging areas may be imaged by imaging devices 101 with different imaging performance, such as resolution and lens performance.

[0051] (Example of functional configuration of analytical device 102) The analysis device 102 analyzes the images captured by the multiple imaging devices 101 and generates analysis results. The analysis device 102 transmits the generated analysis results to the information processing device 103.

[0052] The analysis results will include at least the personal attributes of the people in the video. Personal attributes are the attributes of a person. Personal attributes may include one or more of the following, for example, age (including age group), clothing, location, direction of movement, speed of movement, height, and gender. Note that personal attributes are not limited to those exemplified here, and detailed examples of personal attributes will be described later.

[0053] (Example of the functional configuration of the information processing device 103) The information processing device 103 uses the analysis results from the analysis device 102 to detect the missing child who has been abducted.

[0054] Figure 5 shows an example of the functional configuration of the information processing device 103 according to Embodiment 1. The information processing device 103 includes an analysis result acquisition unit 131, a candidate detection unit 132, a grouping unit 133, a lost item detection unit 134, a display control unit 135, a display unit 136, and a notification unit 137.

[0055] The analysis result acquisition unit 131 acquires the analysis results of the video captured by the multiple imaging devices 101 from the analysis device 102. The analysis result acquisition unit 131 may also acquire the frame images and / or video that were used to generate the analysis results from the analysis device 102, along with the analysis results.

[0056] Here, "A and / or B" means both A and B, or either A or B, and the same applies below.

[0057] The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results acquired by the analysis result acquisition unit 131 to detect potential lost persons from the people shown in the video.

[0058] Candidate criteria are conditions related to potential lost children, and can be pre-set by the user. Ideally, candidate criteria should include attributes of individuals who are likely to get lost. Specifically, candidate criteria may include one or more age-related conditions, such as under 10 years old or over 80 years old.

[0059] The grouping unit 133 uses the person attributes included in the analysis results acquired by the analysis result acquisition unit 131 and predetermined grouping conditions to identify the group to which the person in the video belongs.

[0060] The grouping criteria are conditions used to categorize the people appearing in the video using the person attributes included in the analysis results.

[0061] More specifically, the grouping conditions include, for example, one or more of the following: the persons are within a predetermined distance from each other; the difference in the persons' directions of movement is within a predetermined range; the difference in the persons' speeds of movement is within a predetermined range; and the persons are conversing.

[0062] The lost child detection unit 134 detects a lost child from among the potential lost children based on a comparison of the potential lost child's companions at the first and second time points, if a potential lost child has a companion at the first time point. The lost child detection unit 134 then generates lost child information regarding the detected lost child.

[0063] The second point in time is a point in time prior to the first point in time.

[0064] Lost child information is information relating to a lost child. Lost child information includes, for example, one or more of the following: one or more of the lost child's personal attributes, an image of the lost child, the lost child's location at a first and second point in time, and frame images and videos containing the lost child at the first and second points in time.

[0065] Figure 6 shows an example of the functional configuration of the lost child detection unit 134 according to Embodiment 1. The lost child detection unit 134 includes a discrimination unit 134a, a risk level identification unit 134b, a lost child identification unit 134c, and a lost child information generation unit 134d.

[0066] The discrimination unit 134a determines whether or not there is a companion of the potential lost child at the first time point.

[0067] The risk assessment unit 134b identifies the risk level corresponding to the location of the potential lost child at the first time point.

[0068] In detail, for example, the risk identification unit 134b identifies the risk level of the lost child at the first time point based on the location of the lost child candidate at the first time point and location-specific risk information.

[0069] Location-specific risk information is information that associates the attributes and risk levels of each location within the target area, and it is preferable to set this information in advance.

[0070] If the lost child identification unit 134c determines that a potential lost child has an accompanying person at the first time point, it detects the lost child from among the potential lost children based on the results of comparing the accompanying person at the first time point and the second time point.

[0071] The "results of comparing companions" mentioned above may, for example, be information indicating whether or not the companions have changed. That is, the lost child detection unit 134 may, for example, detect a lost child from among the lost child candidates based on whether or not the companions of the lost child candidate have changed between the first and second time points, if the companions of the lost child candidate are present at the first time point.

[0072] Furthermore, whether or not the companions have changed may be determined by whether or not all of the companions of the potential lost child have changed from those at the second time point (i.e., whether or not the potential lost child is accompanied only by different people than those at the second time point).

[0073] Generally, for example, a child may be accompanied by a guardian at the second time point, and then meet up with other guardians or acquaintances of the guardians at the first time point. By determining whether all of the companions of a potential lost child have changed since the second time point, it is possible to prevent misidentifying such a potential lost child as one who has been abducted. This allows for the detection of lost children who are highly likely to have been abducted, thereby ensuring the safety of lost children.

[0074] Furthermore, whether or not the accompanying persons have changed may be determined by whether at least some of the accompanying persons have changed from the second time point to the first time point. This allows for the detection of a potential lost child in the above situation as a child who has been abducted. Even in the above situation, there is still a possibility that the potential lost child has been abducted, so it becomes possible to ensure the safety of the lost child.

[0075] There may be various methods for determining whether or not a potential lost child has an accompanying person at the first point in time. In this embodiment, the method for determining whether or not a potential lost child has an accompanying person at the first point in time will be explained using an example in which a group identified by the grouping unit 133 is used.

[0076] In other words, the discrimination unit 134a according to this embodiment uses the group to which the lost child candidate belongs at the first time point to determine whether or not there is a companion of the lost child candidate at the first time point.

[0077] Furthermore, there may be various methods for comparing the companions at the first and second points in time. In this embodiment, we will explain using an example in which the lost child detection unit 134 detects a lost child from among the lost child candidates using the group identified by the grouping unit 133.

[0078] In other words, in this embodiment, if there is a companion of the lost child candidate at the first time point, the lost child detection unit 134 compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first time point and the second time point. Then, based on the results of this comparison, the lost child detection unit 134 detects the lost child from among the lost child candidates.

[0079] In detail, for example, if the lost child identification unit 134c determines that a person accompanying the lost child is present at the first time point, it compares individuals belonging to the same group as the lost child at both the first and second time points. Based on the results of this comparison, the lost child identification unit 134c detects the lost child from among the lost child candidates. Here, individuals belonging to the same group as the lost child candidate correspond to the person accompanying the child.

[0080] Furthermore, in this embodiment, we will explain using an example in which the risk level of a potential lost child at a first time point is referenced in order to detect a lost child from among the potential lost children.

[0081] In other words, the lost child detection unit 134 (more specifically, the lost child identification unit 134c) according to this embodiment detects a lost child from among the lost child candidates based on the results of the above comparison and the degree of danger corresponding to the location of the lost child candidate at the first time point, if there is an accompanying person of the lost child candidate at the first time point.

[0082] Furthermore, in order to detect a lost child from a list of potential lost children, the risk level of the potential lost child at the first point in time does not need to be referenced.

[0083] The lost child information generation unit 134d generates lost child information related to the lost child detected by the lost child identification unit 134c.

[0084] More specifically, the lost child information generation unit 134d may generate lost child information that includes some or all of the analysis results related to the lost child from the analysis results acquisition unit 131. The lost child information generation unit 134d may further generate lost child information that includes frame images and / or video. These frame images and / or video may show the lost child, or they may be the source from which the analysis results included in the lost child information were generated. The lost child information generation unit 134d may further generate lost child information that includes the risk level identified for the lost child included in the lost child information.

[0085] Refer to Figure 5 again. The display control unit 135 causes various information to be displayed on the display unit 136. The display unit 136 is a display composed of, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel, which will be described later.

[0086] The display control unit 135 may, for example, display the lost child information generated by the lost child detection unit 134 (more specifically, the lost child information generation unit 134d) on the display unit 136.

[0087] For example, the display control unit 135 may display on the display unit 136 an image or / or video superimposed on at least one of the frame image and video containing the lost child at the first time point, showing the lost child's position at the first time point. For example, the display control unit 135 may display on the display unit 136 an image or / or video superimposed on at least one of the frame image and video containing the lost child at the second time point, showing the lost child's position at the second time point.

[0088] For example, if the lost child detection unit 134 detects multiple lost children, the display control unit 135 may display the lost child information of the multiple lost children on the display unit 136 in order of the degree of danger at the first point in time.

[0089] Such a display control unit 135 and a display unit 136 are examples of display control means and display means, respectively.

[0090] The notification unit 137 transmits the lost child information generated by the lost child detection unit 134 (specifically, the lost child information generation unit 134d) to each of the one or more terminals 104.

[0091] (Example of the functional configuration of terminal 104) Terminal 104 is a device for displaying lost child information. Terminal 104 is carried by a predetermined person, such as an employee or security guard in the target area.

[0092] Figure 7 shows an example of the functional configuration of terminal 104 according to Embodiment 1. Terminal 104 comprises a lost child information acquisition unit 141, a display control unit 142, and a display unit 143.

[0093] The lost child information acquisition unit 141 acquires lost child information from the information processing device 103.

[0094] The display control unit 142 causes various information to be displayed on the display unit 143. The display unit 143 is a display composed of, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel, which will be described later.

[0095] The display control unit 142, for example, causes the lost child information acquired by the lost child information acquisition unit 141 to be displayed on the display unit 143.

[0096] Such display control unit 142 and display unit 143 are other examples of display control means and display means, respectively.

[0097] (Example of the physical configuration of information processing system 100) The information processing system 100 physically comprises, for example, the first to Mth imaging devices 101_1 to 101_M1, the analysis device 102, the information processing device 103, and the first to Nth terminals 104_1 to 104_M2.

[0098] Each of the first to M imaging devices 101_1 to 101_M1 may be physically configured in the same way. Each of the first to N terminals 104_1 to 104_M2 may be physically configured in the same way.

[0099] The physical configuration of the information processing system 100 is not limited to this. For example, the functions of the multiple imaging devices 101, analysis devices 102, and information processing devices 103 described in this embodiment may be physically provided in a single device, or they may be divided and provided in multiple devices in a manner different from this embodiment. The function of transmitting or receiving information between devices 101 to 104 in this embodiment via the network N may, when incorporated into a common physical device, transmit or acquire information via an internal bus or the like instead of the network N.

[0100] (Example of the physical configuration of the imaging device 101) Figure 8 shows an example of the physical configuration of the imaging device 101 according to Embodiment 1. The imaging device 101 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, a user interface 1060, and a camera 1070.

[0101] Bus 1010 is for processor 1020, memory 1030, and storage device 1040. , Ne Network Interface 10 5 0, User interface 1060, The camera 1070 and microphone 1080 are data transmission paths for sending and receiving data to and from each other. However, the method of connecting the processor 1020 and other components to each other is not limited to a bus connection.

[0102] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0103] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0104] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules for realizing each function of the imaging device 101. The processor 1020 reads these program modules into memory 1030 and executes them, thereby realizing each function corresponding to that program module.

[0105] The network interface 1050 is an interface for connecting the imaging device 101 to the network N.

[0106] The user interface 1060 includes touch panels, keyboards, mice, etc., as interfaces for the user to input information, and liquid crystal panels, organic EL (Electro-Luminescence) panels, etc., as interfaces for presenting information to the user.

[0107] Camera 1070 includes an image sensor, an optical system such as a lens, and captures images of the shooting area under the control of processor 1020.

[0108] Furthermore, the imaging device 101 may accept input from the user and present information to the user via an external device connected to the network N (for example, an analysis device 102, an information processing device 103, etc.). In this case, the imaging device 101 may use the user interface 10 6 It is not necessary to include 0.

[0109] (Example of physical configuration of analysis device 102, information processing device 103, and terminal 104) Figure 9 shows an example of the physical configuration of the analysis device 102 according to Embodiment 1. The analysis device 102 physically has, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, and a network interface 1050, similar to those of the imaging device 101. The analysis device 102 also physically has, for example, an input interface 2060 and an output interface 2070.

[0110] However, the storage device 1040 of the analyzer 102 stores program modules for realizing each function of the analyzer 102. Furthermore, the network interface 1050 of the analyzer 102 is an interface for connecting the analyzer 102 to network N.

[0111] The input interface 2060 is an interface for the user to input information, and includes, for example, a touch panel, keyboard, mouse, etc. The output interface 2070 is an interface for presenting information to the user, for example liquid This includes crystal panels, organic EL panels, etc.

[0112] The information processing device 103 and the terminal 104 according to Embodiment 1 may be physically configured in the same way as, for example, the analysis device 102. However, the storage device 1040 of the information processing device 103 and the terminal 104 stores program modules for realizing each of their respective functions. Furthermore, the network interface 1050 of the information processing device 103 and the terminal 104 is an interface for connecting each to a network N.

[0113] We have now described an example configuration of the information processing system 100 according to Embodiment 1. From here, we will describe an example of operation of the information processing system 100 according to Embodiment 1.

[0114] (Example of operation of information processing system 100) The information processing system 100 according to this embodiment performs information processing to detect a lost child who has been abducted. The information processing includes, for example, a photography process, an analysis process, a lost child detection process, and a display process.

[0115] (Example of the imaging process according to Embodiment 1) Figure 10 is a flowchart illustrating an example of the imaging process according to Embodiment 1. The imaging process is a process for imaging a target area. When the imaging device 101 receives a start command from the user via the network N from the information processing device 103, for example, it repeatedly executes the imaging process at a predetermined frame rate until it receives a stop command from the user. Note that the method for starting or ending the imaging process is not limited to these.

[0116] The frame rate can be determined as appropriate, for example, 1 / 30th of a second or 1 / 60th of a second.

[0117] The imaging device 101 captures the imaging area and generates a frame image showing the imaging area (step S101).

[0118] Figure 11 shows an example of a floor map of the target area. The target area shown in Figure 11 includes two floors, and Figure 11(a) shows a floor map of the first floor of the target area. Figure 11(b) shows a floor map of the second floor of the target area. In Figure 11, the areas enclosed by dotted circles represent each imaging area of ​​the imaging device 101. In the example in Figure 11, there are 18 imaging areas, so this is an example where M1 is 18, that is, an example where the information processing system 100 is equipped with 18 imaging devices 101.

[0119] Furthermore, a single imaging device 101 may be configured to capture multiple imaging areas.

[0120] Refer to Figure 10 again. The imaging device 101 generates frame information including the frame image generated in step S101 (step S102).

[0121] Figure 12 shows an example of frame information. Frame information is information that associates, for example, a frame image with a frame ID (Identification), a shooting ID, and a shooting date.

[0122] Frame ID is, frame Mu This is information for identification. The shooting ID is information for identifying the shooting device 101. The shooting time is information indicating the time of shooting. term This consists, for example, of the year, month, day, and time. The time may be expressed in predetermined increments such as 1 / 10th of a second or 1 / 100th of a second.

[0123] Figure 12 shows that frame image FP1 with frame ID "P1" was captured at time "T1" by imaging device 101 with imaging ID "CM1".

[0124] Note that the structure of frame information is not limited to this.

[0125] Refer to Figure 10 again. The imaging device 101 transmits the frame information generated in step S102 to the analysis device 102 (step S103), and terminates the imaging process.

[0126] Each of the imaging devices 101 repeatedly performs this imaging process to generate video footage of the target area and transmit it to the analysis device 102. The imaging process should preferably be performed in real time.

[0127] (Example of analysis processing according to Embodiment 1) Figure 13 is a flowchart illustrating an example of the analysis process according to Embodiment 1. The analysis process is for analyzing the video captured by the imaging device 101. When the analysis device 102 receives a start command from the user via the network N from the information processing device 103, for example, it repeatedly executes the analysis process until it receives a stop command from the user. Note that the method for starting or ending the analysis process is not limited to these.

[0128] The analysis device 102 acquires the frame information transmitted in step S103 from the imaging device 101 (step S201).

[0129] The analysis device 102 stores the frame information acquired in step S201 and analyzes the frame images contained in the frame information (step S202).

[0130] In this analysis, the analysis device 102 may appropriately refer to one or more of the following: frame images taken by other imaging devices 101 at the same time, past frame images, and / or analysis results.

[0131] Here, the other imaging device 101 is a different imaging device 101 from the imaging device 101 that generated the frame image to be analyzed. Furthermore, the past frame image and / or analysis result refers to the frame image and / or the analysis result of the frame image that was generated by each of the multiple imaging devices 101 prior to the frame image to be analyzed.

[0132] In detail, for example, the analysis device 102 has one or more analysis functions for analyzing video. minutes The analytical functions include one or more of the following: (1) object detection function, (2) face analysis function, (3) human figure analysis function, (4) posture analysis function, (5) behavior analysis function, (6) appearance attribute analysis function, (7) gradient feature analysis function, (8) color feature analysis function, and (9) movement path analysis function.

[0133] (1) The object detection function detects objects from the frame image. The object detection function can also determine the position of objects within the frame image. Techniques such as YOLO (You Only Look Once) can be applied to the object detection function. Here, "object" includes people and things, and the same applies below.

[0134] In other words, the object detection function detects people and objects within the shooting area of ​​the framed image, for example. It also determines the positions of people and objects.

[0135] (2) The face analysis function detects human faces from frame images, extracts the features of the detected faces, and classifies the detected faces. The face analysis function can also determine the position of a face within an image. The face analysis function can also determine the identity of people detected from different images based on the similarity between the facial features of people detected from different frame images.

[0136] (3) The human figure analysis function extracts human physical characteristics of people contained in frame images (for example, values ​​indicating overall characteristics such as body shape, height, and clothing), and classifies (categorizes) the people contained in frame images. The human figure analysis function can also identify the position of a person within an image. The human figure analysis function can also determine the identity of people contained in different images based on the human physical characteristics of people contained in those different images.

[0137] (4) The posture analysis function detects the joint points of a person from an image and creates a stick-figure model by connecting the joint points. The posture analysis function then uses the information from the stick-figure model to estimate the person's posture, extracts the estimated posture features (posture features), and classifies the people in the image. The posture analysis function can also determine the identity of people in different images based on the posture features of people in different images.

[0138] For example, the posture analysis function estimates postures such as standing, squatting, and crouching from an image, and extracts posture features that represent each posture.

[0139] For example, the techniques disclosed in Patent Document 2 and Non-Patent Document 1 can be applied to the posture analysis function.

[0140] (5) The behavioral analysis function can estimate human movement using information from stick figure models, changes in posture, etc., extract human movement features (motion features), and classify (categorize) people included in images. functionTherefore, information from stick figure models can be used to estimate a person's height or to determine their position within an image. (Behavioral analysis) function For example, it is possible to estimate actions such as changes or transitions in posture, movement (changes or transitions in position), movement speed, and direction of movement from an image, and to extract motion features of those actions.

[0141] (6) The appearance attribute analysis function can recognize appearance attributes associated with a person. The appearance attribute analysis function extracts features related to the recognized appearance attributes (appearance attribute features) and classifies (categorizes) people included in the image. Appearance attributes are attributes of appearance, and include one or more of the following: age (including age group), gender, clothing color, hairstyle, presence or absence of accessories, and, if accessories are worn, the color of those accessories. Clothing includes one or more items such as clothes and shoes. Accessories include one or more items such as hats, ties, glasses, necklaces, and rings.

[0142] (7) The gradient feature analysis function extracts gradient features (gradient features) from the frame image. Techniques such as SIFT, SURF, RIFF, ORB, BRISK, CARD, and HOG can be applied to the gradient feature detection function.

[0143] (8) The color feature analysis function can detect objects from frame images, extract color features from the detected objects, and classify the detected objects.

[0144] Color features include, for example, color histograms. Color feature analysis functions can, for example, detect people and objects contained in frame images. Furthermore, color feature analysis functions can classify items into predetermined classes.

[0145] (9) The movement path analysis function can determine the movement paths (trajectories of movement) of people included in the video by using, for example, the results of identity determination in any of the analysis functions (2) to (6) described above. Specifically, for example, by connecting people who have been determined to be the same across frame images that are different in time series, it is possible to determine the movement paths of those people. The movement path analysis function can also determine movement paths that span across multiple videos taken in different shooting areas.

[0146] Person attributes include, for example, at least one of the elements found in the person detection results from object detection functions, facial features, anthropometric features, posture features, motion features, appearance attribute features, gradient features, color features, movement patterns, movement speed, and movement direction.

[0147] Note that each of the analysis functions (1) to (9) may utilize the results of analyses performed by other analysis functions as appropriate.

[0148] The analysis device 102 analyzes video, including frame images, using one or more of these analysis functions and generates detection results that include person attributes. In the detection results, it is desirable that each person appearing in the frame image be associated with their person attributes.

[0149] The analyzer 102 generates analysis information by associating the analysis results from step S202 with the frame information acquired in step S201 (step S203).

[0150] The frame information obtained in step S201 includes the frame image that was used to generate the analysis results (i.e., the frame image that was the subject of analysis in step S202).

[0151] The analyzer 102 transmits the analysis information generated in step S203 to the information processing device 103 (step S204).

[0152] This analysis process is preferably performed repeatedly for each of the multiple frame images generated by each of the multiple imaging devices 101. This allows for the analysis of the video footage of the target area and the transmission of the analysis results generated by this analysis to the information processing device 103.

[0153] Furthermore, the analysis device 102 may analyze a portion of the time-series frame images generated by each of the multiple imaging devices 101, for example, by performing analysis processing on frame images at predetermined time intervals. This time interval may be, for example, 1 second. of It is preferable to set a time length that does not affect detection. This reduces the number of frame images that the analysis device 102 processes, while suppressing a decrease in the accuracy of detecting lost animals compared to analyzing all frame images in the time series. Therefore, it is possible to reduce the processing load on the analysis device 102 while suppressing a decrease in the accuracy of detecting lost animals.

[0154] Furthermore, the analytical methods performed by the analyzer 102 are not limited to those described herein and may be modified as appropriate. For example, the analytical functions provided by the analyzer 102 may be modified as appropriate.

[0155] (Example of lost child detection process according to Embodiment 1) Figure 14 is a flowchart showing an example of a lost child detection process according to Embodiment 1. The lost child detection process is a process for detecting a lost child who has been abducted, using the analysis results generated by executing the analysis process.

[0156] For example, when the information processing device 103 receives a start command from the user, it sends a start command to the imaging device 101 and the analysis device 102 and starts the lost child detection process. Then, for example, when the information processing device 103 receives a stop command from the user, it sends a stop command to the imaging device 101 and the analysis device 102 and ends the lost child detection process. In other words, for example, when the information processing device 103 receives a start command from the user, it repeatedly executes the lost child detection process until it receives a stop command from the user. Note that the method of starting or ending the lost child detection process is not limited to these.

[0157] The analysis result acquisition unit 131 receives the analysis transmitted in step S204. result of analysis equipment 10 2 The data is obtained from (step S301). As a result, the analysis result acquisition unit 131 acquires the analysis results and frame images from the analysis device 102.

[0158] The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results obtained in step S301 to detect lost child candidates from the people included in the analysis results (step S302).

[0159] In detail, for example, the candidate detection unit 132 detects individuals as potential lost children based on the personal attributes of each person included in the analysis results obtained in step S301, specifically those individuals associated with personal attributes that satisfy the candidate conditions. If the candidate condition is, for example, 10 years old or younger, the candidate detection unit 132 detects individuals as potential lost children based on personal attributes that include an age of 10 years old or younger.

[0160] The grouping unit 133 uses the person attributes included in the analysis results obtained in step S301 and predetermined grouping conditions to identify the group to which the person in the frame image obtained in step S301 belongs (step S303).

[0161] In detail, for example, the grouping unit 133 detects and groups multiple individuals whose personal attributes satisfy the grouping conditions from among the individuals included in the analysis results obtained in step S301. This allows the grouping unit 133 to identify a group to which multiple individuals who satisfy the grouping conditions belong. This group consists of multiple individuals who travel together.

[0162] For example, the grouping unit 133 groups only those individuals among the individuals included in the analysis results obtained in step S301 for whom there are no other individuals associated with the same grouping criteria. This allows the grouping unit 133 to identify a group to which individuals who do not meet the grouping criteria belong. This group consists of a single individual acting independently.

[0163] The grouping unit 133 may, for example, store the results of the grouping in step 303, that is, the people in the frame image and the group to which each person belongs.

[0164] If a potential lost child has an accompanying person at the first time point, the lost child detection unit 134 detects the lost child from among the potential lost children detected in step S302 (step S304) based on the results of comparing the accompanying person of the potential lost child at the first and second time points.

[0165] Figure 15 is a flowchart showing an example of the detection process (step S304) according to Embodiment 1. If there are multiple lost child candidates detected in step S302, the lost child detection unit 134 may perform the detection process (step S304) for each of the lost child candidates.

[0166] The discrimination unit 134a determines whether or not there is a companion of the lost child at the first time point (step S304a).

[0167] For example, the first time point is the present. In this case, the discrimination unit 134a determines whether the group identified in step S303 includes any person other than the lost child candidate detected in step S302. Thus, the discrimination unit 134a determines whether, at the first time point, there are other people (i.e., companions) belonging to the same group as the lost child candidate.

[0168] If it is determined that there is no accompanying person (step S304a; No), the determination unit 134a terminates the lost child detection process.

[0169] If it is determined that there is a companion (step S304a; Yes), the risk determination unit 134b determines the risk level corresponding to the location of the lost child candidate at the first point in time (step S304b).

[0170] In detail, for example, the risk identification unit 134b obtains the location of the lost child candidate at a first point in time, based on the analysis results obtained in step S301, and the location of the lost child candidate that was determined to have a companion in step S304a. Based on the risk information for each location, the risk identification unit 134b identifies the risk level corresponding to the location of the lost child candidate at a first point in time.

[0171] As mentioned above, location-specific risk information is information that associates the attributes of each location within the target area with its risk level. The risk level is an indicator of the degree of risk to a child getting lost.

[0172] Each location has at least one attribute, such as a parking lot, a shop, or a childcare corner. In this case, the location-specific risk information includes risk levels such as "high," "medium," and "low," corresponding to each of the locations, such as a parking lot, a shop, and a childcare corner. Specifically, a parking lot is often associated with a high risk level because it is less popular. A shop is associated with a medium risk level because it is more popular than a parking lot. A childcare corner is associated with a low risk level because it is likely to be safe.

[0173] Please note that location-specific risk information is not limited to this.

[0174] The risk identification unit 134b, for example, obtains the location attribute to which the location of the lost child candidate at the first point in time belongs, based on the layout information.

[0175] Layout information is information that shows the layout of the target area (i.e., the area where multiple imaging devices 101 will take images). Layout information may include, for example, a floor map as the layout. Layout information may include at least one of the following: the extent of passageways in the target area, the location of each designated area such as a store, the extent of each designated area such as a store, the location of escalators, the location of elevators, etc.

[0176] The risk identification unit 134b then obtains the risk level associated with the attributes of the acquired location from the location-specific risk level information. As a result, the risk identification unit 134b identifies the risk level corresponding to the location of the lost child candidate at the first point in time, who has been identified as being accompanied by someone.

[0177] The lost child identification unit 134c determines whether the risk level identified in step S304b is above a threshold (step S304c). The threshold may be predetermined.

[0178] For example, let's say the threshold is "medium". If the location-specific risk information is as described above, the lost child identification unit 134c determines that the risk level of a potential lost child who is in the "parking lot" or "store" at the first time point is above the threshold. Also, the lost child identification unit 134c determines that the risk level of a potential lost child who is in the "childcare corner" at the first time point is not above the threshold.

[0179] If the system determines that the level of danger is not above a threshold (step S304c; No), the lost child identification unit 134c terminates the lost child detection process. As a result, potential lost children who are in a low-risk, i.e., safe location will no longer be detected as lost.

[0180] If it is determined that the level of danger is above a threshold (step S304c; Yes), the lost child identification unit 134c compares individuals belonging to the same group as the lost child candidate at each of the first and second time points (step S304d).

[0181] More specifically, the second point in time is when the person enters the shopping mall, which is the target area (when entering the store). The first point in time is, for example, the present, as mentioned above. In this case, the lost child identification unit 134c compares individuals belonging to the same group as the lost child candidate at the time of entering the store and at the present.

[0182] Figure 16 is a diagram illustrating the comparison process of companions at the first and second time points (step S304d).

[0183] For example, suppose the current frame image FPA_T1 acquired in step S301 shows a potential lost child LC. Assume this potential lost child LC is accompanied by someone and has a risk level of "medium" or higher.

[0184] The lost child identification unit 134c may refer to the group of people shown in the frame image acquired in step S301 and obtain the person attributes of the people belonging to the same group as the lost child candidate LC. This allows the lost child identification unit 134c to obtain the person attributes of the people accompanying the lost child candidate LC at the moment.

[0185] The lost child identification unit 134c identifies the frame image in which the lost child candidate LC is captured, based on the person attributes obtained from the analysis of each frame image, by going back a predetermined time interval ΔT from the present to the past.

[0186] For example, when searching for the frame image FPA_T1-ΔT containing a potential lost child LC from multiple frame images taken at times T1-ΔT, the lost child identification unit 134c should search sequentially starting with frame images whose shooting areas are close to (e.g., adjacent to) the frame image containing the potential lost child LC at time T. Figure 16 shows an example where the search range to identify the frame image FPA_T1-ΔT containing the potential lost child LC is three frame images.

[0187] By performing this search backward at predetermined time intervals ΔT, the lost child identification unit 134c identifies the frame image in which the lost child candidate LC was first captured, i.e., the frame image FPA_T2 at the time of entry into the store.

[0188] The grouping unit 133 may store the grouping results based, for example, on the analysis results of the frame image FPA_T2 at the time of entry. Alternatively, the grouping unit 133 may identify the group to which each person belongs based on the analysis results of the frame image FPA_T2 at the time of entry.

[0189] The lost child identification unit 134c may refer to the group identified for the frame image FPA_T2 at the time of entry and obtain the person attributes of individuals belonging to the same group as the lost child candidate LC at the time of entry. This allows the lost child identification unit 134c to obtain the person attributes of the companions of the lost child candidate LC at the time of entry.

[0190] The lost child identification unit 134c may, for example, compare the personal attributes of the person accompanying the potential lost child LC at each point in time, both now and at the time of entry into the store. This allows for the comparison of individuals belonging to the same group as the potential lost child at each point in time, both now and at the time of entry into the store.

[0191] Refer to Figure 15 again. The lost child identification unit 134c determines whether or not it has detected a lost child from among the lost child candidates based on the results of the comparison in step S304d (step S304e).

[0192] In detail, for example, the lost child identification unit 134c determines whether there is one or more companions common to the lost child candidate LC at each point in time, based on the personal attributes of the companions at the present time and at the time of entry into the store.

[0193] For example, the lost child identification unit 134c will not detect a lost child (i.e., there is no lost child) if there is one or more common companions at each point in time. and To make a judgment.

[0194] For example, the lost child identification unit 134c determines that a lost child has been abducted if, for instance, there are no common companions at any given time. In other words, in this case, the lost child identification unit 134c detects the lost child from the list of potential lost children.

[0195] If no lost child is detected (step S304e; No), the lost child information generation unit 134d terminates the lost child detection process. If a lost child is detected (step S304e; Yes), the lost child information generation unit 134d generates lost child information regarding the lost child (step S304f) and returns to the lost child detection process.

[0196] Refer to Figure 14 again. The display control unit 135 displays the lost child information generated in step S304f on the display unit 136 (step S305).

[0197] In detail, for example, if multiple lost children are detected in step S304e, the display control unit 135 displays the lost child information generated in step S304f for those multiple lost children on the display unit 136 in the order of the risk levels identified in step S304b.

[0198] The notification unit 137 transmits the lost child information generated in step S304f to each of the one or more terminals 104 (step S306).

[0199] This lost child detection process should be repeatedly executed each time analysis information transmitted during the analysis process is acquired. This makes it possible to detect a lost child who has been abducted. In addition, information about the detected lost child is displayed on the display unit 136, allowing the user to easily notice if a lost child has been abducted.

[0200] (Example of display processing according to Embodiment 1) Figure 17 is a flowchart showing an example of the display process according to Embodiment 1. The display process is a process for displaying the lost child information transmitted by executing the lost child detection process on terminal 104. If there are multiple terminals 104, each terminal 104 may execute the display process.

[0201] Terminal 104 starts display processing, for example, when it launches pre-installed software. Terminal 104 also performs display processing while the software is running. However, the methods for starting and ending display processing are not limited to these.

[0202] The lost child information acquisition unit 141 acquires the lost child information transmitted in step S137 from the information processing device 103 (step S401).

[0203] The display control unit 142 displays the lost child information acquired in step S401 on the display unit 143 (step S402), and then terminates the display process.

[0204] In detail teeth, For example, if lost child information for multiple lost children is acquired in step S401, the display control unit 142 displays the lost child information on the display unit 143 in order of the degree of danger of each lost child included in the lost child information. For example, when terminal 104 receives a predetermined operation to close the lost child information display screen, the display control unit 142 may terminate the display process.

[0205] By performing this display process, the person carrying terminal 104 can quickly notice the missing child who has been taken away and go to rescue the child.

[0206] (Effects / Actions) As described above, according to Embodiment 1, the information processing system 100 includes an analysis result acquisition unit 131, a candidate detection unit 132, and a lost child detection unit 134.

[0207] The analysis result acquisition unit 131 acquires the analysis results of the video footage captured by the multiple camera devices 101. The candidate detection unit 132 uses the person attributes and candidate conditions included in the analysis results to detect potential lost persons from the people shown in the video footage. If a potential lost person has an accompanying person at the first time point, the lost person detection unit 134 detects a lost person from among the potential lost persons based on a comparison of the accompanying person at the first time point and at a second time point prior to the first time point.

[0208] In this way, the system detects lost children from among those who have a companion at the first point in time. Lost children who have a companion at the first point in time are highly likely to have been abducted, and since such lost children can be automatically detected, abducted children can be identified quickly, and measures such as rescue can be taken. Therefore, it becomes possible to ensure the safety of lost children.

[0209] According to Embodiment 1, the candidate conditions include age-related conditions.

[0210] This allows for the detection of lost children by identifying age groups that are more prone to getting lost as potential candidates. This speeds up the detection process compared to, for example, treating all individuals as potential candidates without any age restrictions, enabling earlier detection of children who have been abducted. Consequently, it becomes possible to ensure the safety of lost children.

[0211] According to Embodiment 1, if a potential lost child has an accompanying person at the first time point, the lost child detection unit 134 detects a lost child from among the potential lost children based on whether or not the accompanying person of the potential lost child has changed between the first and second time points.

[0212] This allows for the automatic detection of a missing child who has been abducted, enabling quick detection and the implementation of rescue measures. Therefore, it becomes possible to ensure the safety of lost children.

[0213] According to Embodiment 1, if a person accompanying a potential lost child is present at a first time point, the lost child detection unit 134 detects a lost child from among the potential lost children based on the comparison results and the degree of danger corresponding to the location of the potential lost child at the first time point.

[0214] This makes it possible to detect lost children who are at high risk of danger. Therefore, it becomes possible to ensure the safety of lost children.

[0215] According to Embodiment 1, the information processing system 100 further includes a grouping unit 133 that identifies the group to which a person in a video belongs, using person attributes included in the analysis results and grouping conditions for grouping the people shown in the video. The lost child detection unit 134, if there is a companion of the lost child candidate at the first time point, compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first time point and the second time point, and detects the lost child from among the lost child candidates based on the results of the comparison.

[0216] This allows for easy detection of potential lost children who are accompanied by someone at a second point in time, by grouping individuals using their attributes. Therefore, it can automatically detect children who have been abducted, enabling early detection and rescue efforts. Consequently, it becomes possible to ensure the safety of lost children.

[0217] According to Embodiment 1, the lost child detection unit 134 includes a discrimination unit 134a and a lost child identification unit 134c. The discrimination unit 134a uses the group to which the lost child candidate belongs at the first time point to determine whether or not there is a companion of the lost child candidate at the first time point. If it is determined that there is a companion of the lost child candidate at the first time point, the lost child identification unit 134c compares individuals belonging to the same group as the lost child candidate at the first and second time points, and detects the lost child from among the lost child candidates based on the results of the comparison.

[0218] By grouping individuals using their attributes in this way, it is easy to detect potential lost children who were accompanied by someone at the first point in time. Lost children accompanied by someone at the first point in time are more likely to have been abducted, and since such children can be automatically detected, abducted children can be identified quickly, and measures such as rescue can be taken. Therefore, it becomes possible to ensure the safety of lost children.

[0219] According to Embodiment 1, the lost child information includes at least one of the detected lost child's image and its location at a first time point.

[0220] This makes it easier to find lost children and rescue them more quickly. Therefore, it becomes possible to ensure the safety of lost children.

[0221] According to Embodiment 1, when multiple lost children are detected, the display control unit 135 displays the lost child information of the multiple lost children on the display unit 136 in order of the degree of danger at the first time point.

[0222] This makes it easier to notice lost children who are at high risk of danger. Therefore, it becomes possible to ensure the safety of lost children.

[0223] <Embodiment 2> Generally, parents or guardians accompanying a lost child may visit lost and found centers or management centers to inquire about the child's whereabouts. In such cases, the relevant personnel in the area responding to the parents or guardians may ask them for a description of the lost child. This embodiment describes an example in which an information processing system receives such a description of the lost child, further references that description information, and detects a child that has been abducted.

[0224] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from Embodiment 1.

[0225] The information processing system according to this embodiment includes an information processing device 203 instead of the information processing device 103 according to Embodiment 1. Except for this point, the information processing system according to this embodiment may be configured in the same way as the information processing system 100 according to Embodiment 1.

[0226] Figure 18 shows an example of the functional configuration of the information processing device 203 according to Embodiment 2. The information processing device 203 includes a candidate detection unit 232 and a grouping unit 233 instead of the candidate detection unit 132 and grouping unit 133 according to Embodiment 1. The information processing device 203 further includes a feature acquisition unit 251. Except for these, the information processing device 203 according to this embodiment may be configured in the same way as the information processing device 103 according to Embodiment 1.

[0227] The feature acquisition unit 251 acquires characteristic information of the lost child to be detected based on input from a user who has learned the characteristics of the lost child verbally or otherwise. The feature acquisition unit 251 may further acquire characteristic information of the person (accompanying person) who provided the characteristic information of the lost child based on input from a user. The characteristic information of the accompanying person may include an image of the accompanying person obtained by the user taking a picture of the accompanying person.

[0228] The candidate detection unit 232, similar to the candidate detection unit 132 in Embodiment 1, uses the person attributes and candidate conditions included in the analysis results acquired by the analysis result acquisition unit 131 to detect lost child candidates from the people shown in the video. This embodiment differs from Embodiment 1 in that the candidate conditions include feature information acquired by the feature acquisition unit 251.

[0229] The grouping unit 233, similar to the grouping unit 133 in Embodiment 1, uses the person attributes included in the analysis results and predetermined grouping conditions to identify the group to which the person in the video belongs. In this embodiment, the grouping unit 233 further uses the lost child characteristic information acquired by the feature acquisition unit 251 to identify the group to which the person in the video belongs.

[0230] In detail, for example, the grouping unit 233 may use the characteristics information of the lost child and the characteristics information of the accompanying person to identify the group to which the person in the video belongs. In this case, the grouping unit 233 identifies people whose person attributes included in the analysis results are similar to the characteristics information of the lost child and the accompanying person respectively as belonging to a common group.

[0231] Here, "similar" means similar to a certain extent that satisfies predetermined conditions, specifically, for example, that the degree of similarity is above a threshold. Note that the grouping unit 233 does not necessarily have to use grouping conditions.

[0232] The information processing system according to this embodiment may be physically configured in the same way as the information processing system 100 according to Embodiment 1.

[0233] (Operation of the information processing system according to Embodiment 2) The information processing according to this embodiment includes the same imaging, analysis, and display processes as in Embodiment 1, as well as a lost child detection process that differs from that in Embodiment 1. In this embodiment as well, the lost child detection process is performed by the information processing device 203.

[0234] (Example of lost child detection process according to Embodiment 2) Figure 19 is a flowchart showing an example of the lost child detection process according to Embodiment 2. As shown in the figure, the lost child detection process according to this embodiment involves the same step S30 as in Embodiment 1. 1 This includes step S501, which is executed subsequently, and steps S502 to S503, which replace steps S302 to S303 in Embodiment 1. Aside from these, the lost child detection process in Embodiment 2 may be configured in the same way as the lost child detection process in Embodiment 1.

[0235] The feature acquisition unit 251 acquires feature information based on user input, etc. (step S501).

[0236] In detail, for example, the feature acquisition unit 251 acquires characteristic information of the lost child to be detected and characteristic information of the person accompanying the lost child, based on user input, etc. This person accompanying the lost child is, for example, the child's guardian.

[0237] The candidate detection unit 232 uses the person attributes included in the analysis results obtained in step S301 and the candidate conditions including the characteristics information of the lost child obtained in step S501 to detect a lost child candidate from the people included in the analysis results (step S502).

[0238] In detail, for example, the candidate detection unit 232 detects individuals as potential lost persons based on the personal attributes of each person included in the analysis results obtained in step S301, specifically those associated with personal attributes that satisfy the candidate conditions. Personal attributes that satisfy the candidate conditions may, for example, be personal attributes similar to the feature information included in the candidate conditions.

[0239] The grouping unit 233 uses the person's attributes, predetermined grouping conditions, and the feature information acquired in step S501 to identify the group to which the person in the frame image acquired in step S301 belongs (step S503).

[0240] The person attributes here are those included in the analysis results obtained in step S301. The feature information here is the feature information obtained in step S501, such as the feature information for the lost person and their companion.

[0241] In detail, for example, the grouping unit 233 detects multiple individuals from the analysis results obtained in step S301 who are associated with personal attributes that satisfy the grouping conditions. The grouping unit 233 further detects and groups individuals from among the detected multiple individuals who are associated with personal attributes similar to the characteristic information of the lost person and their companion, respectively.

[0242] By executing the lost child detection process according to this embodiment, it is possible to detect if a lost child has been abducted using characteristic information of the lost child obtained verbally or otherwise.

[0243] (Effects / Actions) As described above, according to Embodiment 2, the information processing system Mu is The system further includes a feature acquisition unit 251 that acquires characteristic information of the lost child to be detected. The candidate conditions include characteristic information of the lost child.

[0244] This allows for the detection of a lost child who has been abducted, using the child's characteristic information. Since abducted children are generally likely to be in dangerous situations, this system enables the early detection of children in such dangerous circumstances. Therefore, it becomes possible to ensure the safety of lost children.

[0245] According to Embodiment 2, the information processing system Mu isThe system further includes a feature acquisition unit 251 that acquires characteristic information of the lost child to be detected. The grouping unit 233 further uses the lost child's characteristic information to identify the group to which the person in the video belongs.

[0246] This allows for the identification of the group to which the lost child belongs and the identification of any companions, thereby more reliably detecting whether or not the lost child has been taken away. Therefore, if the lost child has been abducted, this can be reliably detected. Consequently, the safety of the lost child can be ensured.

[0247] <Embodiment 3> This embodiment describes an example in which the range of movement of a lost child is predicted, and this predicted range of movement is used for the search range and lost child information. Note that the range of movement may be used for either the search range or the lost child information, or only one of them.

[0248] In this embodiment, in order to simplify the explanation, we will mainly describe the differences from Embodiment 1.

[0249] The information processing system according to this embodiment includes an information processing device 303 instead of the information processing device 103 according to Embodiment 1. Except for this point, the information processing system according to this embodiment may be configured in the same way as the information processing system 100 according to Embodiment 1.

[0250] Figure 20 shows an example of the functional configuration of the information processing device 303 according to Embodiment 3. The information processing device 303 includes a lost child detection unit 334 and a display control unit 335 instead of the lost child detection unit 134 and display control unit 135 according to Embodiment 1. 3 03 further comprises a pattern detection unit 361 and a range prediction unit 362. Aside from these, the information processing device 303 according to this embodiment may be configured in the same way as the information processing device 103 according to Embodiment 1.

[0251] The pattern detection unit 361 detects the movement pattern of a person in the video based on the person's attributes between the first and second time points.

[0252] A movement pattern is a tendency related to the movement of a person, and includes, for example, one or more of an average movement speed, a movement speed in front of a store, a time of stopping in front of a store, a type of store where deceleration or stop occurs, a type of store visited, an average movement speed inside a store, and the like.

[0253] A person to be the target of detecting a movement pattern is, for example, one or more of a detected lost child, a lost child candidate, a companion of a lost child, and a companion of a lost child candidate. Note that a person to be the target of detecting a movement pattern is not limited thereto.

[0254] The range prediction unit 362 predicts the movement range of a person shown in an image using person attributes. The range prediction unit 362 may predict the movement range of a person shown in an image using at least one of, for example, a person's position, movement direction, and movement speed among the person attributes.

[0255] The range prediction unit 362 may, for example, predict the movement range of a person shown in an image between a first time point and a second time point. In this case, for example, the range prediction unit 362 may predict the movement range of a person shown in an image between the first time point and the second time point using the movement pattern detected by the pattern detection unit 361 in addition to the person attributes.

[0256] The range prediction unit 362 may, for example, predict the movement range of a person after a first time point. When the first time point is the present, the movement range after the first time point is a future movement range. In this case, for example, the range prediction unit 362 may predict the movement range of a person using the person attributes (for example, at least one of the position, movement direction, and movement speed of the lost child) at the first time point.

[0257] Furthermore, the range prediction unit 362 may predict the range of movement of the person, for example, by further using layout information. In this case, for example, the range prediction unit 362 may predict the range of movement, including movement between floors, of the person based on the positions of escalators and elevators included in the layout information, and at least one of the person's position, direction of movement, and speed of movement. The range prediction unit 362 may store layout information in advance.

[0258] The individuals whose movement range is predicted include, for example, one or more of the following: the detected lost child, the suspected lost child, the lost child's companion, and the suspected lost child's companion. range The individuals who may be detected are not limited to those mentioned above.

[0259] The lost child detection unit 334, similar to the lost child detection unit 134 in Embodiment 1, detects a lost child from among the candidates for lost child and generates lost child information related to the detected lost child.

[0260] Figure 21 shows an example of the functional configuration of the lost child detection unit 334 according to Embodiment 3. The lost child detection unit 334 includes a lost child identification unit 334c and a lost child information generation unit 334d instead of the lost child identification unit 134c and the lost child information generation unit 134d according to Embodiment 1. Except for this point, the lost child detection unit 334 may be configured in the same way as the lost child detection unit 134 according to Embodiment 1.

[0261] The lost child identification unit 334c, similar to the lost child identification unit 134c in Embodiment 1, detects a lost child from among the lost child candidates based on the results of comparing the companions of the lost child candidates at the first and second time points.

[0262] In this embodiment, the lost child identification unit 334c sets the movement range predicted by the range prediction unit 362 for a person as the search range for that person, and detects a lost child candidate from the person that appears within the search range.

[0263] The lost child information generation unit 334d is similar to the lost child information generation unit 134d in Embodiment 1, and the lost child identification unit 3 34c generates lost child information about the lost child it detected.

[0264] The lost child information according to this embodiment may include the range of movement predicted by the range prediction unit 362 for the lost child. In this case, for example, the lost child information may include the range of movement after the first time point in time predicted by the range prediction unit 362 for the lost child.

[0265] Refer to Figure 20 again. The display control unit 335 displays various information on the display unit 136, similar to the display control unit 135 in Embodiment 1. The display control unit 335, for example, is a lost child detection unit. 3 34 (For details, see the Lost Child Information Generation Unit) 3 The lost child information generated by 34d) may be displayed on the display unit 136.

[0266] The lost child information according to this embodiment may further include layout information. In this case, for example, the display control unit 335 may display on the display unit 136 an image on which the movement range predicted by the range prediction unit 362 for the lost child is superimposed on the layout information.

[0267] For example, the display control unit 335 may display an image on the display unit 136 in which the position of the lost child at the first time point is superimposed on the layout information. 3 35 may display an image on the display unit 136 in which the location of the lost child at the second point in time is superimposed on the layout information.

[0268] The information processing system according to this embodiment may be physically configured in the same way as the information processing system 100 according to Embodiment 1.

[0269] (Operation of the information processing system according to Embodiment 3) The information processing according to this embodiment includes the same imaging, analysis, and display processing as in Embodiment 1, as well as a lost child detection process that differs from that in Embodiment 1. In this embodiment as well, the lost child detection process is performed by the information processing device 303.

[0270] (Example of lost child detection process according to Embodiment 3) FIG. 22 is a flowchart showing an example of the lost child detection process according to Embodiment 3. As shown in the figure, the lost child detection process according to the present embodiment includes steps S604 to S605 that replace steps S304 to S305 according to Embodiment 1. Except for these, the lost child detection process according to Embodiment 3 may be configured in the same manner as the lost child detection process according to Embodiment 1.

[0271] The lost child detection unit 334 detects a lost child from the lost child candidates detected in step S302, in the same manner as the lost child detection unit 134 according to Embodiment 1 (step S604). In the present embodiment, the details of the detection process (step S604) are different from the detection process (step S304) according to Embodiment 1.

[0272] FIG. 23 is a flowchart showing an example of the detection process (step S604) according to Embodiment 3. The detection process (step S604) according to the present embodiment includes steps S604d and S604f that replace steps S304d and S304f according to Embodiment 1. The detection process (step S604) according to the present embodiment further includes a step S604g that is executed between step S304e and S604f. Except for these, the detection process (step S604) according to the present embodiment may be configured in the same manner as the detection process (step S304) according to Embodiment 1.

[0273] The lost child identification unit 334c compares the persons belonging to the same group as the lost child candidates at each of the first time point and the second time point when it is determined that the risk level is equal to or higher than the threshold (step S304c; Yes), in the same manner as in Embodiment 1 (step S604d). In the present embodiment, the details of the comparison process (step S604d) are different from the comparison process (step S304d) according to Embodiment 1.

[0274] FIGS. 24 and 25 are flowcharts showing an example of the comparison process (step S604d) according to Embodiment 3.

[0275] The lost child identification unit 334c sets the time of the first point in time T1 to the time of the photograph T (step S604d1). The first point in time is, for example, the present, as in Embodiment 1.

[0276] The lost child identification unit 334c sets the frame image taken at time T, which is a time interval ΔT prior, as the search target (step S604d2).

[0277] For example, if the shooting time T is set to the time T1 of the first point in time, the lost child identification unit 334c sets the frame image from shooting time T1-ΔT as the search target.

[0278] The pattern detection unit 361 detects the movement pattern of the lost child candidate based on the person attributes included in the analysis results (step S604d3).

[0279] For example, in step S604d3, in order to detect the movement pattern of a potential lost child, the analysis results generated based on frame images taken from the time of the first capture to the time of the frame image being searched are used.

[0280] The range prediction unit 362 predicts the range of movement of the lost person candidate using the person's attributes and the movement pattern detected in step S604d3 (step S604d4).

[0281] The lost child identification unit 334c sets a search range for part or all of the frame image to be searched based on the movement range predicted in step S604d4 (step S604d5).

[0282] In more detail, for example, the lost child identification unit 334c sets the search range to the frame images that include the movement range predicted in step S604d4 from among the frame images to be searched.

[0283] The lost child identification unit 334c determines whether or not it has identified a frame image containing a candidate for a lost child from the search range set in step S604d5 (step S604d6).

[0284] In detail, for example, the lost child identification unit 334c searches for a frame image containing a potential lost child within the search range of frame images. If a frame image containing a potential lost child is detected, the lost child identification unit 334c determines that it has identified the frame image containing the potential lost child. If no frame image containing a potential lost child is detected, the lost child identification unit 334c determines that it has not identified a frame image containing a potential lost child.

[0285] If the system determines that it cannot identify a frame image containing a potential lost child (step S604d6; No), the lost child identification unit 334c returns to step S604d5. In the re-executed step S604d5, the lost child identification unit 334c may, for example, set the search range to a frame image containing an area within the search range set in the previous step S604d5 and an adjacent area.

[0286] The lost child identification unit 334c determines whether the time of shooting T is the second time point in time (step S604d7).

[0287] If it is determined that the child is not lost at the second step (step S604d7; No), the lost child identification unit 334c returns to step S604d2.

[0288] Refer to Figure 25. If it is determined that it is the second time point (step S604d7; Yes), the lost child identification unit 334c identifies a person who belongs to the same group as the lost child candidate at the second time point (step S604d8).

[0289] More specifically, for example, the lost child identification unit 334c identifies individuals who belong to the same group as the lost child candidate (i.e., companions of the lost child candidate) using the analysis results and grouping conditions based on the frame images identified in S604d6.

[0290] The lost child identification unit 334c determines whether all the individuals identified as companions of the lost child candidate have changed between the first and second time points (step S604d9).

[0291] In detail, for example, the lost child identification unit 334c obtains the person attributes of the person identified as a companion of the lost child candidate at the first time point in step S304a. The lost child identification unit 334c obtains the person attributes of the person identified as a companion of the lost child candidate at the second time point in step S604d8.

[0292] The lost child identification unit 334c determines whether all of the companions of the lost child candidate have changed between the first and second time points by comparing the personal attributes of the companions at the first and second time points. For example, if the similarity of the personal attributes of all companions of the lost child candidate between the first and second time points is below a predetermined threshold, the lost child identification unit 334c determines that all have changed. Alternatively, for example, if at least one of the companions of the lost child candidate between the first and second time points has a similarity of personal attributes above a predetermined threshold, the lost child identification unit 334c determines that all have not changed.

[0293] If it is determined that everything has changed (step S604d9; Yes), the lost child identification unit 334c detects the lost child (step S604d10) and returns to the detection process (step S604). In other words, in this case, the lost child candidate is detected as a lost child.

[0294] If it is determined that nothing has changed (step S604d9; No), the lost child identification unit 334c returns to the detection process (step S604) without detecting a lost child (step S604d11). In other words, in this case, the lost child candidate is treated as not being lost.

[0295] Refer to Figure 23 again. If a lost child is detected in step S304e, similar to that of Embodiment 1 (step S304e; Yes), the range prediction unit 362 predicts the future range of movement of the lost child based on the person attributes of the lost child detected in step S304e (step S604g).

[0296] The lost child information generation unit 134d generates lost child information regarding the lost child (step S604f) and returns to the lost child detection process. The lost child information generated here includes, for example, the movement range and layout information generated in step S604g.

[0297] Refer to Figure 22 again. The display control unit 335 displays the lost child information generated in step S604f on the display unit 136 (step S605). The display control unit 335 displays on the display unit 136 a screen in which the predicted future movement range of the lost child is superimposed on the layout information.

[0298] In this type of lost child detection process, the search area can be narrowed down from the frame image based on the predicted movement range of the lost child candidate. Therefore, the processing load in the comparison process can be reduced.

[0299] Furthermore, the display unit 136 can display the predicted future movement range of the lost child. This makes it easier to find the lost child who has been abducted and increases the likelihood of finding the child quickly.

[0300] (Effects / Actions) As described above, according to Embodiment 3, the information processing system further includes a range prediction unit 362 that uses person attributes to predict the range of movement of a person shown in the video.

[0301] This reduces processing load and speeds up lost child detection by predicting the range of movement of potential lost children. Furthermore, by referring to the detected lost child's movement range, the child can be easily searched for, facilitating their rescue. Therefore, it becomes possible to ensure the safety of lost children.

[0302] According to Embodiment 3, the range prediction unit 362 further uses layout information of the locations captured by the multiple imaging devices 101 to predict the range of movement of a person.

[0303] This improves the prediction of the child's movement range. Therefore, it becomes possible to further speed up the detection of lost children and make their rescue easier. Consequently, it becomes possible to ensure the safety of lost children.

[0304] According to Embodiment 3, the information processing system further includes a pattern detection unit 361 that detects the movement pattern of a person shown in the video based on the person's attributes between a first time point and a second time point. The range prediction unit 362 further uses the movement pattern to predict the range of movement of the person shown in the video between the first time point and the second time point.

[0305] This improves the prediction of the child's movement range. Therefore, it becomes possible to further speed up the detection of lost children and make their rescue easier. Consequently, it becomes possible to ensure the safety of lost children.

[0306] According to Embodiment 3, the lost child detection unit 334 sets the predicted movement range of a potential lost child as the search range for the potential lost child, and detects the potential lost child from the people who appear within the search range.

[0307] This allows for faster detection of lost children by predicting the range of movement of potential lost children. Therefore, it becomes possible to ensure the safety of lost children.

[0308] According to Embodiment 3, the information processing system further includes a display control unit 335 that displays lost child information on a display unit 136. A range prediction unit 362 predicts the range of movement of the detected lost child. The lost child information includes the predicted range of movement.

[0309] This allows for searching for the lost child by referring to the detected range of movement, making it easier to rescue the child. Therefore, it becomes possible to ensure the safety of the lost child.

[0310] According to Embodiment 3, the lost child information further includes layout information. The display control unit 335 causes the display unit 136 to display an image on which the predicted movement range is superimposed on the layout information.

[0311] This makes it easier to locate a lost child by referring to the detected range of their movement, thus facilitating their rescue. Therefore, it becomes possible to ensure the safety of lost children.

[0312] The embodiments and modifications of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0313] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments and modifications can be combined to the extent that their content is not contradictory.

[0314] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0315] 1. An analysis result acquisition means for acquiring the analysis results of images captured by multiple shooting means, A candidate detection means for detecting a lost child candidate from the person shown in the video, using the person attributes and candidate conditions included in the analysis results, The system includes a lost child detection means that, if there is an accompanying person for the lost child at a first time point, detects the lost child from among the lost child candidates based on a comparison of the accompanying person at the first time point with a second time point prior to the first time point. Information processing system. 2. The aforementioned candidate criteria include age-related criteria. The information processing system described in 1. 3. Further comprising a feature acquisition means for acquiring characteristic information of the lost child to be detected, The aforementioned candidate conditions include the characteristic information of the lost child. The information processing system described in 1. or 2. 4. The lost child detection means detects a lost child from among the lost child candidates based on whether or not the companion of the lost child candidate has changed between the first and second time points, if the lost child candidate has a companion at the first time point. An information processing system described in any one of the following three items. 5. The lost child detection means detects a lost child from among the lost child candidates based on the comparison results and the degree of danger corresponding to the location of the lost child candidate at the first time point, if the lost child candidate has an accompanying person at the first time point. An information processing system described in any one of items 1 through 4. 6. The system further comprises a grouping means for identifying the group to which a person in the video belongs, using the person attributes included in the analysis results and grouping conditions for grouping the people shown in the video. The lost child detection means, if there is a companion of the lost child candidate at the first time point, compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first time point and the second time point, and detects the lost child from among the lost child candidates based on the results of the comparison. An information processing system described in any one of items 1 through 5. 7. The lost child detection means is: A determination means for determining whether or not the lost child candidate has an accompanying person at the first time point, using the group to which the lost child candidate belongs at the first time point, If it is determined that the missing person candidate has an accompanying person at the first time point, the system includes a means for identifying a missing person, which compares individuals belonging to the same group as the missing person candidate at both the first and second time points, and detects the missing person from among the missing person candidates based on the results of the comparison. The information processing system described in 6. 8. Further comprising a feature acquisition means for acquiring characteristic information of the lost child to be detected, The aforementionedThe grouping means further uses the characteristic information of the lost child to identify the group to which the person in the video belongs. The information processing system described in 6. or 7. 9. The system further comprises a range prediction means that uses the aforementioned person attributes to predict the range of movement of the person shown in the video. An information processing system described in any one of items 1 through 8. 10. The range prediction means further uses the layout information of the location captured by the plurality of shooting means to predict the range of movement of the person. The information processing system described in 9. 11. The system further comprises pattern detection means for detecting the movement pattern of a person shown in the video based on the person's attributes between the first and second time points, The range prediction means further uses the movement pattern to predict the range of movement of the person shown in the video between the first time point and the second time point. The information processing system described in 9. or 10. 12. The lost child detection means sets the predicted movement range of the lost child candidate as the search range for the lost child candidate, and detects the lost child candidate from the people who appear within the search range. The information processing system described in 11. 13. The system further comprises a display control means for displaying the lost child information related to the detected lost child on a display means, The range prediction means predicts the range of movement of the detected lost child, The lost child information includes the predicted range of movement. An information processing system described in any one of items 9 through 12. 14. The aforementioned lost child information further includes the aforementioned layout information, The display control means causes the display means to display an image on which the predicted movement range is superimposed on the layout information. The information processing system described in 13. 15. The lost child information includes at least one of the image of the detected lost child and the location at the first time point. The information processing system described in 13. or 14. 16. If there are multiple lost children detected, the display control means will display the lost child information of the multiple lost children on the display means in order of their degree of danger at the first time point. An information processing system described in any one of items 13 through 15. 17. Analysis result acquisition means for acquiring analysis results of video footage captured by multiple shooting means, A candidate detection means for detecting a lost child candidate from the person shown in the video, using the person attributes and candidate conditions included in the analysis results, The system includes a lost child detection means that, if there is an accompanying person for the lost child at a first time point, detects the lost child from among the lost child candidates based on a comparison of the accompanying person at the first time point with a second time point prior to the first time point. Information processing device. 18. One or more computers, We obtained the analysis results of videos captured using multiple shooting methods. Using the person attributes and candidate conditions included in the analysis results, a lost child candidate is detected from the person shown in the video. If a person accompanying the potential lost child is present at the first time point, the system detects the lost child from among the potential lost children based on a comparison of the person accompanying the potential lost child at the first time point and at a second time point prior to the first time point. Information processing methods. 19. On one or more computers, We obtained the analysis results of videos captured using multiple shooting methods. Using the person attributes and candidate conditions included in the analysis results, a lost child candidate is detected from the person shown in the video. If a person accompanying the potential lost child is present at the first time point, the system detects the lost child from among the potential lost children based on a comparison of the person accompanying the potential lost child at the first time point and at a second time point prior to the first time point. A recording medium on which a program to execute a program is stored. [Explanation of symbols]

[0316] 100 Information Processing Systems 101,101_1~101_M1 Imaging device 102 Analyzer 103,203,303 Information Processing Equipment 104,104_1~104_M2 terminals 131 Analysis result acquisition section 132,232 Candidate detection unit 133,233 Grouping Department 134,334 Lost child detection unit 134a Discriminant section 134b Hazard Assessment Section 134c, 334c Lost and Found Department 134d, 334d Lost Child Information Generation Unit 135,335 Display Control Unit 136 Display section 137 Notification Department 141 Lost and Found Information Acquisition Department 142 Display Control Unit 143 Display section 251 Feature Acquisition Unit 361 Pattern detection unit 362 Range prediction unit

Claims

1. An analysis result acquisition means for acquiring the analysis results of images captured by multiple shooting means, A candidate detection means for detecting a lost child candidate from the person shown in the video, using the person attributes and candidate conditions included in the analysis results, A lost child detection means that, when there is a companion of the lost child candidate at a first time point, detects a lost child from among the lost child candidates based on the results of comparing the companion of the lost child candidate at the first time point with a second time point prior to the first time point, A pattern detection means detects a movement pattern for each of the lost child candidates based on the aforementioned person attributes, which includes one or more of the following: speed of movement in front of the store, time spent stopping in front of the store, type of store where the child slows down or stops, type of store visited, and average speed of movement within the store. A range prediction means that uses the aforementioned person attributes and movement patterns to predict the range of movement of a person shown in the video, Equipped with, The lost child detection means sets a search range for the lost child candidate at the second time point, based on the predicted movement range. Information processing system.

2. The system further includes a feature acquisition method for obtaining characteristic information of the lost child to be detected. The aforementioned candidate conditions include the characteristic information of the lost child. The information processing system according to claim 1.

3. The lost child detection means detects a lost child from among the lost child candidates based on whether or not the companion of the lost child candidate has changed between the first and second time points, if the lost child candidate has a companion at the first time point. The information processing system according to claim 1 or 2.

4. The lost child detection means detects a lost child from among the candidates based on the comparison results and the degree of danger corresponding to the location of the lost child candidate at the first time point, if the lost child candidate has a companion at the first time point. The information processing system according to claim 1 or 2.

5. The system further comprises a grouping means that identifies the group to which a person in the video belongs, using the person attributes included in the analysis results and grouping conditions for grouping the people shown in the video. The lost child detection means, if there is a companion of the lost child candidate at the first time point, compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first time point and the second time point, and detects the lost child from among the lost child candidates based on the results of the comparison. The information processing system according to claim 1 or 2.

6. The pattern detection means detects the movement pattern based on the person's attributes between the first time point and the second time point. The information processing system according to claim 1.

7. The lost child detection means determines whether all of the companions at the first time point have changed from the companions at the second time point, if there are companions belonging to the same group as the lost child candidate at the first time point, and detects the lost child if all of them have changed. The information processing system according to claim 1.

8. An analysis result acquisition means for acquiring the analysis results of images captured by multiple shooting means, A candidate detection means for detecting a lost child candidate from the person shown in the video, using the person attributes and candidate conditions included in the analysis results, A lost child detection means that, when there is a companion of the lost child candidate at a first time point, detects a lost child from among the lost child candidates based on the results of comparing the companion of the lost child candidate at the first time point with a second time point prior to the first time point, A pattern detection means detects a movement pattern for each of the lost child candidates based on the aforementioned person attributes, which includes one or more of the following: speed of movement in front of the store, time spent stopping in front of the store, type of store where the child slows down or stops, type of store visited, and average speed of movement within the store. A range prediction means that uses the aforementioned person attributes and movement patterns to predict the range of movement of a person shown in the video, Equipped with, The lost child detection means sets a search range for the lost child candidate at the second time point, based on the predicted movement range. Information processing device.

9. One or more computers, We obtained the analysis results of videos captured using multiple shooting methods. Using the person attributes and candidate conditions included in the analysis results, a lost child candidate is detected from the person shown in the video. If there is a companion of the missing child candidate at the first time point, the missing child is detected from among the missing child candidates based on a comparison of the companion of the missing child candidate at the first time point and at a second time point prior to the first time point. Based on the aforementioned person attributes, a movement pattern is detected for each potential lost child that includes one or more of the following: speed of movement in front of the store, time spent stopping in front of the store, type of store where the child slows down or stops, type of store visited, and average speed of movement within the store. Using the aforementioned person attributes and movement patterns, the range of movement of the person shown in the video is predicted. In the process of detecting a lost child from among the aforementioned lost child candidates, a search range is set for searching the lost child candidates at the second time point based on the predicted movement range. Information processing methods.

10. On one or more computers, We obtained the analysis results of videos captured using multiple shooting methods. Using the person attributes and candidate conditions included in the analysis results, a lost child candidate is detected from the person shown in the video. If there is a companion of the missing child candidate at the first time point, the missing child is detected from among the missing child candidates based on a comparison of the companion of the missing child candidate at the first time point and at a second time point prior to the first time point. Based on the aforementioned person attributes, a movement pattern is detected for each potential lost child that includes one or more of the following: speed of movement in front of the store, time spent stopping in front of the store, type of store where the child slows down or stops, type of store visited, and average speed of movement within the store. Using the aforementioned person attributes and movement patterns, the range of movement of the person shown in the video is predicted. In the process of detecting a lost child from among the aforementioned lost child candidates, the search range for the lost child candidates at the second time point is set based on the predicted movement range. A program to execute.

Images