Information Processing System, Information Processing Apparatus, Information Processing Method, and Program

Abstract

An information processing system (100) comprises an analysis result acquisition unit (131), a candidate detection unit (132), and a missing child detection unit (134). The analysis result acquisition unit (131) acquires an analysis result of video captured by a plurality of video capturing devices (101). The candidate detection unit (132) uses a person attribute included in the analysis result and a candidate condition to detect missing child candidates from among persons captured in the video. The missing child detection unit (134) detects, when the missing child candidate has an accompanying person at a first time point, on the basis of a result of comparison between the accompanying person of the missing child candidate between this first time point and an accompanying person at a second time point earlier than this first time point, a missing child from among the missing child candidates.

Description

Information processing system, information processing device, information processing method, and recording medium

[0001] The present invention relates to an information processing system, an information processing device, an information processing method, and a recording medium.

[0002] For example, Patent Document 1 discloses a technique for detecting a lost child.

[0003] The lost child identification unit described in Patent Document 1 extracts only people of an age where they are particularly likely to become lost, based on person information.

[0004] This personal information is the result of extracting features such as contours from images taken by a surveillance camera installed in a certain location using a feature extraction unit, person extraction unit, and personal feature analysis unit, automatically identifying people and analyzing their personal features such as age, clothing, and physique.

[0005] The lost child identification unit described in Patent Document 1 identifies a person as lost if it determines that there is a possibility that the person is lost based on the results of the person's behavior analysis performed in parallel by the behavior analysis unit, such as anxious expressions and behavior, and whether the person is acting alone (alone).

[0006] Patent Document 2 describes a technology that calculates the feature values ​​of each of multiple key points of a human body contained in an image, searches for images containing human bodies with similar postures or movements based on the calculated feature values, and classifies images with similar postures or movements together.

[0007] Non-Patent Document 1 describes a technique related to human skeleton estimation.

[0008] JP 2021-108149 A International Publication No. 2021 / 084677

[0009] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, P. 7291-7299

[0010] However, as described above, the technology described in Patent Document 1 detects a lost child based on information such as anxious facial expressions and behavior, whether the child is acting alone, etc. Therefore, it is difficult to accurately detect a lost child who has been abducted by a complete stranger.

[0011] For example, it is generally difficult to accurately detect the facial expressions and behavior of a person in an image. Even if it were possible, there is a risk that the facial expressions and behavior of a person in an image cannot be detected with high accuracy if the image quality is poor. If the accuracy of detecting anxious facial expressions and behavior is low, the technology described in Patent Document 1 may not be able to accurately detect a lost child that has been abducted.

[0012] Furthermore, for example, a person of an age where they may get lost may generally look anxious or behave anxiously even when they are with their guardian. In such a case, the technology described in Patent Document 1 may detect the person as lost even when they are with their guardian.

[0013] Furthermore, for example, it is highly likely that an abducted lost child is traveling with a stranger and not traveling alone, so it is difficult to detect an abducted lost child using the fact that the child is traveling alone (alone) in the technology described in Patent Document 1.

[0014] Abduction is likely to be a dangerous situation for the lost child in question, and detecting it is extremely important for the safety of the lost child.

[0015] It should be noted that Patent Document 2 and Non-Patent Document 1 do not disclose a technique for detecting a lost child.

[0016] In view of the above-mentioned problems, an example of an object of the present invention is to provide an information processing system, an information processing device, an information processing method, and a recording medium that solve the problem of ensuring the safety of lost children.

[0017] According to one aspect of the present invention, an information processing system is provided that includes: an analysis result acquisition means for acquiring analysis results of images captured by multiple image capture means; a candidate detection means for detecting potential lost persons from people captured in the images using person attributes and candidate conditions included in the analysis results; and a lost person detection means for detecting a lost person from among the potential lost persons, if there is a companion of the lost person candidate at a first time point, based on the results of comparing the companion of the lost person candidate at the first time point with the companion of the lost person candidate at a second time point that is earlier than the first time point.

[0018] According to one aspect of the present invention, an information processing method is provided in which one or more computers obtain analysis results of images captured by multiple imaging means, use personal attributes and candidate conditions included in the analysis results to detect potential lost persons from among the people captured in the images, and, if the potential lost person has a companion at a first time point, detect a lost person from among the potential lost persons based on the results of comparing the companions of the potential lost person at the first time point with the companions of the potential lost person at a second time point that is earlier than the first time point.

[0019] According to one aspect of the present invention, a recording medium having recorded thereon a program for causing one or more computers to acquire analysis results of images captured by multiple image capture means, detect potential lost children from people captured in the images using personal attributes and candidate conditions included in the analysis results, and, if the potential lost child has a companion at a first time point, detect the lost child from among the potential lost children based on the results of comparing the companions of the potential lost child at the first time point with the companions of the potential lost child at a second time point that is earlier than the first time point.

[0020] According to one aspect of the present invention, it is possible to ensure the safety of lost children.

[0021] 1 is a diagram illustrating an overview of an information processing system according to a first embodiment. FIG. 2 is a diagram illustrating an overview of an information processing device according to the first embodiment. FIG. 3 is a flowchart illustrating an overview of information processing according to the first embodiment. FIG. 4 is a diagram illustrating an example of a configuration of an information processing system. FIG. 5 is a diagram illustrating an example of a functional configuration of an information processing device according to the first embodiment. FIG. 6 is a diagram illustrating an example of a functional configuration of a lost child detection unit according to the first embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of a terminal according to the first embodiment. FIG. 8 is a diagram illustrating an example of a physical configuration of an imaging device according to the first embodiment. FIG. 9 is a diagram illustrating an example of a physical configuration of an analysis device according to the first embodiment. FIG. 10 is a flowchart illustrating an example of an imaging process according to the first embodiment. FIG. 11 is a diagram illustrating an example of a floor map of a target area. FIG. 12 is a diagram illustrating an example of frame information. FIG. 13 is a flowchart illustrating an example of an analysis process according to the first embodiment. FIG. 14 is a flowchart illustrating an example of a lost child detection process according to the first embodiment. FIG. 15 is a flowchart illustrating an example of a detection process according to the first embodiment. FIG. 16 is a diagram for explaining a comparison process according to the first embodiment. FIG. 17 is a flowchart illustrating an example of a display process according to the first embodiment. FIG. 18 is a diagram illustrating an example of a functional configuration of an information processing device according to the second embodiment. FIG. 19 is a flowchart illustrating an example of a lost child detection process according to the second embodiment. FIG. 20 is a diagram illustrating an example of a functional configuration of an information processing device according to the third embodiment. FIG. 21 is a diagram illustrating an example of a functional configuration of a lost child detection unit according to the third embodiment. FIG. 22 is a flowchart illustrating an example of a lost child detection process according to the third embodiment. FIG. 23 is a flowchart illustrating an example of a detection process according to the third embodiment. 11 is a flowchart illustrating an example of a comparison process according to a third embodiment.

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0023] 1 is a diagram showing an overview of an information processing system 100 according to embodiment 1. The information processing system 100 includes an analysis result acquisition unit 131, a candidate detection unit 132, and a lost child detection unit 134.

[0024] The analysis result acquisition unit 131 acquires the analysis results of the videos captured by the multiple image capture devices 101 .

[0025] The candidate detection unit 132 detects lost child candidates from people captured in the video using the person attributes and candidate conditions included in the analysis results.

[0026] When a companion of a candidate for being lost exists at a first time point, the lost child detection unit 134 detects the lost child from among the candidate for being lost based on the result of comparing the companion of the candidate for being lost at the first time point with the companion of the candidate for being lost at a second time point that is earlier than the first time point.

[0027] According to this information processing system 100, it is possible to ensure the safety of lost children.

[0028] FIG. 2 is a diagram showing an overview of the information processing apparatus 103 according to the first embodiment.

[0029] The information processing device 103 includes an analysis result acquisition unit 131, a candidate detection unit 132, and a lost child detection unit 134.

[0030] The analysis result acquisition unit 131 acquires the analysis results of the videos captured by the multiple image capture devices 101 .

[0031] The candidate detection unit 132 detects lost child candidates from people captured in the video using the person attributes and candidate conditions included in the analysis results.

[0032] When a companion of a candidate for being lost exists at a first time point, the lost child detection unit 134 detects the lost child from among the candidate for being lost based on the result of comparing the companion of the candidate for being lost at the first time point with the companion of the candidate for being lost at a second time point that is earlier than the first time point.

[0033] This information processing device 103 makes it possible to ensure the safety of lost children.

[0034] FIG. 3 is a flowchart showing an outline of information processing according to the first embodiment.

[0035] The analysis result acquisition unit 131 acquires the analysis results of the videos captured by the multiple image capture devices 101 (step S301).

[0036] The candidate detection unit 132 detects lost child candidates from people captured in the video using the person attributes and candidate conditions included in the analysis results (step S302).

[0037] If there is a companion of the candidate lost child at a first time point, the lost child detection unit 134 detects the lost child from among the candidate lost children based on the results of comparing the companion of the candidate lost child at the first time point with the companion of the candidate lost child at a second time point that is earlier than the first time point (step S304).

[0038] This information processing makes it possible to ensure the safety of lost children.

[0039] A detailed example of the information processing system 100 according to the first embodiment will be described below.

[0040] (Details) (Configuration Example of Information Processing System 100) FIG. 4 is a diagram showing a configuration example of the information processing system 100. As shown in FIG.

[0041] The information processing system 100 is a system for detecting an abducted lost child. An abducted lost child is a person who has been abducted by a third party. The third party may be, for example, a person other than the guardian of the lost child. The lost child is not limited to a child, but may also be, for example, an elderly person.

[0042] In this embodiment, the target area in which the information processing system 100 detects a lost child is a shopping mall. Note that the target area may be any area that has been determined in advance as appropriate, and may be, for example, various facilities or landmarks, all or part of a building, or a predetermined area on a public road.

[0043] The information processing system 100 includes first to M-th imaging devices 101_1 to 101_M1, an analysis device 102, an information processing device 103, and first to N-th terminals 104_1 to 104_M2.

[0044] M1 is an integer equal to or greater than 2. M2 is an integer equal to or greater than 1. Note that M1 may be 1.

[0045] The first to Mth image capture devices 101_1 to 101_M1 may each have the same configuration. Therefore, hereinafter, any one of the first to Mth image capture devices 101_1 to 101_M1 will also be referred to as the "image capture device 101."

[0046] Furthermore, each of the terminals 104_1 to 104_M2 may have the same configuration. Therefore, hereinafter, any one of the terminals 104_1 to 104_M2 will also be referred to as a "terminal 104."

[0047] Each of the multiple imaging devices 101, the analysis device 102, the information processing device 103, and each of the one or more terminals 104 are connected to each other via a communication network, and can send and receive information to and from each other via the communication network.

[0048] (Example of functional configuration of the image capturing device 101) The image capturing device 101 captures an image of a predetermined image capturing area and generates an image. The image is composed of, for example, a time-series of frame images showing the image capturing area. The image capturing device 101 transmits the image to the analysis device 102. The image capturing area is a part or all of a target area.

[0049] The image capturing area is determined in advance for each of the first to Mth image capturing devices 101_1 to 101_M1. Therefore, the information processing system 100 has a plurality of image capturing areas.

[0050] The multiple imaging areas may be different areas of the target area. The multiple imaging areas are, for example, areas that do not overlap with each other. Note that the multiple imaging areas may be areas where a part or all of a target area overlaps with a part or all of another target area. When the entire imaging areas overlap with each other, these imaging areas may be captured by imaging devices 101 with different imaging capabilities, such as resolution and lens performance.

[0051] (Example of Functional Configuration of Analysis Device 102) The analysis device 102 analyzes the images captured by the multiple image capturing devices 101 and generates an analysis result. The analysis device 102 transmits the generated analysis result to the information processing device 103.

[0052] The analysis results include at least the person attributes of the people included in the video. The person attributes are attributes of a person. The person attributes may include, for example, one or more of age (including age group), clothing, location, movement direction, movement speed, height, and gender. Note that the person attributes are not limited to those exemplified here, and detailed examples of person attributes will be described later.

[0053] (Example of Functional Configuration of Information Processing Device 103) The information processing device 103 uses the analysis results from the analysis device 102 to detect an abducted lost child.

[0054] 5 is a diagram illustrating an example of the functional configuration of the information processing device 103 according to embodiment 1. The information processing device 103 includes an analysis result acquisition unit 131, a candidate detection unit 132, a grouping unit 133, a lost child detection unit 134, a display control unit 135, a display unit 136, and a notification unit 137.

[0055] The analysis result acquisition unit 131 acquires, from the analysis device 102, analysis results of the videos captured by the multiple imaging devices 101. The analysis result acquisition unit 131 may acquire, along with the analysis results, the frame images and / or videos that were the basis for generating the analysis results from the analysis device 102.

[0056] Here, "A and / or B" means both A and B, or either A or B, and the same applies hereinafter.

[0057] The candidate detection unit 132 detects lost child candidates from people captured in the video using the person attributes and candidate conditions included in the analysis results acquired by the analysis result acquisition unit 131.

[0058] The candidate conditions are conditions related to a candidate for a lost child, and are set in advance by, for example, a user. The candidate conditions may be set by setting attributes of a person who is likely to become lost. In detail, for example, the candidate conditions may include one or more conditions related to age, such as age 10 or younger, age 80 or older, etc.

[0059] The grouping unit 133 identifies a group to which a person in the video belongs, using the person attributes included in the analysis result acquired by the analysis result acquisition unit 131 and predetermined grouping conditions.

[0060] The grouping conditions are conditions for grouping people appearing in the video using the person attributes included in the analysis results.

[0061] In detail, for example, the grouping conditions include one or more of the following: the people are within a predetermined distance from each other; the difference in the direction of movement of the people is within a predetermined range; the difference in the speed of movement of the people is within a predetermined range; and the people are talking.

[0062] If a companion of the lost child candidate exists at the first time point, the lost child detection unit 134 detects the lost child from among the lost child candidates based on a result of comparing the companions of the lost child candidate at the first time point and the second time point. Then, the lost child detection unit 134 generates lost child information regarding the detected lost child.

[0063] The second time point is a time point that is earlier than the first time point.

[0064] The lost child information is information about a lost child, and includes, for example, one or more of the following: one or more personal attributes of the lost child; an image of the lost child; the position of the lost child at a first time point and a second time point; and frame images and videos including the lost child at the first time point and the second time point.

[0065] 6 is a diagram illustrating an example of the functional configuration of the lost child detection unit 134 according to embodiment 1. The lost child detection unit 134 includes a determination unit 134a, a risk level identification unit 134b, a lost child identification unit 134c, and a lost child information generation unit 134d.

[0066] The determination unit 134a determines whether or not the candidate lost child has a companion at the first time point.

[0067] The risk identification unit 134b identifies a risk according to the location of the lost child candidate at the first time point.

[0068] In detail, for example, the risk identification unit 134b identifies the risk of the lost child candidate at the first time point based on the position of the lost child candidate at the first time point and the risk information by location.

[0069] The location-specific risk level information is information that associates the attributes of each location within the target area with the risk level, and is preferably set in advance.

[0070] When it is determined that a lost child candidate has a companion at the first time point, the lost child identification unit 134c detects the lost child from among the lost child candidates based on the results of comparing the companions of the lost child candidate at the first time point and the second time point.

[0071] The "result of comparing the accompanying persons" may be, for example, information indicating whether the accompanying person has changed. That is, for example, when the lost child candidate has an accompanying person at the first time point, the lost child detection unit 134 may detect the lost child from among the lost child candidates based on whether the accompanying person of the lost child candidate has changed between the first time point and the second time point.

[0072] Furthermore, whether or not the accompanying person has changed may be determined by whether or not all of the accompanying people of the candidate for lost child at the first time point have changed since the second time point (i.e., whether or not the candidate for lost child is only accompanied by different people than at the second time point).

[0073] Generally, for example, a child may be accompanied by a guardian at the second time point and meet up with another guardian or an acquaintance of the guardian at the first time point. By determining whether or not all of the accompanying persons of the candidate lost child at the first time point have changed since the second time point, it is possible to prevent a candidate lost child in such a situation from being detected as an abducted child. This makes it possible to detect a lost child who is likely to have been abducted, thereby ensuring the safety of the lost child.

[0074] The determination of whether or not the accompanying person has changed may be based on whether or not at least some of the accompanying people have changed at the first time point since the second time point. This allows the lost child candidate in the above situation to be detected as an abducted child. Even in the above situation, the lost child candidate may be an abducted child, so the safety of the lost child can be ensured.

[0075] The method for determining whether or not a companion of the lost child candidate exists at the first time point may also be various. In this embodiment, an example will be described in which the group identified by the grouping unit 133 is used to determine whether or not a companion of the lost child candidate exists at the first time point.

[0076] That is, the determination unit 134a according to this embodiment determines whether or not the lost child candidate has a companion at the first time point, using the group to which the lost child candidate belongs to the first time point.

[0077] In addition, various methods may be used to compare the accompanying person at the first time point and the accompanying person at the second time point. In this embodiment, an example will be described in which the lost child detection unit 134 detects a lost child from among lost child candidates by using the group identified by the grouping unit 133.

[0078] That is, when a companion of a lost child candidate exists at the first time point, the lost child detection unit 134 according to this embodiment compares the companions of the lost child candidate at the first time point and the second time point using the groups to which the lost child candidate belongs at the first time point and the second time point. Then, the lost child detection unit 134 detects the lost child from among the lost child candidates based on the result of the comparison.

[0079] In more detail, for example, when it is determined that a companion of the lost child candidate exists at the first time point, the lost child identification unit 134c compares the lost child candidate with people who belong to the same group as the lost child candidate at each of the first and second time points. Then, the lost child identification unit 134c detects the lost child from among the lost child candidates based on the results of the comparison. Here, the person who belongs to the same group as the lost child candidate corresponds to the companion.

[0080] Furthermore, in this embodiment, an example will be described in which the risk level of a lost child candidate at a first time point is referenced in order to detect a lost child from lost child candidates.

[0081] In other words, the lost child detection unit 134 (more specifically, the lost child identification unit 134c) in this embodiment detects a lost child from among the lost child candidates based on the results of the above comparison and the degree of danger according to the lost child candidate's location at the first time point, when the lost child candidate is accompanied by someone at the first time point.

[0082] In addition, in order to detect a lost child from lost child candidates, the risk level of the lost child candidate at the first time point does not need to be referenced.

[0083] The lost child information generating unit 134d generates lost child information regarding the lost child detected by the lost child identifying unit 134c.

[0084] In detail, for example, the lost child information generator 134d may generate lost child information that includes some or all of the analysis results related to the lost child among the analysis results acquired by the analysis result acquisition unit 131. The lost child information generator 134d may generate lost child information that further includes a frame image and / or video. The frame image and / or video may be one that shows the lost child, or may be the original image that was used to generate the analysis results included in the lost child information. The lost child information generator 134d may generate lost child information that further includes the level of risk identified for the lost child included in the lost child information.

[0085] Referring again to Fig. 5, the display control unit 135 displays various types of information on the display unit 136. The display unit 136 is a display configured, for example, by a liquid crystal panel or an organic EL (Electro-Luminescence) panel, which will be described later.

[0086] The display control unit 135 may, for example, cause the display unit 136 to display the lost child information generated by the lost child detection unit 134 (more specifically, the lost child information generation unit 134d).

[0087] For example, the display control unit 135 may cause the display unit 136 to display an image and / or video in which the position of the lost child at a first time point is superimposed on at least one of a frame image and a video including the lost child at the first time point. For example, the display control unit 135 may cause the display unit 136 to display an image and / or video in which the position of the lost child at a second time point is superimposed on at least one of a frame image and a video including the lost child at the second time point.

[0088] For example, when the lost child detection unit 134 detects multiple lost children, the display control unit 135 may cause the display unit 136 to display information about the multiple lost children in order of the level of danger at the first time point.

[0089] The display control unit 135 and the display unit 136 are examples of a display control means and a display means, respectively.

[0090] The notification unit 137 transmits the lost child information generated by the lost child detection unit 134 (more specifically, the lost child information generation unit 134d) to one or more terminals 104, respectively.

[0091] (Example of functional configuration of terminal 104) The terminal 104 is a device for displaying information about a lost child. The terminal 104 is carried by a predetermined person, such as a person concerned in the target area. Examples of a person concerned in the target area include an employee or a security guard in the target area.

[0092] 7 is a diagram illustrating an example of the functional configuration of the terminal 104 according to embodiment 1. The terminal 104 includes a lost child information acquisition unit 141, a display control unit 142, and a display unit 143.

[0093] The lost child information acquisition unit 141 acquires lost child information from the information processing device 103 .

[0094] The display control unit 142 displays various types of information on the display unit 143. The display unit 143 is a display configured, for example, by a liquid crystal panel or an organic EL (Electro-Luminescence) panel, which will be described later.

[0095] The display control unit 142 causes the display unit 143 to display the lost child information acquired by the lost child information acquisition unit 141, for example.

[0096] The display control unit 142 and the display unit 143 are other examples of a display control means and a display means, respectively.

[0097] (Example of physical configuration of information processing system 100) The information processing system 100 physically includes, for example, first to Mth imaging devices 101_1 to 101_M1, an analysis device 102, an information processing device 103, and first to Nth terminals 104_1 to 104_M2.

[0098] The first to M-th image capturing devices 101_1 to 101_M1 may each have the same physical configuration, and the first to N-th terminals 104_1 to 104_M2 may each have the same physical configuration.

[0099] The physical configuration of the information processing system 100 is not limited to this. For example, the functions of the multiple imaging devices 101, analysis device 102, and information processing device 103 described in this embodiment may be physically provided in a single device, or may be divided and provided in multiple devices in a manner different from that of this embodiment. When the function of transmitting or receiving information via a network N between the devices 101 to 104 according to this embodiment is physically incorporated into a common device, information may be transmitted or acquired via an internal bus or the like instead of the network N.

[0100] 8 is a diagram showing an example of the physical configuration of the image capturing apparatus 101 according to embodiment 1. The image capturing apparatus 101 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, a user interface 1060, and a camera 1070.

[0101] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, user interface 1050, network interface 1060, camera 1070, and microphone 1080. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0102] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0103] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0104] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing each function of the imaging device 101. The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module.

[0105] The network interface 1050 is an interface for connecting the image capturing apparatus 101 to the network N.

[0106] The user interface 1060 includes a touch panel, keyboard, mouse, etc. as an interface for the user to input information, and a liquid crystal panel, organic EL (Electro-Luminescence) panel, etc. as an interface for presenting information to the user.

[0107] The camera 1070 includes an imaging element, an optical system such as a lens, and the like, and captures an image of the imaging area under the control of the processor 1020 .

[0108] The imaging device 101 may accept input from a user and present information to the user via an external device (e.g., the analysis device 102, the information processing device 103, etc.) connected to the network N. In this case, the imaging device 101 does not need to include the user interface 1050.

[0109] 9 is a diagram showing an example of the physical configuration of the analysis device 102 according to embodiment 1. The analysis device 102 physically includes, for example, a bus 1010, a processor 1020, a memory 1030, a storage device 1040, and a network interface 1050 similar to those of the imaging device 101. The analysis device 102 further physically includes, for example, an input interface 2060 and an output interface 2070.

[0110] However, the storage device 1040 of the analysis device 102 stores program modules for realizing each function of the analysis device 102. Furthermore, the network interface 1050 of the analysis device 102 is an interface for connecting the analysis device 102 to the network N.

[0111] The input interface 2060 is an interface for the user to input information, and includes, for example, a touch panel, a keyboard, a mouse, etc. The output interface 2070 is an interface for presenting information to the user, and includes, for example, a liquid crystal panel, an organic EL panel, etc.

[0112] The information processing device 103 and the terminal 104 according to the first embodiment may each be physically configured in the same manner as, for example, the analysis device 102. However, the storage devices 1040 of the information processing device 103 and the terminal 104 store program modules for realizing their respective functions. Furthermore, the network interfaces 1050 of the information processing device 103 and the terminal 104 are interfaces for connecting each to the network N.

[0113] So far, an example of the configuration of the information processing system 100 according to the first embodiment has been described. Next, an example of the operation of the information processing system 100 according to the first embodiment will be described.

[0114] (Example of Operation of Information Processing System 100) The information processing system 100 according to this embodiment executes information processing for detecting an abducted lost child. The information processing includes, for example, a photographing process, an analysis process, a lost child detection process, and a display process.

[0115] (Example of Image Capturing Process According to First Embodiment) Fig. 10 is a flowchart showing an example of image capturing process according to the first embodiment. The image capturing process is a process for capturing an image of a target area. For example, when the image capturing device 101 receives a user's instruction to start the image capturing process from the information processing device 103 via the network N, the image capturing device 101 repeatedly executes the image capturing process at a predetermined frame rate until it receives a user's instruction to end the image capturing process. Note that the method for starting or ending the image capturing process is not limited to these.

[0116] The frame rate may be set appropriately, for example, 1 / 30 seconds, 1 / 60 seconds, or the like.

[0117] The photographing device 101 photographs a photographing area and generates a frame image showing the photographing area (step S101).

[0118] Fig. 11 is a diagram showing an example of a floor map of a target area. The target area shown in Fig. 11 includes two floors, and Fig. 11(a) is a diagram showing a floor map of the first floor of the target area. Fig. 11(b) is a diagram showing a floor map of the second floor of the target area. In Fig. 11, areas surrounded by dotted circles indicate the imaging areas of each of the imaging devices 101. In the example of Fig. 11, there are 18 imaging areas, so M1 is 18, i.e., the information processing system 100 is equipped with 18 imaging devices 101.

[0119] Note that one imaging device 101 may be configured to be able to image a plurality of imaging areas.

[0120] Referring again to Fig. 10, the image capturing apparatus 101 generates frame information including the frame image generated in step S101 (step S102).

[0121] 12 is a diagram showing an example of frame information. The frame information is, for example, information in which a frame image is associated with a frame ID (Identification), a photography ID, and a photography time.

[0122] The frame ID is information for identifying the frame ID. The shooting ID is information for identifying the shooting device 101. The shooting time is information indicating the time when the image was shot. The shooting time is composed of, for example, the year, month, date, and time. The time may be expressed in predetermined increments such as 1 / 10 seconds or 1 / 100 seconds.

[0123] FIG. 12 shows that a frame image FP1 with a frame ID "P1" was photographed at a photographing time "T1" by a photographing device 101 with a photographing ID "CM1".

[0124] The configuration of the frame information is not limited to this.

[0125] Referring again to Fig. 10, the imaging device 101 transmits the frame information generated in step S102 to the analysis device 102 (step S103), and ends the imaging process.

[0126] By repeatedly performing this type of imaging process by each imaging device 101, it is possible to generate an image of the target area and transmit it to the analysis device 102. The imaging process may be performed in real time.

[0127] (Example of analysis processing according to embodiment 1) Fig. 13 is a flowchart showing an example of analysis processing according to embodiment 1. The analysis processing is processing for analyzing video captured by the imaging device 101. For example, when the analysis device 102 receives a user's instruction to start the analysis processing from the information processing device 103 via the network N, the analysis device 102 repeatedly executes the analysis processing until it receives a user's instruction to end the analysis processing. Note that the method for starting or ending the analysis processing is not limited to these.

[0128] The analysis apparatus 102 acquires the frame information transmitted in step S103 from the imaging apparatus 101 (step S201).

[0129] The analysis device 102 stores the frame information acquired in step S201, and analyzes the frame images included in the frame information (step S202).

[0130] In this analysis, the analysis device 102 may refer to one or more of frame images captured at the same time by other image capture devices 101, past frame images and / or analysis results, as appropriate.

[0131] Here, the other image capture devices 101 are image capture devices 101 different from the image capture device 101 that generated the frame image to be analyzed. Furthermore, the past frame images and / or analysis results are frame images and / or analysis results of the frame images generated by each of the multiple image capture devices 101 before the frame image to be analyzed.

[0132] In more detail, for example, the analysis device 102 has one or more analysis functions for analyzing video, such as (1) an object detection function, (2) a face analysis function, (3) a human figure analysis function, (4) a posture analysis function, (5) a behavior analysis function, (6) an appearance attribute analysis function, (7) a gradient feature analysis function, (8) a color feature analysis function, and (9) a movement line analysis function.

[0133] (1) The object detection function detects an object from a frame image. The object detection function can also determine the position of an object within the frame image. For example, a technology such as YOLO (You Only Look Once) can be applied to the object detection function. Here, "object" includes people and objects, and the same applies hereinafter.

[0134] That is, the object detection function detects people and objects in the shooting area that appear in the frame image, and also finds the positions of people and objects, for example.

[0135] (2) The face analysis function detects human faces from frame images, extracts the features of the detected faces (facial feature amounts), and classifies the detected faces (classification). The face analysis function can also determine the position of the face within the image. The face analysis function can also determine the identity of people detected from different images based on the similarity between the facial feature amounts of people detected from different frame images.

[0136] (3) The human form analysis function extracts the physical characteristics of people included in the frame image (for example, values ​​indicating overall characteristics such as whether they are fat or thin, their height, and their clothing), and classifies (classifies) people included in the frame image. The human form analysis function can also identify the position of a person within an image. The human form analysis function can also determine the identity of people included in different images based on the physical characteristics of the people included in different images.

[0137] (4) The posture analysis function detects the joint points of people from the image and creates a stick figure model by connecting the joint points. The posture analysis function then uses the information from the stick figure model to estimate the posture of the people, extract the feature values ​​of the estimated posture (posture feature values), and classify (classify) the people included in the image. The posture analysis function can also determine the identity of people included in different images based on the posture feature values ​​of the people included in different images.

[0138] For example, the posture analysis function estimates postures such as standing, crouching, and bending from images, and extracts posture features that indicate each posture.

[0139] For example, the techniques disclosed in Patent Document 2 and Non-Patent Document 1 can be applied to the posture analysis function.

[0140] (5) The behavior analysis function can estimate human movements using information on stick figure models, changes in posture, and the like, extract features of human movements (movement features), and classify (classify) people included in images. The behavior analysis process can also estimate a person's height and identify the person's position in the image using information on stick figure models. The behavior analysis process can estimate behaviors such as changes or transitions in posture, movement (changes or transitions in position), movement speed, and movement direction from images, and extract movement features of the behavior.

[0141] (6) The appearance attribute analysis function can recognize appearance attributes associated with people. The appearance attribute analysis function extracts features (appearance attribute features) related to the recognized appearance attributes and classifies (classifies) people included in the image. Appearance attributes are attributes related to appearance, and include, for example, one or more of age (including age group), gender, clothing color, hairstyle, whether or not accessories are worn, and the color of the accessories if accessories are worn. Clothing includes one or more of clothing, shoes, etc. Accessories include one or more of hats, ties, glasses, necklaces, rings, etc.

[0142] (7) The gradient feature analysis function extracts gradient feature quantities (gradient feature quantities) in the frame images. For the gradient feature detection function, technologies such as SIFT, SURF, RIFF, ORB, BRISK, CARD, and HOG can be applied.

[0143] (8) The color feature analysis function can detect objects from frame images, extract color features of the detected objects, and classify the detected objects.

[0144] The color feature amount may be, for example, a color histogram. The color feature analysis function may, for example, detect people and objects included in the frame image. Also, for example, the color feature analysis function may classify items into predetermined classes.

[0145] (9) The flow line analysis function can determine the flow line (trajectory of movement) of a person included in a video, for example, using the result of the identity determination in any of the above-mentioned analysis functions (2) to (6). In more detail, for example, by connecting people determined to be the same between different frame images in a time series, the flow line of the person can be determined. Note that the flow line analysis function can also determine the flow line across multiple videos captured in different shooting areas.

[0146] The person attributes include, for example, at least one of elements contained in the person detection results of the object detection function, facial features, human body features, posture features, movement features, appearance attribute features, gradient features, color features, movement line, movement speed, movement direction, etc.

[0147] Each of the analysis functions (1) to (9) may appropriately use the results of analysis performed by other analysis functions.

[0148] The analysis device 102 analyzes the video including the frame images using one or more of these analysis functions and generates a detection result including person attributes. The detection result may associate each person appearing in the frame images with the person attributes.

[0149] The analysis device 102 generates analysis information by associating the analysis result of step S202 with the frame information acquired in step S201 (step S203).

[0150] The frame information acquired in step S201 is frame information including the frame image that was the basis for generating the analysis result (i.e., the frame image that was the subject of analysis in step S202).

[0151] The analysis device 102 transmits the analysis information generated in step S203 to the information processing device 103 (step S204).

[0152] This analysis process may be repeatedly performed for each of the multiple frame images generated by each of the multiple image capturing devices 101. This allows the captured video of the target area to be analyzed, and the analysis results generated by the analysis to be transmitted to the information processing device 103.

[0153] The analysis device 102 may analyze a portion of the time-series frame images generated by each of the multiple image capture devices 101, for example, by performing analysis processing on frame images at a predetermined time interval. This time interval may be set to a length of time, such as one second, that does not affect the detection of a lost child. This allows the analysis device 102 to reduce the number of frame images that are analyzed while preventing a decrease in the accuracy of detecting a lost child, compared to when all of the time-series frame images are analyzed. This allows the processing load on the analysis device 102 to be reduced while preventing a decrease in the accuracy of detecting a lost child.

[0154] Furthermore, the analysis method performed by the analysis device 102 is not limited to the one described here and may be changed as appropriate. For example, the analysis functions provided in the analysis device 102 may be changed as appropriate.

[0155] 14 is a flowchart showing an example of a lost child detection process according to embodiment 1. The lost child detection process is a process for detecting an abducted lost child using an analysis result generated by executing an analysis process.

[0156] For example, when the information processing device 103 receives a start instruction from the user, it transmits the start instruction to the imaging device 101 and the analysis device 102 and starts the lost child detection process. Then, for example, when the information processing device 103 receives an end instruction from the user, it transmits an end instruction to the imaging device 101 and the analysis device 102 and ends the lost child detection process. In other words, for example, when the information processing device 103 receives a start instruction from the user, it repeatedly executes the lost child detection process until it receives an end instruction from the user. Note that the method for starting or ending the lost child detection process is not limited to these.

[0157] The analysis result acquisition unit 131 acquires the analysis information transmitted in step S204 from the information processing device 103 (step S301), thereby acquiring the analysis results and frame images from the analysis device 102.

[0158] The candidate detection unit 132 detects lost child candidates from the people included in the analysis result obtained in step S301 using the person attributes and candidate conditions included in the analysis result (step S302).

[0159] In detail, for example, the candidate detection unit 132 detects, as a lost child candidate, a person associated with a person attribute that satisfies a candidate condition among the person attributes of each person included in the analysis result acquired in step S301. If the candidate condition is, for example, 10 years old or younger, the candidate detection unit 132 detects, as a lost child candidate, a person associated with a person attribute that includes an age of 10 years old or younger.

[0160] The grouping unit 133 uses the person attributes included in the analysis results obtained in step S301 and predetermined grouping conditions to identify the group to which the person in the frame image obtained in step S301 belongs (step S303).

[0161] In more detail, for example, the grouping unit 133 detects and groups multiple people included in the analysis result acquired in step S301 who are associated with personal attributes that mutually satisfy the grouping condition. As a result, the grouping unit 133 identifies a group to which multiple people who mutually satisfy the grouping condition belong. This group is made up of multiple people who accompany each other.

[0162] Furthermore, for example, if there is no person associated with a personal attribute that satisfies the grouping condition among the people included in the analysis result acquired in step S301, the grouping unit 133 groups only the person. In this way, the grouping unit 133 identifies a group to which a person who does not have any other person that satisfies the grouping condition belongs. This group is made up of one person who acts alone.

[0163] The grouping unit 133 may store, for example, the results of grouping in step 303, that is, the people in the frame images and the groups to which each person belongs.

[0164] If the candidate for lost child has a companion at the first time point, the lost child detection unit 134 detects the lost child from among the candidate for lost child detected in step S302 based on the results of comparing the companions of the candidate for lost child at the first time point and the second time point (step S304).

[0165] 15 is a flowchart showing an example of the detection process (step S304) according to embodiment 1. When a plurality of lost child candidates are detected in step S302, the lost child detection unit 134 may execute the detection process (step S304) for each of the lost child candidates.

[0166] The determination unit 134a determines whether or not the potential lost child has a companion at the first time point (step S304a).

[0167] In detail, for example, the first time point is the present. In this case, for the lost child candidate detected in step S302, the determination unit 134a determines whether or not the group identified in step S303 includes any person other than the lost child candidate. In this way, the determination unit 134a determines whether or not there is any other person (i.e., a companion) who belongs to the same group as the lost child candidate at the first time point.

[0168] If it is determined that there is no accompanying person (step S304a; No), the determination unit 134a ends the lost child detection process.

[0169] If it is determined that a companion is present (step S304a; Yes), the risk level identification unit 134b identifies the risk level according to the location at the first point in time of the lost child candidate who is determined to have a companion (step S304b).

[0170] In more detail, for example, the risk level identification unit 134b acquires the location at the first time point of the lost child candidate determined to have a companion at step S304a based on the analysis result acquired at step S301. The risk level identification unit 134b identifies the risk level according to the location of the lost child candidate at the first time point based on the location-specific risk level information.

[0171] As described above, the location-specific risk information is information that associates the attributes of each location in the target area with the risk level. The risk level is an index that indicates the degree of risk of getting lost.

[0172] The attribute for each location is, for example, at least one of parking lot, store, childcare corner, etc. In this case, the risk level information for each location includes, for example, risk levels of "high," "medium," and "low" associated with parking lots, stores, and childcare corners, respectively. That is, parking lots are often less popular, so a risk level of "high" is associated with them. Stores are more popular than parking lots, so a risk level of "medium" is associated with them. Childcare corners are likely to be safe, so a risk level of "low" is associated with them.

[0173] It should be noted that the location-specific risk information is not limited to this.

[0174] The risk level identifying unit 134b acquires the attributes of the location to which the position of the lost child candidate at the first time point belongs, for example, based on the layout information.

[0175] The layout information is information that indicates the layout of the target area (i.e., the location where the multiple image capture devices 101 will capture images). The layout information may include, for example, a floor map as a layout. The layout information may include at least one of the range of aisles in the target area, the location of specific sections such as each store, the range of specific sections such as each store, the location of escalators, and the location of elevators.

[0176] The risk level identification unit 134b then acquires the risk level associated with the attribute of the acquired location from the location-specific risk level information, thereby identifying the risk level according to the location at the first time point of the lost child candidate determined to have a companion.

[0177] The lost child identification unit 134c determines whether the degree of risk identified in step S304b is equal to or greater than a threshold (step S304c). The threshold may be determined in advance.

[0178] In more detail, for example, the threshold value is assumed to be "medium." When the location-specific risk information is as described above, the lost child identifying unit 134c determines that the risk level of a lost child candidate who is in the "parking lot" or "store" at the first time point is equal to or greater than the threshold value. Also, the lost child identifying unit 134c determines that the risk level of a lost child candidate who is in the "childcare corner" at the first time point is not equal to or greater than the threshold value.

[0179] If it is determined that the risk level is not equal to or greater than the threshold (step S304c; No), the lost child identification unit 134c ends the lost child detection process. As a result, the lost child candidate who is in a low-risk, i.e., safe, location will no longer be detected as a lost child.

[0180] If it is determined that the risk level is equal to or greater than the threshold (step S304c; Yes), the lost child identification unit 134c compares the lost child candidate with people who belong to the same group as the lost child candidate at each of the first and second time points (step S304d).

[0181] For example, the second time point is the time when the person enters a shopping mall (the target area) (entering the store). The first time point is the present, as described above. In this case, the lost child identification unit 134c compares people who belong to the same group as the lost child candidate at the time of store entry with people who belong to the present.

[0182] FIG. 16 is a diagram for explaining the process of comparing accompanying persons at the first and second time points (step S304d).

[0183] For example, assume that a lost child candidate LC is captured in the current frame image FPA_T1 acquired in step S301. This lost child candidate LC is accompanied by a companion and is at a risk level of "medium" or higher.

[0184] The lost child identification unit 134c may refer to the groups of people shown in the frame images acquired in step S301 and acquire personal attributes of people who belong to the same group as the lost child candidate LC. This allows the lost child identification unit 134c to acquire personal attributes of people currently accompanying the lost child candidate LC.

[0185] The lost child identifying unit 134c looks back a predetermined time interval ΔT from the present to the past and identifies frame images in which the lost child candidate LC appears, based on the person attributes obtained by analyzing each frame image.

[0186] For example, when searching for a frame image FPA_T1-ΔT that shows the lost child candidate LC from multiple frame images captured at time T1-ΔT, the lost child identification unit 134c may search in order, starting with frame images whose capture areas are close to (for example, adjacent to) the frame image capturing the lost child candidate LC at time T. Fig. 16 shows an example in which the search range for identifying the frame image FPA_T1-ΔT that shows the lost child candidate LC is three frame images.

[0187] By performing such a search going back in time at predetermined time intervals ΔT, the lost child identification unit 134c identifies the frame image in which the lost child candidate LC first appears, that is, the frame image FPA_T2 at the time of entering the store.

[0188] The grouping unit 133 may store the grouping results based on the analysis results of the frame image FPA_T2 at the time of entering the store, for example. Note that the grouping unit 133 may also identify the group to which each person belongs based on the analysis results of the frame image FPA_T2 at the time of entering the store.

[0189] The lost child identification unit 134c may refer to the group identified for the frame image FPA_T2 at the time of store entry and acquire personal attributes of a person who belongs to the same group as the lost child candidate LC at the time of store entry. This allows the lost child identification unit 134c to acquire personal attributes of a person accompanying the lost child candidate LC at the time of store entry.

[0190] The lost child identification unit 134c may compare the personal attributes of the accompanying persons of the lost child candidate LC at the current time and at the time of entering the store, for example. This makes it possible to compare the people who belong to the same group as the lost child candidate LC at the current time and at the time of entering the store.

[0191] Referring again to Fig. 15, the lost child identifying unit 134c determines whether or not a lost child has been detected from among the lost child candidates based on the result of the comparison in step S304d (step S304e).

[0192] In detail, for example, the lost child identification unit 134c determines whether there are one or more common companions at each time point, based on the personal attributes of the companions of the lost child candidate LC at each time point, now and when entering the store.

[0193] For example, if there is one or more common accompanying persons at each time point, the lost child identifying unit 134c determines that no lost child has been detected (i.e., there is no lost child).

[0194] Furthermore, for example, the lost child identification unit 134c determines that a lost child has been abducted when there is no common accompanying person at each time point. In other words, in this case, the lost child identification unit 134c detects a lost child from the lost child candidates.

[0195] If a lost child is not detected (step S304e; No), the lost child information generation unit 134d ends the lost child detection process. If a lost child is detected (step S304e; Yes), the lost child information generation unit 134d generates lost child information about the lost child (step S304f) and returns to the lost child detection process.

[0196] Referring again to Figure 14, the display control unit 135 causes the display unit 136 to display the lost child information generated in step S304f (step S305).

[0197] In detail, for example, if multiple lost children are detected in step S304e, the display control unit 135 causes the display unit 136 to display the lost child information generated in step S304f for the multiple lost children in the order of the risk level identified in step S304b.

[0198] The notification unit 137 transmits the lost child information generated in step S304f to each of one or more terminals 104 (step S306).

[0199] This lost child detection process may be repeatedly executed each time analysis information transmitted in the analysis process is acquired. This allows abducted lost children to be detected. Furthermore, information about the detected lost child may be displayed on the display unit 136, allowing the user to easily notice the abducted lost child.

[0200] 17 is a flowchart showing an example of the display process according to the first embodiment. The display process is a process for displaying the lost child information transmitted by executing the lost child detection process on the terminal 104. When there are multiple terminals 104, each of the terminals 104 may execute the display process.

[0201] For example, when the terminal 104 starts pre-installed software, the terminal 104 starts the display process. For example, the terminal 104 executes the display process while the software is running. Note that the method for starting or ending the display process is not limited to these.

[0202] The lost child information acquisition unit 141 acquires the lost child information transmitted in step S137 from the information processing device 103 (step S401).

[0203] The display control unit 142 causes the display unit 143 to display the lost child information acquired in step S401 (step S402), and ends the display process.

[0204] In detail, for example, when information about multiple lost children is acquired in step S401, the display control unit 142 displays the information about the lost children in order of the risk of each child being lost included in the information about the lost children on the display unit 143. For example, when the terminal 104 receives a predetermined operation for closing the display screen of the information about the lost children, the display control unit 142 may end the display process.

[0205] By executing such a display process, the person carrying the terminal 104 can quickly notice that the lost child has been abducted and go to rescue the lost child.

[0206] (Operations and Effects) As described above, according to the first embodiment, the information processing system 100 includes the analysis result acquisition unit 131, the candidate detection unit 132, and the lost child detection unit 134.

[0207] The analysis result acquisition unit 131 acquires analysis results of the videos captured by the multiple image capture devices 101. The candidate detection unit 132 detects a lost child candidate from people captured in the video using person attributes and candidate conditions included in the analysis results. If the lost child candidate has a companion at a first time point, the lost child detection unit 134 detects the lost child from the lost child candidates based on a result of comparing the companion of the lost child candidate at the first time point with the companion at a second time point that is earlier than the first time point.

[0208] In this way, a lost child is detected from among the lost child candidates who have a companion at the first time point. A lost child who has a companion at the first time point is likely to be an abducted child, and since such a lost child can be automatically detected, it is possible to quickly detect an abducted child and take measures such as rescuing the child. Therefore, it is possible to ensure the safety of lost children.

[0209] According to the first embodiment, the candidate conditions include a condition related to age.

[0210] This allows for the detection of lost children from age groups that are more likely to get lost, and therefore speeds up the detection of lost children and makes it possible to detect abducted children more quickly than when, for example, all people are considered lost children without setting age conditions, thereby ensuring the safety of lost children.

[0211] According to embodiment 1, when a companion of the candidate lost child is present at the first time point, the lost child detection unit 134 detects a lost child from among the candidate lost child based on whether the companion of the candidate lost child has changed between the first time point and the second time point.

[0212] This allows an abducted lost child to be automatically detected, so that the abducted lost child can be detected quickly and measures such as rescuing the child can be taken, thereby ensuring the safety of the lost child.

[0213] According to embodiment 1, when a person accompanying the lost child candidate is present at the first time point, the lost child detection unit 134 detects a lost child from among the lost child candidates based on the comparison results and the degree of danger according to the position of the lost child candidate at the first time point.

[0214] This makes it possible to detect lost children who are at high risk, thereby ensuring the safety of lost children.

[0215] According to the first embodiment, the information processing system 100 further includes a grouping unit 133 that identifies a group to which a person in the video belongs by using personal attributes included in the analysis result and grouping conditions for grouping people shown in the video. If the lost child candidate has a companion at the first time point, the lost child detection unit 134 compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first time point and the second time point, and detects the lost child from the lost child candidates based on the comparison result.

[0216] This allows for easy detection of potential lost children who are accompanied by others at the second time point by grouping people using personal attributes. Therefore, since abducted lost children can be detected automatically, abducted lost children can be detected quickly and measures such as rescuing the child can be taken. Therefore, it is possible to ensure the safety of lost children.

[0217] According to the first embodiment, the lost child detection unit 134 includes a determination unit 134a and a lost child identification unit 134c. The determination unit 134a determines whether the lost child candidate has a companion at the first time point, using the group to which the lost child candidate belongs at the first time point. If it is determined that the lost child candidate has a companion at the first time point, the lost child identification unit 134c compares the lost child candidate with people who belong to the same group as the lost child candidate at each of the first and second time points, and detects the lost child from among the lost child candidates based on the comparison results.

[0218] In this way, by grouping people using personal attributes, it is possible to easily detect potential lost children who are accompanied by others at the first time point. A lost child who is accompanied by others at the first time point is likely to have been abducted, and since such lost children can be detected automatically, it is possible to quickly detect abducted children and take measures such as rescuing them. Therefore, it is possible to ensure the safety of lost children.

[0219] According to the first embodiment, the lost child information includes at least one of an image of the detected lost child and a location of the child at a first time point.

[0220] This makes it easier to find the detected lost child and allows the lost child to be rescued quickly, thereby ensuring the safety of the lost child.

[0221] According to the first embodiment, when a plurality of lost children are detected, the display control unit 135 causes the display unit 136 to display information about the plurality of lost children in order of the degree of danger at the first time point.

[0222] This makes it easier to notice lost children who are at high risk, and therefore makes it possible to ensure the safety of lost children.

[0223] Second Embodiment Generally, a parent or guardian accompanying a lost child may visit a lost child center, management center, or the like to inquire about the lost child. In such cases, a person in the target area who responds to inquiries from parents or guardians may ask the parent or guardian about the characteristics of the lost child. In this embodiment, an example will be described in which an information processing system receives such characteristics of the lost child and further refers to the characteristic information to detect an abducted lost child.

[0224] In this embodiment, for the sake of simplicity, differences from the first embodiment will be mainly described.

[0225] The information processing system according to this embodiment includes an information processing device 203 instead of the information processing device 103 according to the first embodiment. Except for this point, the information processing system according to this embodiment may be configured similarly to the information processing system 100 according to the first embodiment.

[0226] 18 is a diagram showing an example of the functional configuration of an information processing device 203 according to embodiment 2. The information processing device 203 includes a candidate detection unit 232 and a grouping unit 233 instead of the candidate detection unit 132 and the grouping unit 133 according to embodiment 1. The information processing device 203 further includes a feature acquisition unit 251. Except for these components, the information processing device 203 according to this embodiment may be configured similarly to the information processing device 103 according to embodiment 1.

[0227] The characteristic acquisition unit 251 acquires characteristic information of the lost child to be detected based on input from a user who has learned the characteristics of the lost child verbally or otherwise. The characteristic acquisition unit 251 may further acquire characteristic information of a person (companion) who provided the characteristic information of the lost child based on input from the user. The characteristic information of the companion may include an image of the companion obtained by the user photographing the companion.

[0228] Similar to the candidate detection unit 132 according to the first embodiment, the candidate detection unit 232 detects lost child candidates from people captured in the video by using the person attributes and candidate conditions included in the analysis results acquired by the analysis result acquisition unit 131. The candidate conditions according to the present embodiment differ from the first embodiment in that they include feature information acquired by the feature acquisition unit 251.

[0229] The grouping unit 233, like the grouping unit 133 according to the first embodiment, identifies a group to which a person in the video belongs, using person attributes included in the analysis result and predetermined grouping conditions. The grouping unit 233 according to the present embodiment further identifies a group to which a person in the video belongs, using the feature information of the lost child acquired by the feature acquisition unit 251.

[0230] In more detail, for example, the grouping unit 233 may identify a group to which a person in the video belongs by using the characteristic information of the lost child and the characteristic information of the accompanying person. In this case, the grouping unit 233 identifies people whose personal attributes included in the analysis result are similar to the characteristic information of the lost child and the accompanying person as belonging to a common group.

[0231] Here, "similar" means that the similarity is sufficient to satisfy a predetermined condition, and more specifically, the similarity is equal to or greater than a threshold value. Note that the grouping unit 233 does not necessarily need to use a grouping condition.

[0232] The information processing system according to this embodiment may be physically configured in the same manner as the information processing system 100 according to the first embodiment.

[0233] (Operation of Information Processing System According to Embodiment 2) Information processing according to this embodiment includes the same photographing process, analysis process, and display process as in Embodiment 1, and a lost child detection process that differs from Embodiment 1. In this embodiment, the lost child detection process is also executed by the information processing device 203.

[0234] (Example of lost child detection processing according to embodiment 2) Fig. 19 is a flowchart showing an example of lost child detection processing according to embodiment 2. As shown in the figure, the lost child detection processing according to this embodiment includes step S501, which is executed following step S302 similar to embodiment 1, and steps S502 to S503, which replace steps S302 to S303 according to embodiment 1. Except for these, the lost child detection processing according to embodiment 2 may be configured similarly to the lost child detection processing according to embodiment 1.

[0235] The feature acquisition unit 251 acquires feature information based on a user input or the like (step S501).

[0236] In more detail, for example, the characteristic acquisition unit 251 acquires characteristic information of the lost child to be detected and characteristic information of the accompanying person of the lost child based on input from the user, etc. The accompanying person is a companion of the lost child to be detected, such as a guardian of the lost child.

[0237] The candidate detection unit 232 detects lost child candidates from the people included in the analysis results using the person attributes included in the analysis results obtained in step S301 and the candidate conditions including the lost child's characteristic information obtained in step S501 (step S502).

[0238] In detail, for example, the candidate detection unit 232 detects, as a lost child candidate, a person associated with a personal attribute that satisfies a candidate condition among the personal attributes of each person included in the analysis result acquired in step S301. The personal attribute that satisfies the candidate condition may be, for example, a personal attribute similar to the characteristic information included in the candidate condition.

[0239] The grouping unit 233 uses the person attributes, predetermined grouping conditions, and the feature information acquired in step S501 to identify the group to which the person in the frame image acquired in step S301 belongs (step S503).

[0240] The person attributes here are the person attributes included in the analysis results obtained in step S301. The characteristic information here is the characteristic information obtained in step S501, for example, the characteristic information of the lost child and the accompanying person.

[0241] In more detail, for example, the grouping unit 233 detects a plurality of people included in the analysis result acquired in step S301 who are associated with personal attributes that satisfy the grouping conditions. The grouping unit 233 further detects and groups, from among the detected plurality of people, people who are associated with personal attributes similar to the characteristic information of the lost child and the accompanying person.

[0242] By executing the lost child detection process according to this embodiment, it is possible to detect if a lost child has been abducted, using characteristic information of the lost child obtained orally or otherwise.

[0243] (Operations and Effects) As described above, according to the second embodiment, the information processing system 100 further includes the characteristic acquisition unit 251 that acquires characteristic information of the lost child to be detected. The candidate conditions include the characteristic information of the lost child.

[0244] This allows the system to detect a lost child when it has been abducted using the child's characteristic information. Since abducted children are generally likely to be in a dangerous situation, it is possible to quickly detect a lost child in such a dangerous situation. Therefore, it is possible to ensure the safety of lost children.

[0245] According to the second embodiment, the information processing system 100 further includes a feature acquisition unit 251 that acquires feature information of the lost child to be detected. The grouping unit 233 further identifies a group to which a person in the video belongs, using the feature information of the lost child.

[0246] This allows the group to which the lost child belongs to to be identified and the accompanying person of the lost child to be identified, so that it is possible to more reliably detect whether the lost child has an accompanying person. Therefore, if the lost child has been abducted, it can be reliably detected. Therefore, it is possible to ensure the safety of the lost child.

[0247] Third Embodiment In this embodiment, an example will be described in which the movement range of a lost child is predicted and the predicted movement range is used as a search range and lost child information. Note that the movement range may be used as only one of the search range and the lost child information.

[0248] In this embodiment, for the sake of simplicity, differences from the first embodiment will be mainly described.

[0249] The information processing system according to this embodiment includes an information processing device 303 instead of the information processing device 103 according to the first embodiment. Except for this point, the information processing system according to this embodiment may be configured similarly to the information processing system 100 according to the first embodiment.

[0250] 20 is a diagram showing an example of the functional configuration of an information processing device 303 according to embodiment 3. The information processing device 303 includes a lost child detection unit 334 and a display control unit 335 instead of the lost child detection unit 134 and the display control unit 135 according to embodiment 1. The information processing device 203 further includes a pattern detection unit 361 and a range prediction unit 362. Except for these, the information processing device 303 according to this embodiment may be configured similarly to the information processing device 103 according to embodiment 1.

[0251] The pattern detection unit 361 detects a movement pattern of a person captured in a video based on the person attributes between the first time point and the second time point.

[0252] The movement pattern is a tendency regarding a person's movement, and may include, for example, one or more of the average movement speed, the movement speed in front of a store, the time spent stopping in front of a store, the type of store where the person slows down or stops, the type of store visited, and the average movement speed within the store.

[0253] The person whose movement pattern is to be detected may be, for example, one or more of the detected lost child, the candidate lost child, the companion of the lost child, and the companion of the candidate lost child. Note that the person whose movement pattern is to be detected is not limited to these.

[0254] The range prediction unit 362 predicts the movement range of a person shown in the video using the person attributes. The range prediction unit 362 may predict the movement range of a person shown in the video using at least one of the person attributes, for example, the person's position, movement direction, and movement speed.

[0255] The range prediction unit 362 may predict, for example, the movement range of a person appearing in the video between the first time point and the second time point. In this case, for example, the range prediction unit 362 may predict the movement range of a person appearing in the video between the first time point and the second time point by using the movement pattern detected by the pattern detection unit 361 in addition to the person attributes.

[0256] The range prediction unit 362 may predict, for example, the range of movement of the person after a first time point. If the first time point is the present, the range of movement after the first time point is the range of future movement. In this case, for example, the range prediction unit 362 may predict the range of movement of the person using person attributes at the first time point (for example, at least one of the location, direction of movement, and speed of movement of the lost person).

[0257] Furthermore, the range prediction unit 362 may predict the movement range of the person by further using, for example, layout information. In this case, for example, the range prediction unit 362 may predict the movement range including the movement of the person between floors based on the positions of escalators and elevators included in the layout information and at least one of the position, movement direction, and movement speed of the person. The range prediction unit 362 may store the layout information in advance.

[0258] The person whose movement range is to be predicted may be, for example, one or more of the detected lost child, the candidate lost child, the companion of the lost child, and the companion of the candidate lost child. Note that the person whose movement pattern is to be detected is not limited to these.

[0259] The lost child detection unit 334 detects a lost child from among lost child candidates, similar to the lost child detection unit 134 according to the first embodiment, and generates lost child information about the detected lost child.

[0260] 21 is a diagram illustrating an example of the functional configuration of the lost child detection unit 334 according to embodiment 3. The lost child detection unit 334 includes a lost child identification unit 334c and a lost child information generation unit 334d instead of the lost child identification unit 134c and the lost child information generation unit 134d according to embodiment 1. Except for this point, the lost child detection unit 334 may be configured similarly to the lost child detection unit 134 according to embodiment 1.

[0261] Similar to the lost child identification unit 134c according to the first embodiment, the lost child identification unit 334c detects a lost child from among the lost child candidates based on the results of comparing the accompanying persons of the lost child candidates at the first and second time points.

[0262] The lost child identifying unit 334c according to this embodiment sets the movement range predicted for a person by the range predicting unit 362 as a search range for the person, and detects lost child candidates from people captured within the search range.

[0263] The lost child information generating unit 334d generates lost child information about the lost child detected by the lost child identifying unit 134c, similar to the lost child information generating unit 134d according to the first embodiment.

[0264] The lost child information according to this embodiment may include a movement range of the lost child predicted by the range prediction unit 362. In this case, for example, the lost child information may include a movement range after the first time point predicted by the range prediction unit 362 for the lost child.

[0265] 20 again. The display control unit 335, similar to the display control unit 135 according to the first embodiment, causes the display unit 136 to display various information. For example, the display control unit 335 may cause the display unit 136 to display lost child information generated by the lost child detection unit 134 (more specifically, the lost child information generation unit 134d).

[0266] The lost child information according to this embodiment may further include layout information. In this case, for example, the display control unit 335 may cause the display unit 136 to display an image in which the movement range predicted by the range prediction unit 362 for the lost child is superimposed on the layout information.

[0267] For example, the display control unit 335 may cause the display unit 136 to display an image in which the location of the lost child at a first time point is superimposed on layout information. For example, the display control unit 135 may cause the display unit 136 to display an image in which the location of the lost child at a second time point is superimposed on layout information.

[0268] The information processing system according to this embodiment may be physically configured in the same manner as the information processing system 100 according to the first embodiment.

[0269] (Operation of Information Processing System According to Embodiment 3) Information processing according to this embodiment includes the same photographing process, analysis process, and display process as in Embodiment 1, and a lost child detection process that differs from Embodiment 1. In this embodiment, the lost child detection process is also executed by the information processing device 303.

[0270] (Example of lost child detection processing according to embodiment 3) Fig. 22 is a flowchart showing an example of lost child detection processing according to embodiment 3. As shown in the figure, the lost child detection processing according to this embodiment includes steps S604 to S605 instead of steps S304 to S305 according to embodiment 1. Except for these steps, the lost child detection processing according to embodiment 3 may be configured similarly to the lost child detection processing according to embodiment 1.

[0271] The lost child detection unit 334 detects a lost child from among the lost child candidates detected in step S302 (step S604), similar to the lost child detection unit 134 according to the first embodiment. In this embodiment, the details of the detection process (step S604) are different from those of the detection process (step S304) according to the first embodiment.

[0272] 23 is a flowchart showing an example of the detection process (step S604) according to the third embodiment. The detection process (step S604) according to this embodiment includes steps S604d and S604f instead of steps S304d and S304f according to the first embodiment. The detection process (step S604) according to this embodiment further includes step S604g executed between steps S304e and S604f. Except for these, the detection process (step S604) according to this embodiment may be configured similarly to the detection process (step S304) according to the first embodiment.

[0273] As in the first embodiment, when the lost child identification unit 334c determines that the risk level is equal to or greater than the threshold (step S304c; Yes), it compares the lost child candidate with people who belong to the same group as the lost child candidate at each of the first and second time points (step S604d). In this embodiment, the details of the comparison process (step S604d) are different from those of the first embodiment (step S304d).

[0274] 24 and 25 are flowcharts showing an example of the comparison process (step S604d) according to the third embodiment.

[0275] The lost child identifying unit 334c sets the first time point T1 to the photographing time T (step S604d1). The first time point is, for example, the present, as in the first embodiment.

[0276] The lost child identifying unit 334c sets the frame image captured at the shooting time T, which is earlier by the time interval ΔT, as the search target (step S604d2).

[0277] For example, if the first time point T1 is set as the photographing time T, the lost child identifying unit 334c sets the frame image at the photographing time T1-ΔT as the search target.

[0278] The pattern detection unit 361 detects the movement pattern of the potential lost child based on the person attributes included in the analysis result (step S604d3).

[0279] For example, in step S604d3, in order to detect the movement pattern of a potential lost child, analysis results generated based on frame images captured from the first point in time to the capture time of the frame image being searched for are used.

[0280] The range prediction unit 362 predicts the movement range of the lost child candidate using the person attributes of the lost child candidate and the movement pattern detected in step S604d3 (step S604d4).

[0281] The lost child identifying unit 334c sets a search range to a part or all of the frame image to be searched, based on the movement range predicted in step S604d4 (step S604d5).

[0282] In more detail, for example, the lost child identifying unit 334c sets, as the search range, frame images that include the movement range predicted in step S604d4, among the frame images that are search targets.

[0283] The lost child identifying unit 334c determines whether a frame image showing a lost child candidate has been identified from the search range set in step S604d5 (step S604d6).

[0284] In more detail, for example, the lost child identification unit 334c searches for a frame image showing a lost child candidate from among the frame images in the search range. If a frame image showing a lost child candidate is detected, the lost child identification unit 334c determines that a frame image showing a lost child candidate has been identified. If a frame image showing a lost child candidate is not detected, the lost child identification unit 334c determines that a frame image showing a lost child candidate has not been identified.

[0285] If it is determined that a frame image showing a candidate lost child is not to be identified (step S604d6; No), the lost child identifying unit 334c returns to step S604d5. In the re-executed step S604d5, the lost child identifying unit 334c may set the search range to, for example, a frame image showing an area adjacent to the area shown in the search range set in the previous step S604d5.

[0286] The lost child identifying unit 334c determines whether the photographing time T is the second time point or not (step S604d7).

[0287] If it is determined that it is not the second time point (step S604d7; No), the lost child identifying unit 334c returns to step S604d2.

[0288] See Fig. 25. If it is determined that the time point is the second time point (step S604d7; Yes), the lost child identifying unit 334c identifies a person who belongs to the same group as the lost child candidate at the second time point (step S604d8).

[0289] In detail, for example, the lost child identification unit 334c identifies people who belong to the same group as the lost child candidate (i.e., accompanying persons of the lost child candidate) using the analysis results based on the frame image identified in S604d6 and the grouping conditions.

[0290] The lost child identification unit 334c determines whether all of the people determined to be accompanying the lost child candidate have changed between the first time point and the second time point (step S604d9).

[0291] In more detail, for example, the lost child identification unit 334c acquires the personal attributes of the person determined to be a companion of the lost child candidate at the first time point in step S304a. The lost child identification unit 334c acquires the personal attributes of the person determined to be a companion of the lost child candidate at the second time point in step S604d8.

[0292] The lost child identification unit 334c compares the personal attributes of the companions of the lost child candidate between the first and second time points to determine whether all of the companions of the lost child candidate have changed between the first and second time points. For example, if the similarity of the personal attributes of all of the companions of the lost child candidate between the first and second time points is less than a predetermined threshold, the lost child identification unit 334c determines that all have changed. Also, for example, if at least one of the companions of the lost child candidate between the first and second time points has a similarity of personal attributes that is equal to or greater than a predetermined threshold, the lost child identification unit 334c determines that all have not changed.

[0293] If it is determined that all of the information has changed (step S604d9; Yes), the lost child identification unit 334c detects a lost child (step S604d10) and returns to the detection process (step S604). That is, in this case, the lost child candidate is detected as a lost child.

[0294] If it is determined that all of the information has not changed (step S604d9; No), the lost child identification unit 334c does not detect a lost child (step S604d11) and returns to the detection process (step S604). In other words, in this case, the lost child candidate is treated as not being a lost child.

[0295] Referring again to Fig. 23, if a lost child is detected in step S304e, which is the same as in the first embodiment (step S304e; Yes), the range prediction unit 362 predicts the future movement range of the lost child based on the personal attributes of the lost child detected in step S304e (step S604g).

[0296] The lost child information generating unit 134d generates lost child information about the lost child (step S604f), and the process returns to the lost child detection process. The lost child information generated here includes, for example, the movement range and layout information generated in step S604g.

[0297] 22 again, the display control unit 335 causes the display unit 136 to display the lost child information generated in step S604f (step S605). For example, the display control unit 335 causes the display unit 136 to display a screen in which the predicted future movement range of the lost child is superimposed on layout information.

[0298] In this lost child detection process, the search range within the frame images can be narrowed down based on the predicted movement range of the lost child candidate, thereby reducing the processing load in the comparison process.

[0299] Furthermore, the predicted future range of movement of the lost child can be displayed on the display unit 136. This makes it easier to find the abducted lost child, and increases the likelihood of quickly finding the lost child.

[0300] (Operations and Effects) As described above, according to the third embodiment, the information processing system further includes the range prediction unit 362 that predicts the movement range of a person captured in a video using person attributes.

[0301] This reduces the processing load and speeds up the detection of lost children by predicting the movement range of potential lost children. Also, since the movement range of the detected lost child can be referenced to search for the lost child, it becomes easier to rescue the lost child. Therefore, it becomes possible to ensure the safety of lost children.

[0302] According to the third embodiment, the range prediction unit 362 predicts the movement range of a person by further using layout information of the location where the multiple image capture devices 101 capture images.

[0303] This improves the prediction of the movement range, which in turn speeds up the detection of lost children and makes it easier to rescue them, thereby ensuring the safety of lost children.

[0304] According to the third embodiment, the information processing system further includes a pattern detection unit 361 that detects a movement pattern of a person captured in the video based on the person's attributes between the first and second time points. The range prediction unit 362 further uses the movement pattern to predict the movement range of the person captured in the video between the first and second time points.

[0305] This improves the prediction of the movement range, which in turn speeds up the detection of lost children and makes it easier to rescue them, thereby ensuring the safety of lost children.

[0306] According to the third embodiment, the lost child detection unit 334 sets the predicted movement range of the lost child candidate as the search range for the lost child candidate, and detects the lost child candidate from people captured within the search range.

[0307] This reduces the processing load and speeds up the detection of lost children by predicting the movement range of potential lost children, thereby ensuring the safety of lost children.

[0308] According to the third embodiment, the information processing system further includes a display control unit 335 that displays information about the detected lost child on the display unit 136. The range prediction unit 362 predicts the movement range of the detected lost child. The lost child information includes the predicted movement range.

[0309] This allows the lost child to be searched for by referring to the detected range of movement of the lost child, making it easier to rescue the lost child and therefore ensuring the safety of the lost child.

[0310] According to the third embodiment, the lost child information further includes layout information. The display control unit 335 causes the display unit 136 to display an image in which the predicted movement range is superimposed on the layout information.

[0311] This makes it possible to easily search for the lost child by referring to the detected range of movement of the lost child, making it easier to rescue the lost child and therefore ensuring the safety of the lost child.

[0312] Although the embodiments and modifications of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0313] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps performed in each embodiment is not limited to the order shown. In each embodiment, the order of steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments and variations can be combined as long as the content is not contradictory.

[0314] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0315] 1. An information processing system comprising: an analysis result acquisition means for acquiring analysis results of video captured by multiple image capture means; a candidate detection means for detecting a lost child candidate from people captured in the video using person attributes and candidate conditions included in the analysis results; and a lost child detection means for detecting a lost child from the lost child candidate based on a result of comparing the lost child candidate's accompanying companions at a first time point with the lost child candidate at a second time point that is earlier than the first time point, when the lost child candidate has an accompanying companion at the first time point. 2. The information processing system described in 1., in which the candidate conditions include a condition related to age. 3. The information processing system described in 1. or 2., further comprising characteristic acquisition means for acquiring characteristic information of the lost child to be detected, the candidate conditions including the lost child's characteristic information. 4. The information processing system described in any one of 1. to 3., in which the lost child detection means detects a lost child from the lost child candidate based on whether the lost child candidate's accompanying companions have changed between the first and second time points, when the lost child candidate has an accompanying companion at the first time point. 5. 5. The information processing system of any one of 1. to 4., wherein the lost child detection means, when there is a companion of the lost child candidate at the first time point, detects a lost child from the lost child candidate based on the result of the comparison and a risk level according to the position of the lost child candidate at the first time point. 6. The information processing system of any one of 1. to 5., further comprising grouping means for specifying a group to which a person in the video belongs, using a person attribute included in the analysis result and a grouping condition for grouping people shown in the video, wherein the lost child detection means, when there is a companion of the lost child candidate at the first time point, compares the companion of the lost child candidate at the first time point and the second time point using the group to which the lost child candidate belongs at the first and second time points, and detects the lost child from the lost child candidate based on the result of the comparison.7. The information processing system described in 6., wherein the lost child detection means includes: a determination means for determining whether or not the lost child candidate has a companion at the first time point, using a group to which the lost child candidate belongs at the first time point; and a lost child identification means for, when it is determined that the lost child candidate has a companion at the first time point, comparing people who belong to the same group as the lost child candidate at each of the first and second time points, and detecting a lost child from among the lost child candidates based on the comparison results. 8. The information processing system described in 6. or 7., further comprising: a feature acquisition means for acquiring feature information of a lost child to be detected, wherein the grouping means further identifies a group to which a person in the video belongs, using the feature information of the lost child. 9. The information processing system described in any one of 1. to 8., further comprising a range prediction means for predicting a movement range of a person captured in the video, using the person attributes. 10. The information processing system described in 9., wherein the range prediction means further uses layout information of locations captured by the multiple image capture means to predict the movement range of the person. 11. The information processing system described in 9. or 10., further comprising pattern detection means for detecting a movement pattern of a person captured in the video based on person attributes between the first time point and the second time point, wherein the range prediction means further uses the movement pattern to predict a movement range of the person captured in the video between the first time point and the second time point. 12. The information processing system described in 11., wherein the lost child detection means sets a predicted movement range for the lost child candidate as a search range for the lost child candidate, and detects the lost child candidate from people captured in the search range. 13. The information processing system described in any one of 9. to 12., further comprising display control means for displaying lost child information regarding the detected lost child on display means, wherein the range prediction means predicts a movement range of the detected lost child, and the lost child information includes the predicted movement range. 14. 14. The information processing system according to Item 13, wherein the lost child information further includes the layout information, and the display control means causes the display means to display an image in which the predicted movement range is superimposed on the layout information.15. The information processing system described in 13. or 14., wherein the lost child information includes at least one of an image of the detected lost child and a position at the first time point. 16. The information processing system described in any one of 13. to 15., wherein, when there are multiple detected lost children, the display control means causes the display means to display the lost child information for the multiple lost children in order of risk at the first time point. 17. An information processing device comprising: analysis result acquisition means for acquiring analysis results of videos captured by multiple imaging means; candidate detection means for detecting a lost child candidate from people captured in the video using person attributes and candidate conditions included in the analysis results; and lost child detection means for detecting a lost child candidate from among the lost child candidate candidates, when there is a companion of the lost child candidate at the first time point, based on a result of comparing the companion of the lost child candidate at the first time point with the companion of the lost child candidate at the first time point and a second time point earlier than the first time point. 18. An information processing method in which one or more computers acquire analysis results of videos captured by multiple imaging means, detect lost child candidate people from those captured in the video using person attributes and candidate conditions included in the analysis results, and, if the lost child candidate has a companion at a first time point, detect a lost child from among the lost child candidate people based on a result of comparing the lost child candidate's companion at the first time point with the lost child candidate's companion at a second time point that is earlier than the first time point. 19. A recording medium having recorded thereon a program for causing one or more computers to acquire analysis results of videos captured by multiple imaging means, detect lost child candidate people from those captured in the video using person attributes and candidate conditions included in the analysis results, and, if the lost child candidate has a companion at a first time point, detect a lost child from among the lost child candidate people based on a result of comparing the lost child candidate's companion at the first time point with the lost child candidate's companion at a second time point that is earlier than the first time point.

[0316] 100 Information processing system 101, 101_1 to 101_M1 Imaging device 102 Analysis device 103, 203, 303 Information processing device 104, 104_1 to 104_M2 Terminal 131 Analysis result acquisition unit 132, 232 Candidate detection unit 133, 233 Grouping unit 134, 334 Lost child detection unit 134a Discrimination unit 134b Risk level identification unit 134c, 334c Lost child identification unit 134d, 334d Lost child information generation unit 135, 335 Display control unit 136 Display unit 137 Notification unit 141 Lost child information acquisition unit 142 Display control unit 143 Display unit 251 Feature acquisition unit 361 Pattern detection unit 362 Range prediction unit

Claims

1. an analysis result acquisition means for acquiring an analysis result of the images captured by the plurality of image capture means; a candidate detection means for detecting a candidate for a lost child from among people captured in the video by using a person attribute and a candidate condition included in the analysis result; A lost child detection means is provided for detecting a lost child from among the lost child candidates based on a result of comparing the companions of the lost child candidate at a first time point with the companions of the lost child candidate at the first time point and a second time point earlier than the first time point when the lost child candidate is accompanied by a companion at the first time point. Information processing system.

2. The system further includes a feature acquisition means for acquiring feature information of a lost child to be detected, The candidate condition includes characteristic information of the lost child. The information processing system according to claim 1 .

3. The lost child detection means detects a lost child from among the lost child candidates based on whether or not the accompanying person of the lost child candidate has changed between the first time point and the second time point when the accompanying person of the lost child candidate has a companion at the first time point.

3. The information processing system according to claim 1 or 2.

4. The lost child detection means detects a lost child from among the lost child candidates based on a result of the comparison and a degree of danger according to a position of the lost child candidate at the first time point, when the lost child candidate has a companion at the first time point.

3. The information processing system according to claim 1 or 2.

5. a grouping unit that identifies a group to which a person in the video belongs by using a person attribute included in the analysis result and a grouping condition for grouping the people in the video, The lost child detection means, when there is a companion of the lost child candidate at the first time point, compares the companion of the lost child candidate at the first time point and the second time point using a group to which the lost child candidate belongs at the first time point and the second time point, and detects the lost child from the lost child candidates based on the result of the comparison.

3. The information processing system according to claim 1 or 2.

6. The apparatus further includes a range prediction means for predicting a movement range of a person captured in the video by using the person attributes.

3. The information processing system according to claim 1 or 2.

7. a display control unit that causes a display unit to display information about the detected lost child, The range prediction means predicts a movement range of the detected lost child, The lost child information includes the predicted movement range.

7. The information processing system according to claim 6.

8. an analysis result acquisition means for acquiring an analysis result of the images captured by the plurality of image capture means; a candidate detection means for detecting a candidate for a lost child from among people captured in the video by using a person attribute and a candidate condition included in the analysis result; A lost child detection means is provided for detecting a lost child from among the lost child candidates based on a result of comparing the companions of the lost child candidate at a first time point with the companions of the lost child candidate at the first time point and a second time point earlier than the first time point when the lost child candidate is accompanied by a companion at the first time point. Information processing device.

9. One or more computers Obtaining the analysis results of the images captured by the multiple imaging means; Detecting a candidate for a lost child from among the people shown in the video using the person attributes and candidate conditions included in the analysis result; When the lost child candidate has a companion at a first time point, the lost child is detected from the lost child candidates based on a result of comparing the companion of the lost child candidate at the first time point with the companion at a second time point that is earlier than the first time point. Information processing methods.

10. On one or more computers, Obtaining the analysis results of the images captured by the multiple imaging means; Detecting a candidate for a lost child from among the people shown in the video using the person attributes and candidate conditions included in the analysis result; When the lost child candidate has a companion at a first time point, the lost child is detected from the lost child candidates based on a result of comparing the companion of the lost child candidate at the first time point with the companion at a second time point earlier than the first time point. A program for executing.

Images