Information Processing Apparatus, Information Processing Method, and Program

Abstract

To perform image processing to detect when a person is at risk.SOLUTION: An information processing device includes a process (1) of acquiring an image from a camera installed in a vehicle, a process (2) of acquiring a feature amount of a person on the basis of the image, a process (3) of acquiring relative position information of a plurality of predetermined parts of the person on the basis of the image, and a process (4) of determining whether information regarding the vehicle needs to be output on the basis of the feature amount of the person and the relative position information of the plurality of predetermined parts of the person.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In recent years, image processing has been performed for the purpose of ensuring safety. For example, Patent Document 1 describes that an image acquisition means such as a doorbell is provided for each residence in a specific area, and by using these image acquisition means and an image of a registered resident, it is detected that a person other than the resident has taken suspicious actions.

[0003] Further, Patent Document 2 describes that in an area where a person moves while holding a ticket (for example, in an entertainment facility such as an amusement park, a park, a movie theater, a concert hall, etc., or in a facility such as a department store or a large shopping mall), the identification information attached to the ticket is used to group companions, and by reading the identification information of the ticket at the time of leaving the venue, it is determined whether a person has left the venue together with a person other than the grouped person.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] There is a risk that a person may be forced to move without their intention. An example of the object of the present invention is to detect that a risk has occurred to a person by image processing.

Means for Solving the Problems

[0006] The information processing apparatus in the present disclosure performs a process of acquiring an image from a camera provided in a vehicle, a process of acquiring a feature amount of a person based on the image, a process of acquiring relative position information of a plurality of predetermined parts of the person based on the image, a process of determining whether to output information regarding the vehicle based on the feature amount of the person and the relative position information of the plurality of predetermined parts of the person, and executes.

[0007] The information processing method in the present disclosure includes a computer acquiring an image from a camera provided in a vehicle, acquiring a feature amount of a person based on the image, acquiring relative position information of a plurality of predetermined parts of the person based on the image, and determining whether to output information regarding the vehicle based on the feature amount of the person and the relative position information of the plurality of predetermined parts of the person.

[0008] The program in the present disclosure causes a computer to function as means for acquiring an image from a camera provided in a vehicle, means for acquiring a feature amount of a person based on the image, means for acquiring relative position information of a plurality of predetermined parts of the person based on the image, and means for determining whether to output information regarding the vehicle based on the feature amount of the person and the relative position information of the plurality of predetermined parts of the person.

Advantages of the Invention

[0009] According to the present invention, it is possible to detect that a risk has occurred to a person by image processing.

Brief Description of the Drawings

[0010] ​​The above-described object, as well as other objects, features, and advantages, will become more apparent from the preferred embodiments described below and the accompanying drawings.

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same reference numerals are given to the same components, and the description will be omitted as appropriate.

[0013] (Embodiment) FIG. 1 is a diagram for explaining the usage environment of the image processing apparatus 10 according to the embodiment. The image processing apparatus 10 is used together with at least one (preferably a plurality of) imaging devices 20.

[0014] The imaging device 20 is installed in a target area such as a street (e.g., a road) or indoors, and is used, for example, as a surveillance camera. The imaging device 20 may be installed at a predetermined location, or may be held by a moving body (e.g., at least one of a surveillance officer, a police officer, and a vehicle) moving within the target area. When a plurality of imaging devices 20 are installed in the target area, these plurality of imaging devices 20 are installed at locations separated from each other. In this case, it is preferable that the target area is covered by the imaging ranges of the plurality of imaging devices 20. And a part of the imaging range of at least one imaging device 20 may overlap with the imaging range of the imaging device 20 located adjacent to the said imaging device 20.

[0015] The frame rate of the image generated by the imaging device 20 is, for example, 0.5 frames / second or more, but it may also be less than this. And the imaging device 20 immediately transmits the generated image to the image processing device 10 together with the generation time of the said image. When a plurality of imaging devices 20 are installed, the imaging device 20 also transmits the identification information of the said imaging device 20 to the image processing device 10 together with the image. Here, when the imaging device 20 is held by a moving body, the imaging device 20 transmits, together with the identification information or in place of the identification information, position information (e.g., information based on GPS) indicating the position where the imaging device 20 generated the image to the image processing device 10.

[0016] The image processing device 10 determines whether a risk is occurring to a person in the target area by processing the image acquired from the imaging device 20. Examples of risks include being forced into actions or movements that the person does not intend, such as abduction or being taken away. In this case, the person at risk is often being pulled by someone else. The image processing device 10 detects this state.

[0017] When the image processing device 10 determines that a risk has occurred to a person in the target area, it outputs predetermined information. When the image processing device 10 has a display, the output destination of the predetermined information may be this display, or may be a device different from the image processing device 10 (hereinafter referred to as the external device 30). The predetermined information indicates that a risk has occurred to a person in the target area, and preferably further includes at least one of information indicating the position of the person at risk and an image of the person.

[0018] The external device 30 may be a display, may be a terminal installed in a monitoring center, or may be a terminal possessed by a person in a supervisor center, a supervisor in the target area, or a police officer. When the external device 30 acquires predetermined information from the image processing device 10, it performs a predetermined display. This predetermined display indicates that a risk has occurred to a person in the target area. Further, when the predetermined information output by the image processing device 10 includes at least one of information indicating the position of the person at risk and an image of the person, the predetermined display performed by the external device 30 preferably includes these.

[0019] Note that the image processing device 10 may be arranged in a monitoring center or may be a so-called cloud server.

[0020] FIG. 2 is a diagram showing an example of the functional configuration of the image processing device 10. The image processing device 10 includes an image processing unit 110 and a determination unit 120.

[0021] The image processing unit 110 acquires a plurality of images (e.g., frame images) generated by the imaging device 20. The plurality of images acquired by the image processing unit 110 are generated at different timings from each other. Here, the image processing unit 110 may acquire images generated by each of the plurality of imaging devices 20. Then, the image processing unit 110 processes the images among these plurality of images that contain the same person (hereinafter referred to as the first person), and detects a change in the angle of the body of the first person. In this detection, the image processing unit 110 may use a plurality of images generated by the plurality of imaging devices 20.

[0022] Also, when at least the magnitude of this change satisfies a criterion, the image processing unit 110 detects information regarding a second person or object (hereinafter referred to as the target of attention) existing in the direction in which the body is tilted. The information detected here is at least the presence or absence of the target of attention and its type, which were described with reference to FIG. 1. Objects that can be the target of attention are, for example, vehicles such as automobiles or buildings.

[0023] Here, when the change in the angle of the body of the first person satisfies the criterion (for example, becomes equal to or greater than a reference value), and a second person exists as the target of attention, there is a possibility that the first person is being pulled by the second person. Also, when the change in the angle of the body of the first person satisfies the criterion and an object such as a vehicle or a building exists as the target of attention, there is a possibility that the first person is being pulled by a person inside the object.

[0024] The determination unit 120 outputs predetermined information using the processing result of the image processing unit 110. As an example, the determination unit 120 outputs predetermined information when a first criterion regarding at least one of the above-described target of attention and the first person is satisfied. Specific examples of the first criterion will be described later with reference to a flowchart. Also, specific examples of the predetermined information are as described with reference to FIG. 1.

[0025] Note that the image processing unit 110 identifies a person's inclination using, for example, the relative positions of a plurality of predetermined parts of the first person. The plurality of predetermined parts are, for example, as shown in FIG. 3, the head A1, the neck A2, the right shoulder A31, the left shoulder A32, the right elbow A41, the left elbow A42, the right hand A51, the left hand A52, the right hip A61, the left hip A62, the right knee A71, the left knee A72, the right foot A81, and the left foot A82. And the posture of a person is represented by a line connecting adjacent keypoints. And the image processing unit 110 identifies an index indicating the inclination of the person using these lines. As an example, the image processing unit 110 uses the inclination of the line connecting the neck and the right hip and the inclination of the line connecting the neck and the left hip as indices indicating the inclination of the person's body.

[0026] In the example shown in FIG. 2, the image processing apparatus 10 further includes a storage unit 130. The storage unit 130 stores information necessary when the image processing unit 110 and the determination unit 120 perform processing, and stores the analysis result of the image by the image processing unit 110 and the information acquired from the imaging apparatus 20 together with the image.

[0027] Note that the information stored in the storage unit 130 may include, for example, a model generated by machine learning. This model outputs the inclination of the person included in the image using, for example, the image or its feature amount (for example, the relative positions of the predetermined parts of the person described above) as an input. When a plurality of imaging apparatuses 20 are installed in the target area, this model may be generated for each of the plurality of imaging apparatuses 20.

[0028] Also, the information stored in the storage unit 130 may include information for calculating the position of the person in the real space using the position (xy coordinates) of the person in the image. This information is set for each imaging apparatus 20, for example, by pre-learning.

[0029] FIG. 4 is a diagram showing an example of information stored in the storage unit 130. The storage unit 130 stores each of a plurality of images generated by the imaging device 20 together with the generation time of the image and the analysis result of the image by the image processing unit 110. When a plurality of imaging devices 20 are installed, the storage unit 130 stores, for each of the plurality of imaging devices 20, the image generated by the imaging device 20, the generation time of the image, and the analysis result of the image. As an example, the storage unit 130 stores the above-described pieces of information in association with the identification information of the imaging device 20 that generated the image. When the imaging device 20 is held by a moving body, the storage unit 130 further stores, for each image, the position information of the imaging device 20 when the image was generated.

[0030] FIG. 5 is a diagram showing an example of the analysis result stored in the storage unit 130. The storage unit 130 stores the information shown in this figure for each image. In the example shown in this figure, the storage unit 130 stores the feature amount of the person included in the image, the position of the person, and the inclination of the person. Here, the position of the person is preferably the position in the real space. Here, the image processing unit 110 calculates the position of the person in the real space using the above-described information stored in the storage unit 130 corresponding to the identification information of the imaging device 20 that generated the image and the position (xy coordinates) of the person in the image. When the imaging device 20 is held by a moving body, the position information of the imaging device at the time when the image was generated, the orientation and the field angle of the imaging device 20, and the position of the person in the image are used to calculate the position of the person in the real space. Then, the image processing unit 110 determines whether the person included in the image generated at a certain timing and the person included in the image generated at another timing are the same person using the feature amount of the person. Also, the image processing unit 110 determines whether the person included in the image captured by a certain imaging device 20 and the person included in the image captured by another imaging device 20 at the same timing are the same person using at least one (preferably both) of the feature amount of the person and the position in the real space.

[0031] Then, the image processing unit 110 identifies the change in the inclination of a person by using the difference between the inclination of a certain person in a past image (for example, an image a predetermined number of images ago or an image a predetermined time ago) stored in the storage unit 130 and the inclination of the person in the image newly acquired this time. Here, the time difference between the generation timing of the past image used here and the present is, for example, 1 second or more and 5 seconds or less.

[0032] FIG. 6 is a diagram showing an example of the hardware configuration of the image processing apparatus 10. The image processing apparatus 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0033] The bus 1010 is a data transmission path for the processor 1020, the memory 1030, the storage device 1040, the input / output interface 1050, and the network interface 1060 to transmit and receive data to and from each other. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0034] The processor 1020 is a processor realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0035] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0036] The storage device 1040 is an auxiliary storage device realized by, for example, a HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, or a ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the image processing apparatus 10 (for example, the image processing unit 110 and the determination unit 120). When the processor 1020 reads and executes these program modules onto the memory 1030, each function corresponding to the program module is realized. Further, the storage device 1040 also functions as a storage unit 130.

[0037] The input / output interface 1050 is an interface for connecting the image processing apparatus 10 and various input / output devices. For example, the image processing apparatus 10 communicates with an external device 30 via the input / output interface 1050.

[0038] The network interface 1060 is an interface for connecting the image processing apparatus 10 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the network may be a wireless connection or a wired connection. The image processing apparatus 10 communicates with the imaging device 20 via the network interface 1060. Further, the image processing apparatus 10 may communicate with the external device 30 via the network interface 1060.

[0039] FIG. 7 is a flowchart showing an example of the processing performed by the image processing apparatus 10. The image processing apparatus 10 performs the processing shown in this figure each time an image is acquired from the imaging device 20.

[0040] When the image processing unit 110 of the image processing apparatus 10 acquires a new image from the imaging apparatus 20 (step S10), it detects the feature amounts and positions of the people included in the image, and also detects the objects included in the image and their positions. The objects detected here are at least objects that can be a target of attention (for example, vehicles and buildings). Further, the image processing unit 110 calculates the inclination of the people included in the image. Then, the image processing unit 110 calculates the change in the inclination of the people included in the image acquired in step S10 by using the past processing results stored in the storage unit 130 and the processing results of step S20 (step S20). Here, when a plurality of people are included in one image, the image processing unit 110 calculates the change in the inclination of each of the plurality of people.

[0041] Next, the determination unit 120 of the image processing apparatus 10 determines whether the change in inclination satisfies a criterion, for example, whether the change in inclination during a predetermined time is equal to or greater than a reference value (step S30). When the criterion is satisfied (step S30), the determination unit 120 determines whether there is a target of attention in the direction in which the person (the first person) is inclined among the surroundings of the person (step S40). In this process, the positions of the people and objects detected in step S20 are used.

[0042] When there is a target of attention (step S40: Yes), the determination unit 120 determines whether a first criterion regarding at least one of the target of attention and the first person is satisfied (step S50).

[0043] For example, when the target of attention is a second person, the first criterion includes that the second person is not a companion of the first person.

[0044] For example, when the imaging device 20 is installed at the entrance of the target area, the image processing unit 110 treats a plurality of people as a single group in the image generated by the imaging device 20 when the distance between the plurality of people continues to be within the reference distance for a reference time or more. At this time, the image processing unit 110 may treat people holding hands as a single group, or may treat people whose faces are facing each other repeatedly (or continuously for a certain period of time) as a single group. Then, the image processing unit 110 associates the feature amounts of the people constituting the same group with each other and stores them in the storage unit 130. And the determination unit 120 uses this information to determine whether the second person belongs to the same group as the first person, in other words, whether the second person is a companion of the first person.

[0045] Further, the determination unit 120 may determine whether the second person is a companion, for example, using the distance between the first person and the second person before the magnitude of the change satisfies the criterion. In this case, the determination unit 120 determines that the second person is not a companion of the first person when the distance between the first person and the second person continues to be more than the reference distance for a reference time or more, for example, using the past processing results stored in the storage unit 130. The reference time is, for example, 10 seconds or more. Also, the reference distance is, for example, 2 m or more. Here, the determination unit 120 also determines that the distance is more than the reference distance even when the second person is not detected.

[0046] Also, when the object of attention is an object such as a vehicle or a building, the first criterion is that after the magnitude of the change satisfies the criterion, the first person is not detected from the image generated by the imaging device 20 within the reference time without timing out from the imaging device 20 that generated the image in which the first person was detected in step S20. Such a condition is set because when this condition is satisfied, it is highly likely that the first person has been taken (or pushed) into the vehicle or the building.

[0047] The image processing unit 110 can estimate the attributes of the first person from the feature amounts of the first person. And the determination unit 120 may use, as a first criterion, that the attributes of the first person satisfy the criterion and that there is an object of attention. Here, the criterion that the attributes of the first person should satisfy is at least one of the first person being a child or a female. The image processing unit 110 can determine whether the person included in the image is a child, for example, by using whether the height of the first person is equal to or less than a reference value (for example, a predetermined value of 120 cm or less). Also, the image processing unit 110 can determine the gender of the first person using the feature amounts of the first person.

[0048] And when the first criterion is satisfied (step S50: Yes), the determination unit 120 outputs predetermined information (step S60). The content and output destination of the predetermined information are as described with reference to FIG. 1. When the object of attention is a vehicle and the number of the license plate of the vehicle can be estimated or specified by the image processing by the image processing unit 110, the predetermined information may include this number. At this time, the predetermined information may include the position of the vehicle when step S50 is satisfied. Also, when the object of attention is a building, the predetermined information may include the position information (for example, address) or name of this building. The determination unit 120 specifies this position information using, for example, the identification information of the imaging device 20 that generated the image and the position of the building in the image. Note that the position information and name of the building may be stored in the storage unit 130 in advance in association with the identification information of the imaging device 20 and the position of the building in the image.

[0049] Thereafter, the predetermined information or a warning display based on the predetermined information is displayed on the external device 30 or a large display connected thereto. This display may be, for example, overlaid on the map image displayed on the screen. In this case, the display position preferably corresponds to, for example, the position of the object of attention.

[0050] When the external device 30 is arranged at the monitoring center, a person at the monitoring center comprehensively judges the presence or absence of an incident by the predetermined information and other information displayed on the external device 30 or the like, and inputs the determination result into the external device 30. Note that this process may be performed by the external device 30. In this case, the external device 30 may use a model generated by machine learning.

[0051] When it is determined that there is an incident, the image processing device 10 or the external device 30 identifies a police officer or security guard (hereinafter referred to as a police officer) who is expected to be able to reach the position of the attention target (i.e., the position where the incident is presumed to have occurred) most quickly, and transmits the position of the attention target when step S50 is satisfied to the terminal possessed by this police officer. The terminal to be the transmission destination is selected, for example, using the position information and speed (including the moving direction) of the terminal at that time. Note that the position of the terminal (i.e., the position of the police officer) is calculated using, for example, GPS, and the speed is calculated using the position history. Note that when the speed is used, the means of movement (e.g., walking, bicycle, automobile, motorcycle) of this police officer can be estimated. The image processing device 10 or the external device 30 may use this means of movement instead of the speed when selecting a police officer.

[0052] Note that marks indicating the positions and means of movement of the police officers in the vicinity may also be displayed on the screen of the external device 30 or the like.

[0053] Here, the information transmitted to the terminal and displayed on the terminal may further include the urgency, the type of the assumed incident, the tools required for countermeasures, a display for selecting whether or not to be able to respond (e.g., selection buttons), and information indicating by when to arrive (hereinafter referred to as the available response time). In this case, the police officer possessing the terminal judges whether or not he / she can respond using the displayed information, and operates the terminal according to the judgment result. The terminal transmits the search result, i.e., the selection result, to the image processing device 10 or the external device 30.

[0054] If the police officer selects that they can respond, the image processing apparatus 10 or the external apparatus 30 generates information indicating a movement route (e.g., the shortest route) from the current position of the terminal to the position where the incident is presumed to have occurred, and transmits it to the terminal for display. At this time, the image processing apparatus 10 or the external apparatus 30 may calculate the estimated arrival time using the current time and the congestion level on the movement route, and transmit it to the terminal for display.

[0055] On the other hand, if the police officer selects that they cannot respond, the image processing apparatus 10 or the external apparatus 30 performs the above-described processing on the terminal carried by the police officer who is expected to be able to reach the position of the next attention target of the police officer.

[0056] Note that when it is selected that the response is possible, the image processing apparatus 10 or the external apparatus 30 may further transmit the following information to the terminal. · An image and detailed information (e.g., the owner, etc.) of the attention target (e.g., a vehicle) · Audio data recorded around the attention target in step S50 · A list of people to be interviewed (e.g., important witnesses)

[0057] Note that when the terminal carried by the police officer receives a list of people to be interviewed (e.g., important witnesses), for each person included in the list, it may display whether the interview has not been conducted, is in progress, or has been completed. In this case, initially, "interview not conducted" is displayed for all people. Then, the police officer operates the terminal according to the progress of the interview to change "interview not conducted" to "interview in progress", and also change "interview in progress" to "interview completed". Then, the terminal transmits information indicating this change to the image processing apparatus 10 or the external apparatus 30.

[0058] In addition, when a police officer is wearing sensors such as a camera and a microphone, the terminal may save the data detected by the sensors during a hearing, or may transmit the data to the image processing apparatus 10 or the external apparatus 30. When saving the data, the terminal may convert the audio data into text data, extract necessary information from the text data, and save it.

[0059] As described above, according to the present embodiment, the image processing apparatus 10 calculates a change in the inclination of a person existing in the target area by processing the image generated by the imaging apparatus 20. When the change in inclination satisfies a criterion, there is a target of attention which is a second person or an object existing in the direction in which the body is inclined, and when a first criterion regarding at least one of the target of attention and the first person is satisfied, since there is a high possibility that a risk has occurred to that person, predetermined information is output. Therefore, it is possible to detect that a risk has occurred to a person by image processing.

[0060] Note that the determination unit 120 may omit step S50 in FIG. 7. In this case, when the change in the inclination of the first person satisfies a criterion and there is a target of attention, the determination unit 120 outputs predetermined information without determining whether the first condition is satisfied.

[0061] (Modification Example 1) FIG. 8 is a flowchart according to Modification Example 1 of FIG. 7. In this modification example, the determination unit 120 of the image processing apparatus 10 sets, after the first criterion is satisfied (step S50: Yes), the fact that the second criterion, that is, the exception condition is not satisfied (step S52: Yes), as a condition for performing the process shown in step S60. Examples of the exception condition include the distance between the first person and the target of attention increasing after a predetermined time has elapsed.

[0062] According to this modification example, when the second condition, which is an exception condition, is satisfied, the determination unit 120 does not output predetermined information even if the first condition is satisfied. Therefore, the detection accuracy of the occurrence of risk to people is increased. In this modification example, when the second criterion, that is, the exception condition, is satisfied, an image indicating a person or object to be noted (for example, a second person or a vehicle) may be stored in the storage unit 130. This is because the second person or vehicle at this time may cause risk to other people in the future.

[0063] (Modification Example 2) FIG. 9 is a flowchart according to Modification Example 2 of FIG. 7. In this modification example, at least one imaging device 20 is installed at the entrance of the target area. Then, the image processing unit 110 of the image processing device 10 selects a person to be monitored by processing the image generated by this imaging device 20, and stores the feature amount of this person to be monitored in the storage unit 130. Here, the person to be monitored is, for example, at least one of a child, a woman, and an elderly person. In addition, the image processing unit 110 also stores the feature amount of the companion of the person to be monitored in the storage unit 130. Here, the criterion for determining the companion is the same as that described in the embodiment.

[0064] Then, the image processing device 10 executes the processing shown in Modification Example 1 for the person to be monitored by processing the image generated by another imaging device 20.

[0065] Specifically, when the image processing unit 110 of the image processing device 10 acquires a new image from the imaging device 20 (step S10), it detects the feature amount and position of the person included in the image, and also detects the object included in the image and its position. The object detected here is at least an object that can be a target of attention (for example, a vehicle or a building). In addition, the image processing unit 110 detects the feature amount of the person included in the image, and determines whether it matches the feature amount of the person to be monitored stored in the storage unit 130. Thereby, the image processing unit 110 detects the person to be monitored (step S12). Then, the image processing unit 110 and the determination unit 120 perform the processing after step S20 on the person to be monitored detected in step S12.

[0066] In this modification, the first criterion may be that the person to be monitored (e.g., a child) is separated from the accompanying person (e.g., a parent) by a certain distance or more. In this case, as the second criterion, it is used that the distance between the person to be monitored and the accompanying person becomes less than a certain distance within a predetermined time.

[0067] Also in this modification, when the body line of the accompanying person rather than the person to be monitored inclines (e.g., lies down) above a reference value and then the distance between the person to be monitored and the accompanying person becomes separated by a reference value or more, the determination unit 120 may output predetermined information. At this time, the feature amounts of the people who were located in the transition of the person to be monitored and the accompanying person may be stored in the storage unit 130 as the feature amounts of the persons requiring attention. Then, the image processing apparatus 10 may perform tracking of the persons requiring attention by processing the images generated by the plurality of imaging apparatuses 20.

[0068] When the target area is within a facility such as an airport or a shopping mall, at least one imaging apparatus 20 is installed at the entrance to the facility and at least one imaging apparatus 20 is installed at the exit from the facility. Here, when the entrance and the exit are the same, at least one imaging apparatus 20 is installed at this entrance / exit. And when the person to be monitored exits the facility with a person other than the accompanying person or when the person to be monitored cannot be detected at the exit even after a certain time has passed (e.g., it is assumed that the person is pushed into a bag), the determination unit 120 may output predetermined information. In particular, when the person to be monitored exits the facility with a person other than the accompanying person, the output destination of the predetermined information may be a gate installed at the exit or a security guard (or police officer) at the exit.

[0069] As described above, the embodiments of the present invention have been described with reference to the drawings, but these are examples of the present invention, and various configurations other than the above can also be adopted.

[0070] Also, in the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order. However, the execution order of the steps executed in each embodiment is not limited to the order of the description. In each embodiment, the order of the illustrated steps can be changed within a range that does not interfere with the content. Also, the above-described embodiments can be combined within a range where the content does not conflict with each other.

[0071] Some or all of the above-described embodiments can be described as follows in the appended claims, but are not limited thereto. 1. An image processing means for processing a plurality of images including a first person and generated at different timings from each other, A determination means for outputting predetermined information using the processing result of the image processing means, Comprising, The image processing means, By processing the plurality of images, a change in the angle of the body of the first person is detected, When at least the magnitude of the change satisfies a criterion, information regarding an object of attention, which is a second person or an object existing in the direction in which the body is tilted, is detected, The determination means is an image processing apparatus that outputs the predetermined information when at least one of the first criterion regarding the object of attention and the first person is satisfied. 2. In the image processing apparatus according to 1 above, The object of attention is the second person, The first criterion includes that the second person is not a companion of the first person, an image processing apparatus. 3. In the image processing apparatus according to 1 above, The object of attention is the second person, The image processing means has detected the position of the first person and the position of the second person in each of the plurality of images, The first criterion includes that the distance between the first person and the second person has been continuously separated by a reference distance or more for a reference time or more before the magnitude of the change satisfies the criterion, an image processing apparatus. 4. In the image processing apparatus according to 1 above, The object of attention is a vehicle, The first criterion is that, after the magnitude of the change satisfies the criterion, the first person cannot be detected within the reference time without frame out. An image processing apparatus. 5. In the image processing apparatus according to 1 above, The first criterion is that the object of attention exists and the attribute of the first person satisfies the criterion. An image processing apparatus. 6. A computer Image processing for processing a plurality of images including a first person and generated at different timings from each other, Judgment processing for outputting predetermined information using the processing result of the image processing, is performed In the image processing, the computer By processing the plurality of images, detects a change in the angle of the body of the first person, Detects information on a second person or an object that is the object of attention and exists in the direction in which the body is tilted, at least when the magnitude of the change satisfies the criterion, In the judgment processing, the computer outputs the predetermined information when at least a first criterion regarding the object of attention and the first person is satisfied. An image processing method. 7. In the image processing method according to 6 above, The object of attention is the second person, The first criterion includes that the second person is not a companion of the first person. An image processing method. 8. In the image processing method according to 6 above, The object of attention is the second person, In the image processing, the computer detects the position of the first person and the position of the second person in each of the plurality of images, The first criterion includes that the distance between the first person and the second person was continuously greater than or equal to the reference distance for a reference time or more before the magnitude of the change satisfied the criterion. An image processing method. 9. In the image processing method according to 6 above, The object of attention is a vehicle, The first criterion is that after the magnitude of the change satisfies the criterion, the first person cannot be detected within the reference time without going out of the frame, an image processing method. 10. In the image processing method according to 6 above, The first criterion is that the object of attention exists and the attributes of the first person satisfy the criterion, an image processing method. 11. A computer is provided with An image processing function for processing a plurality of images generated at different times and including a first person, A judgment function for outputting predetermined information using the processing result of the image processing function, and The image processing function By processing the plurality of images, detects a change in the angle of the body of the first person, When at least the magnitude of the change satisfies the criterion, detects information about an object of attention, which is a second person or an object existing in the direction in which the body is tilted, The judgment function is a program that outputs the predetermined information when at least one of the first criterion regarding the object of attention and the first person is satisfied. 12. In the program according to 11 above, The object of attention is the second person, The first criterion includes that the second person is not a companion of the first person, a program. 13. In the program according to 11 above, The object of attention is the second person, The image processing function has detected the position of the first person and the position of the second person in each of the plurality of images, The first criterion includes that the distance between the first person and the second person has been continuously separated by a reference distance or more for a reference time or more before the magnitude of the change satisfies the criterion, a program. 14. In the program according to 11 above, The object of attention is a vehicle, The first criterion is that after the magnitude of the change satisfies the criterion, the first person cannot be detected within the reference time without timing out. Program. 15. In the program according to 11 above, The first criterion is that the object of attention exists and the attributes of the first person satisfy the criterion. Program.

Explanation of symbols

[0072] 10 Image processing device 20 Imaging device 30 External device 110 Image processing unit 120 Judgment unit 130 Storage unit

Claims

1. Acquiring images from a camera installed in a vehicle; A process of acquiring a feature amount of a person based on the image; A process of acquiring relative position information of a plurality of predetermined parts of the person based on the image; a process of determining whether or not a posture of the person has been changed by another person based on whether or not another person is present in the direction of the inclination when it is determined that a change in the inclination of the person during a predetermined time period is equal to or greater than a reference value based on the feature amount of the person and relative position information of a plurality of predetermined parts of the person; a process of detecting and outputting information on the other person or an object related to the other person from the image when it is determined that the posture of the person has been changed by the other person; An information processing device that executes the above.

2. further performing a process of classifying the person based on the feature amount of the person; The information processing device according to claim 1 .

3. The category includes a child or a woman, The information processing device according to claim 2 .

4. The plurality of predetermined parts include a person's shoulders, elbows, and hands. The information processing device according to claim 1 .

5. The computer Acquire images from a camera installed in the vehicle, Acquiring a feature amount of a person based on the image; acquiring relative position information of a plurality of predetermined parts of the person based on the image; when it is determined that the change in the inclination of the person during a predetermined time period is equal to or greater than a reference value based on the feature amount of the person and the relative position information of a plurality of predetermined parts of the person, it is determined whether or not the posture of the person has been changed by another person based on whether or not the other person is present in the direction of the inclination; when it is determined that the posture of the person has been changed by the other person, detecting information of the other person or an object related to the other person from the image and outputting the information; An information processing method comprising the steps of:

6. The computer, classifying the person based on the feature amount of the person; The information processing method according to claim 5 , further comprising:

7. The category includes a child or a woman, The information processing method according to claim 6.

8. The plurality of predetermined parts include a person's shoulders, elbows, and hands. The information processing method according to any one of claims 5 to 7.

9. Computer, A means for acquiring images from a camera mounted on the vehicle; A means for acquiring a feature amount of a person based on the image; means for acquiring relative position information of a plurality of predetermined parts of the person based on the image; a means for determining whether or not a posture of the person has been changed by another person based on whether or not another person is present in the direction of the inclination when it is determined that a change in the inclination of the person during a predetermined time period is equal to or greater than a reference value based on the feature amount of the person and relative position information of a plurality of predetermined parts of the person; a means for detecting and outputting information on the other person or an object related to the other person from the image when it is determined that the posture of the person has been changed by the other person; A program to function as a

10. The computer, A means for classifying the person based on the feature amount of the person; The program according to claim 9, further functioning as

11. The category includes a child or a woman, The program according to claim 10.

12. The plurality of predetermined parts include a person's shoulders, elbows, and hands. The program according to any one of claims 9 to 11.

13. the process of determining whether the posture of the person has been changed by another person includes a process of detecting a change in an angle of the person's body from relative position information of a plurality of predetermined parts of the person; The information processing device according to claim 1 .

14. The computer, and detecting a change in an angle of the person's body from relative position information of a plurality of predetermined parts of the person when determining whether or not the posture of the person has been changed by another person. The information processing method according to claim 5.

15. the means for determining whether or not the posture of the person has been changed by another person detects a change in the angle of the person's body from relative position information of a plurality of predetermined parts of the person; The program according to claim 9.

Images