Image analysis device, image analysis method, and program
The image analysis device and method facilitate understanding the relationship between a specific person and related individuals by processing video images to display organizational connections.
Patent Information
- Application Number
- JP2023075670
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-01
- Publication Date
- 2024-11-14
AI Technical Summary
Existing image analysis technologies fail to determine the relationship between a specific person and related individuals identified from a video.
An image analysis device and method that acquires attribute information from multiple images, identifies a designated person, and outputs a screen showing the relationship between the designated person and related individuals, including organizational information.
Enables easy understanding of the relationship between individuals in a video by visualizing their organizational connections.
Smart Images

Figure 2024160549000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image analysis device, an image analysis method, and a program. [Background technology]
[0002] Technology for analyzing images and identifying specific people is used in a variety of situations.
[0003] An example of a technology for identifying a specific person by analyzing an image is disclosed in the following Patent Document 1. Patent Document 1 discloses a technology for identifying a specific person and the person's potential companions based on an input video. For example, Patent Document 1 discloses a method for determining potential companions for a specific person based on the number of times that the specific person and other people appear simultaneously. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2023-018113 A Summary of the Invention [Problem to be solved by the invention]
[0005] When a specific person and a person related to that person are identified from a video, there is a need to understand the relationship between those people. The technology in Reference 1 does not determine this relationship.
[0006] One example of the objective of the present invention is to provide an image analysis device, an image analysis method, and a program that, in consideration of the above-mentioned problems, make it possible to easily understand the relationship between a specific person identified from an image and people related to that person. [Means for solving the problem]
[0007] According to one aspect of the present invention, an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing a plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person by using the attribute information acquired for each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; An image analysis device is provided, comprising:
[0008] According to one aspect of the present invention, At least one computer By processing the multiple images, attribute information indicating attributes of each person appearing in each image is obtained; Accepting a person designation input indicating a designated person; Identifying a related person who is a person related to the specified person using the attribute information acquired for each of the persons; outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; An image analysis method is provided, comprising:
[0009] According to one aspect of the present invention, At least one computer an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing the plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person, using the attribute information acquired about each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; A program is provided to function as such. Effect of the Invention
[0010] According to one aspect of the present invention, for example, a person performing the task of checking video can easily understand the relationships between multiple people identified from the video. [Brief description of the drawings]
[0011] [Figure 1] 1 is a diagram showing an overview of an image analyzing device according to a first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of a system configuration of an image analysis system including an image analysis device. [Diagram 3] FIG. 11 is a diagram illustrating an example of information stored in a database. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that realizes an image analyzing apparatus. [Diagram 5] 10 is a flowchart illustrating a process executed by an image analyzing device according to the second embodiment. [Figure 6] FIG. 4 is a diagram showing an example of a screen output by a display output unit. [Figure 7] FIG. 11 is a diagram showing another example of a screen output by the display output unit. [Figure 8] FIG. 11 is a diagram showing another example of a screen output by the display output unit. [Figure 9] FIG. 13 is a diagram for explaining the operation of an image analyzing device (display output unit) according to a first modified example of the second embodiment. [Figure 10] FIG. 13 is a diagram for explaining the operation of an image analyzing device (display output unit) according to a first modified example of the second embodiment. [Figure 11] FIG. 13 is a diagram for explaining the operation of an image analyzing device (an associated person identifying unit and a display output unit) according to a second modified example of the second embodiment. [Figure 12] FIG. 13 is a diagram illustrating an example of a screen updated in response to a criterion change input. [Figure 13] FIG. 13 is a diagram for explaining the operation of an image analyzing device (an associated person identifying unit and a display output unit) according to a third modified example of the second embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of a screen updated in response to a criterion change input. [Figure 15] FIG. 13 is a diagram illustrating an example of a system configuration of an image analysis system 1 according to a third embodiment. [Figure 16] FIG. 11 is a diagram illustrating an example of information stored in a posture information database. [Figure 17] FIG. 13 is a diagram for explaining the operation of the image analyzing device (associated person identifying section and display output section) according to the third embodiment. [Figure 18] FIG. 13 is a diagram for explaining the operation of the image analyzing device (associated person identifying section and display output section) according to the third embodiment. [Figure 19] FIG. 13 is a diagram for explaining the operation of the image analyzing device (associated person identifying section and display output section) according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all drawings, similar components are given similar reference numerals, and descriptions thereof will be omitted as appropriate. Furthermore, unless otherwise specified, in each block diagram, each block represents a functional configuration, not a hardware configuration. Furthermore, when elements such as blocks in a diagram are linked by arrows, the direction of the arrows is merely intended to make the flow of information easier to understand. In this case, the direction of the arrows does not limit the direction of communication (one-way communication / two-way communication) unless otherwise specified.
[0013] In the following description, "acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and inputting data or information output from another device to the device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.
[0014] First embodiment 1 is a diagram showing an overview of an image analyzing device 10 according to a first embodiment. The image analyzing device 10 shown in the figure includes an attribute information acquiring unit 110, a designated input accepting unit 120, a related person identifying unit 130, and a display output unit 140.
[0015] The attribute information acquisition unit 110 processes a plurality of input images. Based on the processing result of each image, the attribute information acquisition unit 110 acquires attribute information indicating the attributes of each person appearing in each image. The designated input acceptance unit 120 accepts a person designation input indicating a certain person. In the following description, the person indicated by the person designation input is also referred to as a "designated person". The related person identification unit 130 identifies a person related to the designated person using the attribute information acquired for each person. In the following description, the person related to the designated person is also referred to as a "related person". The display output unit 140 outputs a screen including the person related information and organization information to an output device such as a display. Here, the "person related information" is information indicating the relationship between the designated person and the related person. Also, the "organization information" is information regarding the organization to which the designated person and the related person belong. Specific examples of the person related information and organization information will be described later.
[0016] According to this embodiment, attribute information of each person is acquired for each image by processing each of the multiple images. Then, when a person is identified as a designated person, a person related to the designated person is identified based on the attribute information acquired for each image. Then, a screen including person relationship information indicating the relationship between the designated person and the specific person and organization information regarding the organization to which each person belongs is output. That is, an image analysis device and an image analysis method are provided for easily understanding the relationship between a specific person and a person related to that person. In addition, as will be understood from the description of the second embodiment described later, a program for realizing the function of the image analysis device 10, which is a means for solving the above problem, on at least one computer, and a non-transitory computer-readable recording medium for storing the program are provided.
[0017] Hereinafter, a detailed example of the image analyzing device 10 in another embodiment will be described.
[0018] Second embodiment FIG. 2 is a diagram illustrating a system configuration of an image analysis system 1 including an image analysis device 10. The image analysis system 1 illustrated in FIG. 2 includes the image analysis device 10, a database 20, and a display 30. The image analysis device 10 and the database 20 are connected via a network 40. The display 30 is connected as a display for the image analysis device 10 via an input interface (not shown) of the image analysis device 10. The configuration of the image analysis system 1 is not limited to the example shown in this figure. For example, the database 20 may be constructed in a storage area inside the image analysis device 10. For example, the display 30 may be a display of a computer (not shown) connected to the image analysis device 10 via the network 40.
[0019] As illustrated in FIG. 3, the database 20 stores various information related to various people. FIG. 3 is a diagram showing an example of information stored in the database 20. In the example of FIG. 3, the database 20 stores registration information for face recognition, the name of the person, personal attributes, and organization information. Here, the registration information for face recognition includes at least one of a face image and a feature amount that can be extracted based on the face image. The personal attributes include any attribute other than the face feature that can be used to identify at least one person from the image. For example, the personal attributes include information on the physical features of the person (a combination of various features such as the shape of the skeleton, height, body size, hair color, and skin color). In addition to these any attributes, the personal attributes may further include other attributes such as age and sex. The organization information includes information identifying the organization to which the target person belongs, information indicating the position and rank within the organization, and the like. The organization information may further include information indicating the system (structure) of the relevant organization.
[0020] Information such as that exemplified in Fig. 3 is accumulated, for example, as follows. First, image analyzing device 10 or a management terminal (not shown) receives as input a facial image of a target person, as well as information on the person's attributes and information on the person's affiliated organization. Then, image analyzing device 10 or a management terminal (not shown) compiles the information received as input to generate one record. Then, image analyzing device 10 or a management terminal (not shown) stores the generated record in database 20. In this manner, information such as that exemplified in Fig. 3 is accumulated in database 20.
[0021] The image analyzing device 10 includes at least the attribute information acquiring unit 110, the designated input receiving unit 120, the related person identifying unit 130, and the display output unit 140 described in the first embodiment.
[0022] FIG. 4 is a diagram illustrating an example of a hardware configuration of a computer 1000 that realizes the image analyzing device 10. As shown in FIG.
[0023] The computer 1000 has a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.
[0024] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to a bus connection.
[0025] The processor 1020 may be a Central Processing Unit (CPU) or a Graphics Processing Unit (GPU).
[0026] The memory 1030 is a main storage device realized by a random access memory (RAM) or the like.
[0027] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), etc. The storage device 1040 stores at least program modules that realize the functions of the image analysis system 1 described above (the attribute information acquisition unit 110, the designated input reception unit 120, the related person identification unit 130, and the display output unit 140). The storage device 1040 may also function as the database 20.
[0028] The processor 1020 loads a program module read from the storage device 1040 onto the memory 1030 and executes the program module, thereby realizing a function corresponding to the program module. For example, the processor 1020 reads a program module corresponding to the attribute information acquisition unit 110 onto the memory 1030 and executes the program module, thereby realizing the function of the attribute information acquisition unit 110 described in the present disclosure. Similarly, the processor 1020 reads a program module corresponding to the designated input reception unit 120 onto the memory 1030 and executes the program module, thereby realizing the function of the designated input reception unit 120 described in the present disclosure. Similarly, the processor 1020 reads a program module corresponding to the related person identification unit 130 onto the memory 1030 and executes the program module, thereby realizing the function of the related person identification unit 130 described in the present disclosure. Similarly, the processor 1020 reads a program module corresponding to the display output unit 140 onto the memory 1030 and executes the program module, thereby realizing the function of the display output unit 140 described in the present disclosure. The operation of the processor 1020 is common to each embodiment included in the present disclosure.
[0029] The above-described program modules may be recorded in a recording medium other than the storage device 1040. The recording medium on which the program modules are recorded includes any non-transitory tangible medium usable by the computer 1000. The recording medium on which the program modules are recorded may have embedded therein a program code readable by the computer 1000 (processor 1020).
[0030] The input / output interface 1050 is an interface for connecting the computer 1000 (image analyzing device 10) to various input / output devices. For example, an output device such as a display 30 is connected to the computer 1000 (image analyzing device 10) via the input / output interface 1050.
[0031] The network interface 1060 is an interface for connecting the computer 1000 (image analyzing device 10) to a network. This network includes, for example, a local area network (LAN) and a wide area network (WAN). The network interface 1060 connects the computer 1000 (image analyzing device 10) to various networks by wireless or wired connection. The computer 1000 (image analyzing device 10) can communicate with, for example, the database 20 via the network interface 1060.
[0032] Furthermore, the computer 1000 (image analyzing device 10) can be connected to various devices (not shown) via the input / output interface 1050 or the network interface 1060. The "various devices" referred to here are not particularly limited, but include, for example, input devices such as operation buttons, keyboards, touch panels, and mice, and output devices such as speakers, microphones, and printers.
[0033] FIG. 5 is a flowchart illustrating the processing executed by the image analyzing device 10 according to the second embodiment.
[0034] First, the attribute information acquisition unit 110 acquires a plurality of images (e.g., video data) to be processed and analyzes each image (step S102). As a result of the analysis process of each image, the attribute information acquisition unit 110 acquires attribute information of each person for each image.
[0035] As an example, the attribute information acquisition unit 110 acquires a face image of each person or a face feature amount based on the face image, a whole body image of each person or skeletal information based on the whole body image, etc. as attribute information of each person. In this case, the attribute information acquisition unit 110 first identifies an image area (hereinafter also referred to as a "person area") corresponding to each person appearing in a plurality of images to be processed. The attribute information acquisition unit 110 generates skeletal information of each person from the person area identified for each person. The attribute information acquisition unit 110 can generate skeletal information from the person area using a known image processing algorithm (for example, an image processing algorithm that identifies a person's joints, etc. from an image of a person). The attribute information acquisition unit 110 can further extract a face area (hereinafter also referred to as a "face area") from the person area. The attribute information acquisition unit 110 can extract the face area of each person using a known face recognition algorithm. In this case, the attribute information acquisition unit 110 may generate face feature amount information from the face area of each person. Then, the attribute information acquiring unit 110 acquires the information obtained about each person (such as face area, face feature information, and skeletal information) as attribute information of each person. The attribute information of each person acquired by the attribute information acquiring unit 110 is collated with information about each person in the database 20. As a result of the collation, each person appearing in each image is identified. The identification results of each person are stored in a storage area of the image analyzing device 10, such as the memory 1030 or the storage device 1040.
[0036] The image analyzing device 10 may further have a function of generating a list of identified persons based on the identification results of each person, and outputting the list to the display 30. The image analyzing device 10 may further have a function of performing a process of adding information indicating the area of each person (e.g., a rectangular frame) to each image, and outputting a screen including the processed image to the display 30.
[0037] Next, the designated input receiving unit 120 acquires a person designation input (step S104). For example, the designated input receiving unit 120 receives an input operation of information identifying the designated person via an input device connected to the input / output interface 1050. The input operation includes, for example, an operation of selecting the designated person from a person list displayed on the display 30, an operation of selecting an area corresponding to the designated person within an image displayed on the display 30, and the like. The designated input receiving unit 120 may also receive another image (query image) related to the designated person as an input.
[0038] Next, the related person identification unit 130 identifies related people to the designated person indicated by the person designation input (step S106). The related person identification unit 130 identifies related people to the designated person by using attribute information of each person acquired from each image in the process of step S102.
[0039] For example, the related person identification unit 130 can identify a related person as follows. First, the related person identification unit 130 calculates at least one of the time spent together with the designated person and the number of times the person appears together with the designated person based on the attribute information of each person acquired from each image. For example, it is assumed that the attribute information acquisition unit 110 processes video data (multiple images) shot at 30 fps, and as a result, attribute information of a first person corresponding to the designated person and attribute information of a second person different from the designated person are simultaneously acquired in three sections in the video. It is also assumed that the three sections are a first section (300 frames), a second section (900 frames), and a third section (600 frames). In this case, the related person identification unit 130 can calculate the time the first person and the second person are together by adding up the times corresponding to each section. That is, the related person identification unit 130 can calculate the time during which the first person (designated person) and the second person are together as "60 seconds" by adding up the time "10 seconds" corresponding to the first section, the time "30 seconds" corresponding to the second section, and the time "20 seconds" corresponding to the third section. The related person identification unit 130 can also calculate the number of sections in the video, "3", as the number of times the first person (designated person) and the second person appear together. The related person identification unit 130 then compares at least one of the time the second person is together with the first person (designated person) and the number of times the second person appears together with the first person (designated person), which are calculated for the second person, with a criterion set in advance for determining a related person. The criterion defines at least one of the reference time for determining a related person and the number of times the second person appears. The criterion is also stored in advance in a storage area accessible by the related person identification unit 130, such as the storage device 1040. For example, if the length of time calculated for the second person is equal to or greater than a preset reference time, the associated person identification unit 130 may identify the second person as an associated person. Also, for example, if the number of appearances calculated for the second person is equal to or greater than a preset reference number, the associated person identification unit 130 may identify the second person as an associated person.Furthermore, for example, if both the time and the number of occurrences calculated for a second person exceed the respective set criteria, the associated person identification section 130 may identify the second person as an associated person.
[0040] Next, the display output unit 140 generates personal relationship information and organization information indicating the relationship between the specified person and related people (step S108). Then, the display output unit 140 outputs a screen including the personal relationship information and organization information to the display 30 (step S110). For example, the display output unit 140 performs the following operation.
[0041] First, the display output unit 140 reads out from the database 20 organizational information of the designated person specified by the person designation input received by the designation input receiving unit 120. The display output unit 140 also reads out from the database 20 organizational information of the person specified as the related person by the related person identifying unit 130. Here, it is assumed that the organizational information of the designated person indicates a first organization, and the organizational information of the related person indicates a second organization different from the first organization. In this case, the display output unit 140 acquires an organizational chart (first organizational chart) of the first organization to which the designated person belongs, and an organizational chart (second organizational chart) of the second organization to which the related person belongs. Note that data on the organizational charts of each organization is stored in advance in a storage area accessible by the display output unit 140, such as the storage device 1040. After acquiring the data of each of the first organizational chart and the second organizational chart from the storage area, the display output unit 140 generates drawing data of a screen including the first organizational chart and the second organizational chart as the above-mentioned organizational information.
[0042] Furthermore, the display output unit 140 generates personal relationship information indicating the relationship between the designated person and the related person using the first organizational chart and the second organizational chart. For example, the display output unit 140 identifies a part of the first organizational chart corresponding to the designated person (a position on the organizational chart determined by a job title, a rank, etc.) based on the organizational information acquired about the designated person. Also, the display output unit 140 identifies a part of the second organizational chart corresponding to the related person (a position on the organizational chart determined by a job title, a rank, etc.) based on the organizational information acquired about the related person. Then, the display output unit 140 generates personal relationship information between the two organizational charts using the position of the designated person on the first organizational chart and the position of the related person on the second organizational chart. The display output unit 140 combines the generated person identification information with the drawing data of the screen and outputs it to the display 30.
[0043] As an example, the display output unit 140 outputs a screen as illustrated in FIG. 6 to the display 30. FIG. 6 is a diagram showing an example of a screen output by the display output unit 140. The screen illustrated in this figure includes a line that associates the position of the designated person (person H) with the positions of related people (persons C and F) between the first organizational chart and the second organizational chart. Such a line visualizes the relationship between the designated person and the related people. For example, in the example of this figure, the state in which person C and person F are identified as people related to the designated person H is visualized by two lines L1 and L2. By utilizing such a screen, it is possible to easily grasp the relationship between the organizations of the designated person and the related people identified from the analysis result of the input video.
[0044] As another example, the display output unit 140 outputs a screen as illustrated in FIG. 7 to the display 30. FIG. 7 is a diagram showing another example of a screen output by the display output unit 140. The screen shown in this figure includes a star-shaped mark as a common mark that associates the designated person with the related person between the first organizational chart and the second organizational chart. The common mark may be any mark as long as the corresponding relationship between the designated person and the related person can be identified. In this way, the relationship between the designated person and the related person is visualized by giving a common mark to the designated person and the related person. For example, in the example of this figure, a state in which person C and person F are identified as people related to the designated person H is visualized by a star-shaped mark given to the position of each person. By utilizing such a screen, the relationship between the organizations of the designated person and the related person identified from the analysis result of the input video can be easily understood.
[0045] As another example, the display output unit 140 outputs a screen as exemplified in FIG. 8 to the display 30. FIG. 8 is a diagram showing another example of a screen output by the display output unit 140. For the screen exemplified in this figure, the display output unit 140 sets a part corresponding to the designated person in the first organizational chart and a second part corresponding to the related person in the second organizational chart to a common display mode (common background pattern). The common display mode is arbitrary as long as the corresponding relationship between the designated person and the related person can be identified. By utilizing such a screen, the relationship between the organizations of the designated person and the related person identified from the analysis result of the input video can be easily understood.
[0046] Furthermore, the display output unit 140 may be configured to output to the display 30 a screen that combines two or more of the display examples of FIG.
[0047] As described above, according to this embodiment, a screen including personal relationship information indicating the relationship between the designated person and the related person, and organizational information regarding the organization to which the designated person and the related person belong, is output to the display 30. Such a screen allows a person (operator) who is responsible for checking the image to understand the relationship between the designated person and the related person in more detail. For example, the operator can easily understand the chain of command within and between organizations by checking the screen output by the display output unit 140. In addition, for example, if the operator is a person involved in the investigation, the screen output by the display output unit 140 can improve the efficiency of a group crime investigation by multiple members.
[0048] Modification of the second embodiment As a first modified example, the display output unit 140 may be configured to output additional information corresponding to the selected personal relationship information when an input for selecting personal relationship information is received on a screen such as those illustrated in FIGS. 6 to 8. As an example, the display output unit 140 receives an input for selecting a portion on the screen corresponding to the personal relationship information. Then, the display output unit 140 identifies a designated person and related people indicated by the selected personal relationship information. Then, the display output unit 140 identifies one or more images in which the designated person and related people appear together from among the multiple images. Then, the display output unit 140 causes the identified one or more images to be displayed on the display 30.
[0049] The operation of the image analyzing device 10 (display output unit 140) according to the first modified example will be described with reference to the drawings. FIG. 9 and FIG. 10 are diagrams for explaining the operation of the image analyzing device 10 (display output unit 140) according to the first modified example of the second embodiment. First, the display output unit 140 accepts an input for selecting a portion (for example, line L1 / line L2) corresponding to the person relationship information on the screen displayed on the display 30 as exemplified in FIG. 9. In this figure, a state in which the line L1 corresponding to the person relationship information indicating the relationship between the designated person (person H) and the first related person (person C) is selected is depicted. In response to the selection input as exemplified in FIG. 9, the display output unit 140 further outputs, for example, a screen as exemplified in FIG. 10. The screen of FIG. 10 includes an image in which both person H and person C are shown, and information indicating the area of each person in the image. Note that the image included in the screen exemplified in FIG. 10 is a representative image selected from a plurality of images processed by the attribute information acquisition unit 110. For example, the display output unit 140 can select an image in which each person is shown most prominently as a representative image. Also, as exemplified in Fig. 10, the display output unit 140 may further include a button on the screen for switching to another image. Also, the image included on the screen exemplified in Fig. 10 may be partial video data obtained by cutting out a portion of the original video data in which each person is shown together.
[0050] According to the first modification, a person checking the video can easily view an actual image in which the designated person and related people appear together.
[0051] As a second modified example, the display output unit 140 may be configured to output an element for changing the criteria used to identify an associated person in the associated person identification unit 130 on the screen. In this case, the display output unit 140 outputs, for example, a screen as shown in FIG. 11 to the display 30. FIG. 11 is a diagram for explaining the operation of the image analyzing device 10 (the associated person identification unit 130 and the display output unit 140) according to the second modified example of the second embodiment. The screen illustrated in FIG. 11 includes an element E for setting and changing the criteria for identifying an associated person in the upper right portion. The user performs an input on the element E to change the criteria for at least one of the time and the number of times, for example, via an input device connected to the input / output interface 1050.
[0052] When an input is made to change the criteria in element E illustrated in FIG. 11, the related person identification unit 130 updates the identification result of the related person for the designated person based on the new criteria changed by the input. For example, in the screen illustrated in FIG. 11, two related people (person C and person F) are identified for the designated person (person H). Here, it is assumed that the time that person C was with person H and the number of times that person C appeared with person H are calculated as "100 seconds" and "3 times" based on the processing result of each image by the attribute information acquisition unit 110 (attribute information of each person acquired from each image). Similarly, it is assumed that the time that person F was with person H and the number of times that person F appeared with person H are calculated as "200 seconds" and "5 times", respectively. Then, it is assumed that an input is made to change the reference time to "200 seconds" or the reference number of times to "5 times" in element E of the screen illustrated in FIG. 11. In this case, person C will no longer satisfy the new criteria changed by the input. As a result, the associated person identification unit 130 excludes person C from the associated persons and updates the screen display, for example, as shown in Fig. 12. Fig. 12 is a diagram illustrating an example of a screen updated in response to a criterion change input. Unlike the screen in Fig. 11, the screen in Fig. 12 only retains person relationship information (line L2) indicating the relationship between person H (designated person) and person F (associated person).
[0053] As explained in the second modified example, by making the criteria for identifying related persons changeable, it is possible to more flexibly identify persons related to a specified person.
[0054] As a third modified example, the related person identification unit 130 may be configured to further use the "distance between people" to identify a related person. In this case, the attribute information acquisition unit 110 further acquires, for example, the position of each person (the position in the image coordinate system, or the position in the real world coordinate system corresponding to the position in the image coordinate system) as an analysis result of the image to be processed. The attribute information acquisition unit 110 can use various known methods as a method for calculating the position of an object appearing in an image. Then, the related person identification unit 130 can calculate the distance between the designated person and each of the other people based on information on the position of each person acquired from each image. For example, the related person identification unit 130 calculates the distance to the designated person (average distance) based on the position information acquired from each image for each person who is with the designated person. Then, the related person identification unit 130 identifies the person as a "related person" if the calculated distance for a certain person satisfies a predetermined criterion (for example, the calculated distance is equal to or less than the criterion distance).
[0055] In this case, the display output unit 140 may be configured to output a screen as exemplified in FIG. 13. FIG. 13 is a diagram for explaining the operation of the image analyzing device 10 (the associated person identification unit 130 and the display output unit 140) according to the third modified example of the second embodiment. The screen exemplified in FIG. 13 includes an element E for setting and changing the criteria for identifying an associated person in the upper right portion. Unlike the diagrams (FIGS. 11 and 12) according to the second modified example, the element E in this diagram is configured to be able to accept an input for changing the criteria for "distance". For example, the user performs an input on the element E via an input device connected to the input / output interface 1050 to change the criteria for at least one of the time, the number of times, and the distance.
[0056] When an input is made to change the criteria in element E illustrated in FIG. 13, the related person identification unit 130 updates the identification result of related people for the designated person based on the new criteria changed by the input. For example, in the screen illustrated in FIG. 13, two related people (person C and person F) are identified for the designated person (person H). Here, based on the processing result of each image by the attribute information acquisition unit 110 (attribute information of each person acquired from each image), the time that person C was with person H, the number of times that person C appeared with person H, and the distance when person C was with person H are calculated as "100 seconds," "3 times," and "0.5 m," respectively. Similarly, the time that person F was with person H, the number of times that person F appeared with person H, and the distance when person F was with person H are calculated as "200 seconds," "5 times," and "1.0 m," respectively. Then, it is assumed that an input is made to change the reference distance to "0.5 m" in element E of the screen illustrated in FIG. 13. In this case, person F no longer satisfies the new criteria changed by the input. As a result, the associated person identification unit 130 excludes person F from the associated persons. As a result, the display output unit 140 updates the screen display, for example, as shown in FIG. 14. FIG. 14 is a diagram illustrating an example of a screen updated in response to the criteria change input. Unlike the screen in FIG. 13, the screen in FIG. 14 only retains person relationship information (line L1) indicating the relationship between person H (designated person) and person C (associated person).
[0057] According to the third modified example, a new index called "distance between persons" is used to accurately identify persons who are likely to be related to a specified person.
[0058] The image analyzing device 10 may have a combination of the configuration according to the first modified example and the configuration according to the second modified example or the configuration according to the third modified example.
[0059] Third embodiment Fig. 15 is a diagram illustrating an example of a system configuration of an image analysis system 1 according to a third embodiment. In the example of this figure, the database 20 includes a posture information database 22. The posture information database 22 stores information on various postures that a person can take, for example, as shown in Fig. 16. Fig. 16 is a diagram illustrating an example of information stored in the posture information database. The posture information database 22 illustrated in this figure stores a plurality of posture information including posture images, posture feature amounts (skeletal information), and posture names.
[0060] The image analyzing device 10 according to this embodiment has the same functional configuration (eg, FIG. 1) as the first and second embodiments, except for the points described below.
[0061] The attribute information acquisition unit 110 according to this embodiment further specifies the posture or behavior of each person using the skeletal information of each person appearing in the image to be processed. For example, the attribute information acquisition unit 110 compares skeletal information that can be extracted from an image area of a certain target person with skeletal information of various postures registered in the posture information database 22 (hereinafter, also referred to as "registered skeletal information"). Based on the result of the comparison, the attribute information acquisition unit 110 specifies registered skeletal information with the highest similarity from among the multiple registered skeletal information. Then, the attribute information acquisition unit 110 obtains the name of the posture or behavior (e.g., "handing over" or "sitting") corresponding to the specified registered skeletal information as information indicating the posture or behavior of the target person. The attribute information acquisition unit 110 executes such processing for each image, and can specify the posture or behavior of each person appearing in each image. Then, the attribute information acquisition unit 110 adds information indicating the posture or behavior specified for each person appearing in each image to the attribute information of that person. Furthermore, in this embodiment, the associated person identification unit 130 is configured to be able to narrow down associated people for a specified person based on the posture and behavior of each person included in the attribute information of that person.
[0062] The operation of the image analyzing device 10 according to this embodiment will be described with reference to Fig. 17 to Fig. 19. Fig. 17 to Fig. 19 are diagrams for explaining the operation of the image analyzing device 10 (the associated person identifying section 130 and the display output section 140) according to the third embodiment.
[0063] First, the associated person identification unit 130 accepts an input specifying a specific posture or behavior. In this case, the display output unit 140 displays a screen as exemplified in FIG. 17, for example, in order to accept the input specifying the posture or behavior. The screen exemplified in FIG. 17 has an element E for setting and changing the criteria for identifying an associated person in the upper right portion. Unlike the screen shown in the example described above, the element E in this figure is configured to be able to accept an input for changing the criteria related to "posture / behavior". For example, the user can make an input for changing the criteria related to posture or behavior on the element E via an input device connected to the input / output interface 1050.
[0064] Here, as illustrated in FIG. 18, it is assumed that a specific posture (behavior) of "handing over (some object)" is specified by the user. In this case, the related person identification unit 130 uses attribute information of the specified person acquired from each image to identify an image in which the specified person takes the "handing over" posture. For example, the related person identification unit 130 can identify an image as shown in FIG. 19. In the image illustrated in FIG. 19, the specified person (person H) takes the "handing over" posture. In addition, in the image illustrated in FIG. 19, "person C" is shown as another person. The related person identification unit 130 narrows down the other person ("person C" in the example of FIG. 19) who appears together in the image thus identified as a related person. As a result, person F is excluded from the related person. In response to this result, the display output unit 140 updates the screen display as shown in FIG. 18. 18, unlike the screen of FIG. 17, only the person relationship information (line L1) indicating the relationship between person H (designated person) and person C (related person) remains.
[0065] Using the new indicator of "person posture," it is possible to accurately identify people who are likely to be related to a specified person.
[0066] In this embodiment, the associated person identification unit 130 may further narrow down the associated persons in detail using a criterion related to "distance", as described in the modified example of the second embodiment.
[0067] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.
[0068] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the figures can be changed to the extent that does not cause problems in terms of content. In addition, each of the above-mentioned embodiments can be combined to the extent that the content is not contradictory.
[0069] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing a plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person by using the attribute information acquired for each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; An image analysis device comprising: 2. The display output means includes: A first organizational chart of the organization to which the designated person belongs and a second organizational chart of the organization to which the related person belongs are included on the screen as the organizational information; generating the personal relationship information using the first organizational chart and the second organizational chart; 1. An image analysis device according to claim 1. 3. the display output means generates, as the person relationship information, a line that associates the designated person with the related person between the first organizational chart and the second organizational chart; 2. An image analysis device according to claim 1. 4. the display output means generates, as the person relationship information, a common mark that associates the designated person with the related person between the first organizational chart and the second organizational chart; 2. An image analysis device according to claim 1. 5. the display output means generates the person relationship information by setting a first portion in the first organizational chart corresponding to the designated person and a second portion in the second organizational chart corresponding to the related person in a common display mode. 2. An image analysis device according to claim 1. 6. the display output means, in response to receiving an input for selecting a portion corresponding to the person relationship information on the screen, displays one or more images in which the designated person and the related person indicated by the selected person relationship information are both shown. 6. An image analysis device according to any one of 2. to 5. 7. The related person identification means includes: calculating, for each person, at least one of a time spent together with the designated person and a number of times the person has appeared together with the designated person based on the attribute information of each of the people acquired from each image; identifying the related person for the specified person based on a result of comparing at least one of the time and the number of times calculated for each person with a preset standard for at least one of the time and the number of times; 7. An image analysis device according to any one of 1 to 6. 8. the related person identification means, in response to receiving a criterion change input for changing the criterion, updates the identification result of the related person for the specified person based on the new criterion changed by the criterion change input. 7. An image analysis device according to claim 7. 9. The attributes of the person include attributes related to physical characteristics of the person. 9. An image analysis device according to any one of 1 to 8. 10. At least one computer By processing the multiple images, attribute information indicating attributes of each person appearing in each image is obtained; Accepting a person designation input indicating a designated person; Identifying a related person who is a person related to the specified person using the attribute information acquired for each of the persons; outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; The image analysis method includes: 11. The at least one computer: A first organizational chart of the organization to which the designated person belongs and a second organizational chart of the organization to which the related person belongs are included on the screen as the organizational information; generating the personal relationship information using the first organizational chart and the second organizational chart; The image analysis method according to claim 10, further comprising: 12. The at least one computer: generating a line that associates the designated person with the related person between the first organizational chart and the second organizational chart as the person relationship information; 12. The image analysis method according to claim 11, further comprising: 13. The at least one computer: generating a common mark that associates the designated person with the related person between the first organizational chart and the second organizational chart as the person relationship information; 12. The image analysis method according to claim 11, further comprising: 14. The at least one computer: generating the person relationship information by setting a first portion in the first organizational chart corresponding to the specified person and a second portion in the second organizational chart corresponding to the related person in a common display mode; 12. The image analysis method according to claim 11, further comprising: 15. The at least one computer: displaying one or more images showing both the designated person and the related person indicated by the selected personal relationship information in response to receiving an input of selecting a portion corresponding to the personal relationship information on the screen; 15. The image analysis method according to any one of 11 to 14, further comprising: 16. The at least one computer: calculating, for each person, at least one of a time spent together with the designated person and a number of times the person has appeared together with the designated person based on the attribute information of each of the people acquired from each image; identifying the related person for the specified person based on a result of comparing at least one of the time and the number of times calculated for each person with a preset standard for at least one of the time and the number of times; 16. The image analysis method according to any one of 10. to 15., 17. The at least one computer: In response to receiving a criterion change input for changing the criterion, updating the identification result of the related person for the designated person based on the new criterion changed by the criterion change input. 17. The image analysis method according to claim 16, further comprising: 18. The attributes of the person include attributes related to physical characteristics of the person. 18. An image analysis method according to any one of 10. to 17. 19. At least one computer an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing the plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person, using the attribute information acquired about each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; A program to function as a 20. The display output means includes: A first organizational chart of the organization to which the designated person belongs and a second organizational chart of the organization to which the related person belongs are included on the screen as the organizational information; generating the personal relationship information using the first organizational chart and the second organizational chart; 19. The program described in. twenty one. the display output means generates, as the person relationship information, a line that associates the designated person with the related person between the first organizational chart and the second organizational chart; 20. The program described in. twenty two. the display output means generates, as the person relationship information, a common mark that associates the designated person with the related person between the first organizational chart and the second organizational chart; 20. The program described in. twenty three. the display output means generates the person relationship information by setting a first portion in the first organizational chart corresponding to the designated person and a second portion in the second organizational chart corresponding to the related person in a common display mode. 20. The program described in. twenty four. the display output means, in response to receiving an input for selecting a portion corresponding to the person relationship information on the screen, displays one or more images in which the designated person and the related person indicated by the selected person relationship information are both shown. 2. A program according to any one of 20. to 23. twenty five. The related person identification means includes: calculating, for each person, at least one of a time spent together with the designated person and a number of times the person has appeared together with the designated person based on the attribute information of each of the people acquired from each image; identifying the related person for the specified person based on a result of comparing at least one of the time and the number of times calculated for each person with a preset standard for at least one of the time and the number of times; 25. The program according to any one of 19. to 24. 26. the related person identification means, in response to receiving a criterion change input for changing the criterion, updates the identification result of the related person for the specified person based on the new criterion changed by the criterion change input. 25. The program described in. 27. The attributes of the person include attributes related to physical characteristics of the person. 27. The program according to any one of 19. to 26. [Explanation of symbols]
[0070] 1. Image analysis system 10. Image analysis device 1000 Computers 1010 Bus 1020 Processor 1030 Memory 1040 Storage Device 1050 Input / Output Interface 1060 Network Interface 110 Attribute information acquisition section 120 Designated input reception section 130 Related Person Identification Division 140 Display output section 20 Database 22 Posture Information Database 30 Display 40 Network
Claims
1. an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing a plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person by using the attribute information acquired for each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; An image analysis device comprising:
2. The display output means includes: a first organizational chart of an organization to which the designated person belongs and a second organizational chart of an organization to which the related person belongs are included on the screen as the organizational information; generating the personal relationship information using the first organizational chart and the second organizational chart; The image analysis device according to claim 1 .
3. the display output means generates, as the person relationship information, a line that associates the designated person with the related person between the first organizational chart and the second organizational chart; The image analysis device according to claim 2 .
4. the display output means generates, as the person relationship information, a common mark for associating the designated person with the related person between the first organizational chart and the second organizational chart; The image analysis device according to claim 2 .
5. the display output means generates the person relationship information by setting a first portion in the first organizational chart corresponding to the designated person and a second portion in the second organizational chart corresponding to the related person in a common display mode. The image analysis device according to claim 2 .
6. the display output means, in response to receiving an input for selecting a portion corresponding to the person relationship information on the screen, displays one or more images in which the designated person and the related person indicated by the selected person relationship information are both shown. The image analysis device according to claim 2 .
7. The related person identification means includes: calculating, for each person, at least one of a time spent together with the designated person and a number of times the person has appeared together with the designated person based on the attribute information of each of the people acquired from each image; identifying the related person for the specified person based on a result of comparing at least one of the time and the number of times calculated for each person with a preset standard for at least one of the time and the number of times; The image analysis device according to claim 1 .
8. the related person identification means, in response to receiving a criterion change input for changing the criterion, updates the identification result of the related person for the specified person based on the new criterion changed by the criterion change input. The image analysis device according to claim 7.
9. The attributes of the person include attributes related to physical characteristics of the person. The image analysis device according to claim 1 .
10. At least one computer By processing the multiple images, attribute information indicating attributes of each person appearing in each image is obtained; Accepting a person designation input indicating a designated person; Identifying a related person who is a person related to the specified person using the attribute information acquired for each of the persons; outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; The image analysis method includes:
11. At least one computer an attribute information acquiring means for acquiring attribute information indicating attributes of each person appearing in each image by processing the plurality of images; A designated person input receiving means for receiving a person designation input indicating a designated person; a related person identification means for identifying a related person who is a person related to the specified person, using the attribute information acquired about each of the persons; a display output means for outputting a screen including personal relationship information indicating a relationship between the designated person and the related person, and organizational information regarding organizations to which the designated person and the related person each belong; A program to function as a
Citation Information
Patent Citations
Method and system for detecting suspicious person
JP2010191620A
Information processing system, administrative server and information processing method
JP2013045160A
Human relationship estimation system
JP2013182560A
Personal correlation chart providing device and personal correlation chart providing system
JP2018200663A
METHOD, IDENTIFICATION DEVICE AND IDENTIFICATION PROGRAM FOR DISCOVERING MULTI-LAYER POTENTIAL MATE - Patent application
JP2021536182A
