Information processing apparatus, information processing system, information processing method, and program

Abstract

To accurately track a moving body imaged by a plurality of cameras.SOLUTION: An information processing device according to an embodiment performs information processing from images captured by a plurality of cameras. The information processing apparatus includes an association unit, an attribute information acquisition unit, a movement information generation unit, and an associated information generation unit. When a moving object detected from a first image captured by the first camera and a moving object detected from a second image captured by the second camera are estimated to be the same, the association unit associates the moving object detected from the first image with the moving object detected from the second image. The attribute information acquisition unit acquires attribute information related to a property or a feature of the moving object from the image. The movement information generation unit generates movement information on movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image. The association information generation unit statistically processes the movement information according to the attribute information and generates inter-camera association information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] A program is known that uses multiple cameras to track or locate a moving object, such as a person, animal, or robot, by calculating in advance the relationship between the two cameras, such as the travel time of the moving object between the two cameras. The conventional program detects the moving object from video image data captured by each of the two cameras. If a feature of an image portion of the moving object captured by one of the two cameras matches a feature of an image portion of the moving object captured by the other of the two cameras, the conventional program associates the moving object captured by one camera with the moving object captured by the other camera as being identical. The conventional program then calculates the relationship between the two cameras based on the position and time when the associated moving object exits the imaging range of one camera and the position and time when it enters the imaging range of the other camera.

[0003] The movement behavior of a moving object, such as its speed and route, varies depending on its characteristics, and for example, there are individual differences in the case of a person. Furthermore, the speed and route of a moving object may change due to changes in the environment along the moving route, such as the presence of an obstacle on the route, a wet and slippery portion of the route, or congestion on the route. For this reason, when tracking or locating a moving object using multiple cameras installed on or around the route, accurate results may not be obtained. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5983033 Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to enable tracking or position search with higher accuracy of a moving object captured by multiple cameras. [Means for solving the problem]

[0006] An information processing device according to an embodiment processes information from images captured by multiple cameras. The information processing device includes a matching unit, an attribute information acquisition unit, a movement information generation unit, and an association information generation unit. When a moving object detected in a first image captured by a first camera and a moving object detected in a second image captured by a second camera are estimated to be the same, the matching unit matches the moving object detected in the first image with the moving object detected in the second image. The attribute information acquisition unit acquires attribute information relating to the nature or characteristics of the moving object from the images. The movement information generation unit generates movement information relating to the movement of the associated moving object from the imaging range of the first camera related to the first image to the imaging range of the second camera related to the second image. The association information generation unit statistically processes the movement information according to the attribute information to generate inter-camera association information relating to the moving object. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a configuration diagram of an information processing system according to a first embodiment. [Figure 2] FIG. 4 is a diagram showing inter-camera association information for each attribute according to the first embodiment. [Figure 3] FIG. 1 is a diagram showing the configuration of a generation device according to a first embodiment. [Figure 4] 4 is a flowchart showing the flow of processing by the generation device according to the first embodiment. [Figure 5] FIG. 10 is a configuration diagram of an information processing system according to a second embodiment. [Figure 6]FIG. 11 is a diagram showing inter-camera association information for each situation according to the second embodiment. [Figure 7] FIG. 10 is a diagram showing the configuration of a generating device according to a second embodiment. [Figure 8] 10 is a flowchart showing the flow of processing by a generation device according to a second embodiment. [Figure 9] FIG. 10 is a configuration diagram of an information processing system according to a third embodiment. [Figure 10] 13A and 13B are diagrams showing inter-camera association information for each attribute and each situation according to the third embodiment. [Figure 11] FIG. 11 is a diagram showing the configuration of a generating device according to a third embodiment. [Figure 12] 10 is a flowchart showing the flow of processing by a generation device according to a third embodiment. [Figure 13] 10 is a flowchart showing the flow of processing by a generation device according to a fourth embodiment. [Figure 14] FIG. 4 is a diagram showing a first display image. [Figure 15] FIG. 10 is a diagram showing a second display image. [Figure 16] FIG. 1 is a hardware configuration diagram of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0008] (First embodiment) FIG. 1 is a diagram showing the configuration of an information processing system 10 according to the first embodiment.

[0009] The information processing system 10 includes a plurality of cameras 12 and an information processing device 20. The information processing system 10 performs analysis of the movement of people, such as detecting or tracking people captured by the plurality of cameras 12. For example, the information processing system 10 performs a detection process to detect all or some of the people in the image, and a tracking process to identify the movement positions of all or some of the detected people over time, to determine whether people are entering or exiting a venue. Note that a person is an example of a moving object, and the information processing system 10 may perform analysis of the movement of moving objects, such as detecting or tracking moving objects captured by the plurality of cameras 12.

[0010] The multiple cameras 12 include, for example, a first camera 12-1, a second camera 12-2, ..., an N-th camera 12-N (N is an integer equal to or greater than 2). Each of the multiple cameras 12 is arranged with a fixed imaging range and imaging direction. Each of the multiple cameras 12 generates video data capturing an image of a person present in the imaging range.

[0011] The multiple cameras 12 are installed at different positions or angles, and have different imaging ranges. For example, the second camera 12-2 is installed at a different position or angle from the first camera 12-1. Each of the multiple cameras 12 may be installed indoors or outdoors. Furthermore, the imaging range of each of the multiple cameras 12 may partially overlap with the imaging range of the other cameras 12.

[0012] The information processing device 20 includes an analysis device 22 , a video storage unit 24 , an information storage unit 26 , and a generation device 30 .

[0013] Analysis device 22 acquires video data captured by each of the multiple cameras 12. Analysis device 22 writes the acquired video data into video storage unit 24, distinguishing between the data captured by each of the multiple cameras 12. Analysis device 22 then analyzes the movement of people based on the video data captured by each of the multiple cameras 12. Analysis device 22 may perform the analysis in real time, or may perform the analysis offline based on the video data stored in video storage unit 24.

[0014] The analysis device 22 outputs the analysis results of the analysis of the person's movement to an external device. The analysis device 22 may also display the analysis results on a display device.

[0015] The video image storage unit 24 stores the video image data acquired by the analysis device 22 separately for each of the multiple cameras 12.

[0016] The information storage unit 26 stores, for each pair of two cameras 12 included in the plurality of cameras 12, inter-camera association information for each person attribute such as gender and age.

[0017] The generation device 30 generates inter-camera association information for each attribute based on video image data for each of the multiple cameras 12 stored in the video image storage unit 24. The generation device 30 generates inter-camera association information for each attribute for each pair of two cameras 12 included in the multiple cameras 12. The pair of two cameras 12 is an ordered pair in which a person who exits the imaging range of one camera 12 may enter the imaging range of the other camera 12.

[0018] For example, if there is a possibility that a person who has left the imaging range of first camera 12-1 will enter the imaging range of second camera 12-2, generation device 30 generates inter-camera association information for each attribute by treating first camera 12-1 and second camera 12-2 as an ordered pair of two cameras 12. Note that the inter-camera association information for each attribute will be described in further detail with reference to FIG. 2.

[0019] The generating device 30 stores the generated inter-camera association information for each attribute in the information storage unit 26. Furthermore, the generating device 30 may output the generated inter-camera association information for each attribute to an external device or display it on a display device.

[0020] Here, the analysis device 22 identifies the attributes of the person being analyzed, and performs an analysis of the movement of the person being analyzed using the inter-camera association information for the identified attributes from the inter-camera association information for each attribute stored in the information storage unit 26.

[0021] For example, in analyzing the movement of a person, analysis device 22 performs a correlation process to correlate a person who has left the imaging range of first camera 12-1 with a person who has entered the imaging range of second camera 12-2 using feature information and the similarity thereof extracted from the appearances, movements, etc. of those people. In this case, analysis device 22 performs the correlation process by narrowing down candidates, correcting the feature information and the similarity thereof, prioritizing, etc., using the inter-camera association information of a specified attribute out of the inter-camera association information for each attribute corresponding to the pair of first camera 12-1 and second camera 12-2.

[0022] People's travel behavior, such as travel speed or travel route, often differs depending on attributes such as gender and age. Therefore, for example, the travel time and travel route from leaving the imaging range of the first camera 12-1 to entering the imaging range of the second camera 12-2 often differ depending on attributes. If the analysis device 22 uses inter-camera association information generated regardless of attributes, it may be unable to refer to appropriate inter-camera association information due to differences in the person's gender or age, and may be unable to accurately narrow down candidates for the matching process, correct feature information and its similarity, prioritize, etc. In contrast, the analysis device 22 according to the first embodiment uses inter-camera association information for each attribute, thereby enabling accurate narrowing down of candidates for the matching process, correcting feature information and its similarity, prioritizing, etc. Therefore, the analysis device 22 according to the first embodiment can accurately analyze people's movements or movements.

[0023] FIG. 2 is a diagram illustrating an example of inter-camera association information for each attribute according to the first embodiment.

[0024] The inter-camera association information for each attribute is generated for each pair of two cameras 12 included in the plurality of cameras 12. The pairs of two cameras 12 are ordered pairs. For example, the pair of first camera 12-1 and second camera 12-2 representing the pair of the order in which a person who has left the imaging range of first camera 12-1 enters the imaging range of second camera 12-2 is different from the pair of second camera 12-2 and first camera 12-1 representing the pair of the order in which a person who has left the imaging range of second camera 12-2 enters the imaging range of first camera 12-1.

[0025] However, two pieces of inter-camera association information that are in a reverse order pairing relationship may have the same content. In this case, the two pieces of inter-camera association information that are in a reverse order pairing relationship may be commonly described.

[0026] The inter-camera association information for each attribute represents, for a pair of ordered two cameras 12, the relationship between one camera 12 and the other camera 12 when a person moves from the imaging range of one camera 12 to the imaging range of the other camera 12. For example, the inter-camera association information for each attribute for a pair of first camera 12-1 and second camera 12-2 represents the relationship between first camera 12-1 and second camera 12-2 when a person moves from the imaging range of first camera 12-1 to the imaging range of second camera 12-2.

[0027] For example, the inter-camera association information for each attribute includes a value for at least one of the following items: travel time, transition rate, exit position, entry position, exit angle, entry angle, exit direction, and entry direction.

[0028] The travel time represents the time it takes for a person to enter the imaging range of one camera 12 in a pair of two ordered cameras 12. For example, the travel time for a pair of first camera 12-1 and second camera 12-2 represents the time it takes for a person to exit the imaging range of the first camera 12-1 and enter the imaging range of the second camera 12-2.

[0029] The transition rate represents the ratio of the number of people who move from the imaging range of one camera 12 to the number of people who exit from the imaging range of one camera 12 in a pair of two ordered cameras 12. For example, the transition rate for a pair of first camera 12-1 and second camera 12-2 represents the ratio of the number of people who enter the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who exit from the imaging range of first camera 12-1.

[0030] The transition rate may also represent the ratio of the number of people who move from the imaging range of one camera 12 to the imaging range of the other camera 12 in a pair of two ordered cameras 12, relative to the number of people who enter the imaging range of the other camera 12. For example, the transition rate of a pair of first camera 12-1 and second camera 12-2 may represent the ratio of the number of people who enter the imaging range of second camera 12-2 from the imaging range of first camera 12-1.

[0031] The exit position represents the position at which a person exits the imaging range of one camera 12 in a pair of two ordered cameras 12, when the person exits the imaging range of one camera 12 and enters the imaging range of the other camera 12. For example, the exit position of a pair of first camera 12-1 and second camera 12-2 represents the position at which a person exits the imaging range of first camera 12-1 and enters the imaging range of second camera 12-2, when the person exits the imaging range of first camera 12-1 and enters the imaging range of second camera 12-2.

[0032] The entry position represents the position at which a person who leaves the imaging range of one camera 12 in a pair of two ordered cameras 12 enters the imaging range of the other camera 12. For example, the entry position of a pair of first camera 12-1 and second camera 12-2 represents the position at which a person who leaves the imaging range of first camera 12-1 and enters the imaging range of second camera 12-2 enters the imaging range of second camera 12-2.

[0033] The exit angle represents the angle or orientation of the body of a person, when the person exits the imaging range of one camera 12 in a pair of two ordered cameras 12 and enters the imaging range of the other camera 12, relative to the one camera 12 at the time the person exits the imaging range of the one camera 12. For example, the exit angle of a pair of first camera 12-1 and second camera 12-2 represents the angle or orientation of the body of the person, when the person exits the imaging range of the first camera 12-1 and enters the imaging range of the second camera 12-2, relative to the first camera 12-1 at the time the person exits the imaging range of the first camera 12-1.

[0034] The entrance angle represents the angle or orientation of the body of a person who leaves the imaging range of one camera 12 in a pair of two ordered cameras 12 and enters the imaging range of the other camera 12 at the time the person enters the imaging range of the other camera 12. For example, the entrance angle of a pair of a first camera 12-1 and a second camera 12-2 represents the angle or orientation of the body of a person who leaves the imaging range of the first camera 12-1 and enters the imaging range of the second camera 12-2 at the time the person enters the imaging range of the second camera 12-2.

[0035] The exit direction represents the movement direction of a person who leaves the imaging range of one camera 12 in a pair of two ordered cameras 12 and enters the imaging range of the other camera 12 at the time of exiting the imaging range of one camera 12. For example, the exit direction of a pair of first camera 12-1 and second camera 12-2 represents the movement direction of a person who leaves the imaging range of first camera 12-1 and enters the imaging range of second camera 12-2 at the time of exiting the imaging range of first camera 12-1.

[0036] The entry direction represents the movement direction of a person who leaves the imaging range of one camera 12 in a pair of two ordered cameras 12 and enters the imaging range of the other camera 12 at the time of entering the imaging range of the other camera 12. For example, the entry direction of a pair of first camera 12-1 and second camera 12-2 represents the movement direction of a person who leaves the imaging range of first camera 12-1 and enters the imaging range of second camera 12-2 at the time of entering the imaging range of second camera 12-2.

[0037] By using such inter-camera association information for each attribute, the analysis device 22 can perform the association process with higher accuracy.

[0038] Furthermore, the inter-camera association information for each attribute includes a value for each item for each of a plurality of attributes.

[0039] Each of the multiple attributes is person-dependent information that represents the nature or characteristics of the person, such as gender, age group, clothing, social role, presence or absence of specific belongings, belongings, presence or absence of other people accompanying the person, movement, posture, or facial expression.

[0040] The gender may be male or female, etc. The gender may be neither male nor female, or may be unknown.

[0041] The age group may be, for example, under 10 years old, teens, 20s, 30s, etc.

[0042] Clothing includes, for example, wearing sportswear, a business suit, high heels, etc. Social roles include being an event guide, being a police officer, etc. Presence or absence of specific belongings includes having large luggage of a certain size or larger, etc. Possessions include carrying a suitcase, a backpack, etc.

[0043] The presence or absence of other people moving together includes moving in a group of a certain number of people or more, etc. The movement includes running, moving with large arm movements, etc. The posture includes walking with one's head down, etc. The facial expression includes a cheerful expression, a depressed expression, etc.

[0044] The movement behavior of a person, such as the speed of movement and the route of movement, changes depending on these attributes. Therefore, when performing the association process, the analysis device 22 can perform the association more accurately by using inter-camera association information with the same attributes as the attributes of the person to be associated.

[0045] FIG. 3 is a diagram showing the configuration of a generating device 30 according to the first embodiment.

[0046] The generating device 30 generates inter-camera association information for each attribute for a pair of a first camera 12-1 and a second camera 12-2 among the multiple cameras 12. Note that the generating device 30 can also generate inter-camera association information for each attribute with a similar configuration for a pair of two cameras 12 other than the pair of the first camera 12-1 and the second camera 12-2 among the multiple cameras 12 by processing one of the pair of two cameras 12 as the first camera 12-1 and the other as the second camera 12-2.

[0047] The generation device 30 includes a first acquisition unit 32, a second acquisition unit 34, a first person detection unit 36, a second person detection unit 38, a matching unit 40, an attribute information acquisition unit 42, a movement information generation unit 44, a person information generation unit 46, a person information storage unit 48, an association information generation unit 50, an association information storage unit 52, an output unit 54, and a display control unit 56.

[0048] The first acquisition unit 32 acquires the first moving image data captured by the first camera 12-1 from the moving image storage unit 24. The second acquisition unit 34 acquires the second moving image data captured by the second camera 12-2 from the moving image storage unit 24.

[0049] First person detection unit 36 ​​detects people who have left the imaging range of first camera 12-1 and are included in the first moving image data captured by first camera 12-1. Second person detection unit 38 detects people who have entered the imaging range of second camera 12-2 and are included in the second moving image data captured by second camera 12-2.

[0050] The association unit 40 associates each of a plurality of first person images, which detect a person exiting the imaging range of the first camera 12-1, with a second person image that is estimated to include the same person as the corresponding first person image, among a plurality of second person images, which detect a person entering the imaging range of the second camera 12-2 and are included in the second moving image data captured by the second camera 12-2. In this case, the estimation of whether or not the first person image and the second person image are the same person is made based on one or more pieces of information selected from characteristic information extracted from the person's appearance or behavior, the degree of similarity therebetween, and the time and place at which the image was captured.

[0051] For example, the association unit 40 extracts feature information of each of the plurality of first person images and feature information of each of the plurality of second person images, and then compares the feature information of each of the plurality of first person images with the feature information of each of the plurality of second person images to associate the first person images and the second person images that are estimated to include the same person.

[0052] The associating unit 40 may also generate a value representing the reliability of the association, and associate the first person image with the second person image when the value representing the reliability exceeds a preset threshold. For example, the associating unit 40 may determine that the reliability exceeds the threshold when the similarity is greater than the threshold, when the dissimilarity is less than the threshold, when the difference in the image capturing times is within a predetermined range, when a component of a feature amount of a specific element is greater than a threshold, or when at least two of these are combined.

[0053] The plurality of first person images may include a first person image for which no corresponding second person image exists. A person included in a first person image for which no corresponding second person image exists is estimated to have moved from the imaging range of first camera 12-1 to a range different from the imaging range of second camera 12-2.

[0054] The plurality of second person images may also include a second person image for which no corresponding first person image exists. A person included in a second person image for which no corresponding first person image exists is estimated to have moved from a range different from the imaging range of first camera 12-1 into the imaging range of second camera 12-2.

[0055] The attribute information acquisition unit 42 acquires attribute information relating to the nature or characteristics of the moving object from the image. More specifically, the attribute information acquisition unit 42 acquires one or more relevant attributes representing the nature or characteristics of the target person included in the associated first person image and second person image from among a plurality of predetermined attributes. Note that, when there are multiple target people, the attribute information acquisition unit 42 acquires one or more relevant attributes for each of the multiple target people.

[0056] The attribute information acquisition unit 42 may also acquire one or more relevant attributes for a person included in a first person image that is not associated with a second person image. The attribute information acquisition unit 42 may also acquire one or more relevant attributes for a person included in a second person image that is not associated with a first person image.

[0057] For example, the attribute information acquisition unit 42 detects the attributes of a target person included in the associated first person image and second person image by performing image analysis on at least one of the first moving image data and the second image data. Furthermore, the attribute information acquisition unit 42 detects the attributes of a person included in a first person image that is not associated with a second person image by performing image analysis on the first moving image data. Furthermore, the attribute information acquisition unit 42 detects the attributes of a person included in a second person image that is not associated with a first person image by performing image analysis on the second moving image data.

[0058] Furthermore, a person may have a physical tag or the like capable of detecting information indicating attributes. In such a case, attribute information acquisition unit 42 may acquire data of the physical tag or the like read by a reading device provided together with first camera 12-1 or second camera 12-2, and acquire attributes based on the acquired data of the physical tag or the like.

[0059] The movement information generating unit 44 generates movement information regarding the movement of a target person included in the associated first person image and second person image from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2. Note that if there are multiple target people, the movement information generating unit 44 generates movement information for each of the multiple target people.

[0060] The movement information includes information that is the source of the values ​​of each item in the inter-camera association information for each attribute. For example, the movement information includes a value of at least one item from among movement time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction. The movement information generation unit 44 calculates the value of each item for a target person included in the associated first person image and second person image based on the first moving image data and second image data.

[0061] The person information generation unit 46 generates person information including one or more relevant attributes and movement information for a target person included in the associated first person image and second person image. When there are multiple target people, the person information generation unit 46 generates person information for each of the multiple target people.

[0062] In addition, the person information generation unit 46 may generate person information for a person included in a first person image to which no second person image is associated, the person information including one or more relevant attributes and information indicating that the person has left the imaging range of the first camera 12-1 but has not entered the imaging range of the second camera 12-2.

[0063] In addition, the person information generation unit 46 may generate person information for a person included in a second person image to which no first person image is associated, the person information including one or more relevant attributes and information indicating that the person has entered the imaging range of the second camera 12-2 but has not exited the imaging range of the first camera 12-1.

[0064] The person information storage unit 48 stores the person information of each of the plurality of people generated by the person information generation unit 46.

[0065] The association information generation unit 50 statistically processes the movement information according to the attribute information and generates inter-camera association information related to the moving object. More specifically, the association information generation unit 50 generates inter-camera association information for a pair of the first camera 12-1 and the second camera 12-2 for each of a plurality of attributes based on the personal information of each of a plurality of people stored in the personal information storage unit 48.

[0066] For example, the association information generation unit 50 performs statistical calculations for each of the multiple attributes on the values ​​of each of the items, namely, movement time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, contained in the movement information of multiple target persons contained in the associated first person image and second person image, to generate inter-camera association information for each of the multiple attributes.

[0067] For example, the association information generating unit 50 selects one or more target persons who include one or more first attributes among the multiple attributes of the multiple target persons. Next, the association information generating unit 50 calculates the average value, median, standard deviation, or quartile deviation of the values ​​of each of the items, i.e., travel time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, included in the movement information of the selected one or more target persons. The association information generating unit 50 then sets the average value, median, standard deviation, or quartile deviation of the values ​​of each item as the values ​​of each of the items, i.e., travel time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, in the inter-camera association information of the first attribute. The association information generating unit 50 may use other values ​​obtained by statistical calculations as the values ​​of the corresponding items in the inter-camera association information, rather than the average value, median, standard deviation, or quartile deviation of the values ​​of the items included in the movement information of the selected one or more target persons.

[0068] Furthermore, the association information generating unit 50 may calculate, for each of a plurality of attributes, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who exited the imaging range of first camera 12-1, as the value of the transition ratio item in the inter-camera association information. Furthermore, the association information generating unit 50 may calculate, for each of a plurality of attributes, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who entered the imaging range of second camera 12-2, as the value of the transition ratio item in the inter-camera association information.

[0069] Then, the association information generating unit 50 stores the inter-camera association information for each of the multiple attributes in the association information storage unit 52.

[0070] The output unit 54 writes the inter-camera association information for each of the multiple attributes stored in the association information storage unit 52 into the information storage unit 26 as inter-camera association information for each attribute. The display control unit 56 displays the generated inter-camera association information for each of the multiple attributes on a display device or the like in response to, for example, a user operation.

[0071] 4 is a flowchart showing the flow of processing performed by the generating device 30 according to the first embodiment. The generating device 30 according to the first embodiment executes processing according to the flow shown in FIG.

[0072] First, in S11, the generating device 30 detects a person who has entered the image capturing range of the first camera 12-1 and is included in the first moving image data captured by the first camera 12-1.

[0073] Next, in S12, the generation device 30 detects a person who has entered the imaging range of the second camera 12-2 and is included in the second moving image data captured by the second camera 12-2.

[0074] Next, in S13, the generation device 30 associates each of a plurality of first person images, which detect a person exiting the imaging range of the first camera 12-1, with a second person image, which is estimated to include the same person as the corresponding first person image, from among a plurality of second person images, which detect a person entering the imaging range of the second camera 12-2 and are included in the second moving image data captured by the second camera 12-2.

[0075] Next, in S14, the generating device 30 acquires one or more relevant attributes representing the nature or characteristics of the target person included in the associated first person image and second person image from among a plurality of predetermined attributes. The generating device 30 also acquires one or more relevant attributes for the person included in the first person image not associated with the second person image and the person included in the second person image not associated with the first person image.

[0076] Next, in S15, the generation device 30 generates movement information for the target person included in the associated first person image and second person image. For example, the generation device 30 generates movement information for the target person including a value for at least one item of travel time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction.

[0077] Next, in S16, the generating device 30 generates person information including one or more relevant attributes and movement information for a target person included in the associated first person image and second person image. The generating device 30 also generates person information including one or more relevant attributes and information indicating that the target person has not moved from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2 for a person included in the first person image not associated with the second person image and a person included in the second person image not associated with the first person image.

[0078] Next, in S17, the generating device 30 generates inter-camera association information for a pair of the first camera 12-1 and the second camera 12-2 for each of a plurality of attributes based on the person information of each of the plurality of people. For example, the generating device 30 performs statistical calculations for each of the plurality of attributes on the values ​​of each of the items, namely, movement time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, included in the movement information of the plurality of target people included in the associated first person image and second person image, and generates inter-camera association information for each of the plurality of attributes.

[0079] Furthermore, generation device 30 may calculate, for each of a plurality of attributes, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who exited the imaging range of first camera 12-1, as the value of the transition ratio item in the inter-camera association information. Furthermore, generation device 30 may calculate, for each of a plurality of attributes, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who entered the imaging range of second camera 12-2, as the value of the transition ratio item in the inter-camera association information.

[0080] Next, in S18, the generating device 30 outputs the inter-camera association information for each of the plurality of attributes to the information storage unit 26 as inter-camera association information for each attribute. Furthermore, the generating device 30 may display the generated inter-camera association information for each of the plurality of attributes on a display device or the like.

[0081] When the generation device 30 completes the process of S18, it ends this flow.

[0082] As described above, the generation device 30 according to the first embodiment generates inter-camera association information for each person attribute, representing the relationship between the first camera 12-1 and the second camera 12-2 when a person moves from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2. Travel behavior, such as a person's travel speed and travel route, varies greatly from person to person. By generating inter-camera association information for each person attribute, the generation device 30 according to the first embodiment can generate inter-camera association information for each person who has similar travel behavior trends, such as travel speed and travel route. As a result, the information processing system 10 according to the first embodiment can generate inter-camera association information that accurately represents the relationship between the first camera 12-1 and the second camera 12-2. Furthermore, the information processing system 10 according to the first embodiment uses the inter-camera association information for each attribute generated by the generation device 30, thereby enabling accurate analysis of person movement.

[0083] (Second embodiment) Next, an information processing system 10 according to a second embodiment will be described. The information processing system 10 according to the second embodiment has substantially the same components and functions as the information processing system 10 according to the first embodiment described with reference to Figures 1 to 4, and therefore the substantially same components will be described in the same manner, and detailed description will be omitted except for the differences.

[0084] FIG. 5 is a diagram showing the configuration of an information processing system 10 according to the second embodiment.

[0085] The generating device 30 generates inter-camera association information for each situation, such as the surrounding weather, the surrounding congestion level, and road conditions, for each pair of two cameras 12 included in the plurality of cameras 12. Note that the inter-camera association information for each situation will be described in further detail with reference to FIG. 6.

[0086] The generating device 30 stores the generated inter-camera association information for each situation in the information storage unit 26. Furthermore, the generating device 30 may output the generated inter-camera association information for each situation to an external device or display it on a display device.

[0087] The information storage unit 26 stores, for each pair of two cameras 12 included in the plurality of cameras 12, inter-camera association information for each situation.

[0088] Here, the analysis device 22 identifies the situation around the pair of two cameras 12, and performs an analysis of the movement of the person being analyzed using the inter-camera association information for the identified situation from the inter-camera association information for each situation stored in the information storage unit 26.

[0089] For example, in analyzing the movement of a person, analysis device 22 performs an association process to associate a person who has left the imaging range of first camera 12-1 with a person who has entered the imaging range of second camera 12-2. In this case, analysis device 22 performs the association process using the inter-camera association information of the identified situation among the inter-camera association information for each situation corresponding to the pair of first camera 12-1 and second camera 12-2.

[0090] A person's movement speed or route often varies depending on the surrounding weather, the degree of congestion in the surrounding area, road conditions, and the like. Therefore, for example, the movement time and movement route from when the person leaves the imaging range of first camera 12-1 to when the person enters the imaging range of second camera 12-2 often vary depending on the situation. If analysis device 22 uses inter-camera association information generated regardless of the situation, there is a possibility that the correlation process cannot be performed accurately depending on the situation. In contrast, analysis device 22 according to the second embodiment performs correlation using inter-camera association information for each situation, and therefore can perform correlation process accurately. Therefore, analysis device 22 according to the second embodiment can perform analysis of person movement with high accuracy or in more detail.

[0091] FIG. 6 is a diagram illustrating an example of inter-camera association information for each situation according to the second embodiment.

[0092] The inter-camera association information for each situation is generated for each pair of ordered two cameras 12 included in the plurality of cameras 12. For example, the inter-camera association information for each situation includes a value for at least one item of travel time, transition rate, exit position, entry position, exit angle, entry angle, exit direction, and entry direction.

[0093] Furthermore, the inter-camera association information for each situation includes values ​​for each item for each of a plurality of situations.

[0094] Each of the multiple situations is information that does not depend on people in the area between the imaging range of first camera 12-1 and the imaging range of second camera 12-2. For example, each of the multiple situations represents one of the time period, date, month, day of the week, congestion level, weather information, presence or absence of obstacles, imaging direction of first camera 12-1, and imaging direction of second camera 12-2.

[0095] A time period represents a range of time into which a day is divided. For example, a time period may be a commuting time period and a non-commuting time period. A time period may also be a time range into which a day is divided into predetermined time periods. A time period may also be a time range such as morning, noon, evening, or night.

[0096] Days and months may be units of days, which are obtained by dividing a month into a predetermined number of days, or units of months, which are obtained by dividing a year into a predetermined number of months, or units of seasons, which are obtained by dividing a year into seasons.

[0097] The days of the week may be ranges obtained by dividing a week into days of the week, or may be units obtained by dividing a weekday into a weekend and a weekday.

[0098] The congestion level represents the degree of congestion of the path along which a person moves from the imaging range of first camera 12-1 to the imaging range of second camera 12-2.

[0099] The weather information includes the weather, such as rain or sunshine, temperature, humidity, etc., in the area between the imaging range of first camera 12-1 and the imaging range of second camera 12-2.

[0100] The presence or absence of an obstacle indicates whether or not an obstacle exists on the path moving from the imaging range of first camera 12-1 to the imaging range of second camera 12-2.

[0101] The imaging direction of first camera 12-1 represents the installation angle of first camera 12-1. The imaging direction of first camera 12-1 may represent, for example, whether first camera 12-1 is installed in a normal imaging direction, or whether first camera 12-1 is installed in an imaging direction different from the normal imaging direction when it cannot be installed in the normal imaging direction due to an obstacle such as a vehicle.

[0102] The imaging direction of second camera 12-2 represents the installation angle of second camera 12-2. The imaging direction of second camera 12-2 may represent, for example, whether second camera 12-2 is installed in a normal imaging direction, or whether second camera 12-2 is installed in an imaging direction different from the normal imaging direction when it cannot be installed in the normal imaging direction due to an obstacle such as a vehicle.

[0103] A person's movement behavior, such as their movement speed and movement route, changes depending on the situation. Therefore, when performing the association process, the analysis device 22 can perform the association more accurately by using the inter-camera association information for the same situation as the situation at the time of analysis.

[0104] FIG. 7 is a diagram showing the configuration of a generating device 30 according to the second embodiment.

[0105] Compared to the generation device 30 according to the first embodiment, the generation device 30 according to the second embodiment includes a situation information acquisition unit 62 instead of the attribute information acquisition unit 42.

[0106] The situation information acquisition unit 62 acquires situation information that indicates the situation around the associated moving object. More specifically, the situation information acquisition unit 62 acquires one or more relevant situations during the movement of the target person included in the associated first person image and second person image from among a plurality of predetermined situations in the area between the imaging range of the first camera 12-1 and the imaging range of the second camera 12-2. Note that, when there are multiple target people, the situation information acquisition unit 62 acquires one or more relevant situations for each of the multiple target people.

[0107] Furthermore, the situation information acquisition unit 62 may also acquire one or more relevant situations when a person included in a first person image that is not associated with a second person image exits from the imaging range of the first camera 12-1. Furthermore, the situation information acquisition unit 62 may also acquire one or more relevant situations when a person included in a second person image that is not associated with a first person image enters the imaging range of the second camera 12-2.

[0108] For example, the situation information acquisition unit 62 detects the situation of a target person included in the associated first person image and second person image based on the time when the person exits the imaging range of the first camera 12-1 and enters the imaging range of the second camera 12-2. Also, the situation information acquisition unit 62 detects the situation of a person included in a first person image not associated with a second person image based on the time when the person exits the imaging range of the first camera 12-1. Also, the situation information acquisition unit 62 detects the situation of a person included in a second person image not associated with a first person image based on the time when the person enters the imaging range of the second camera 12-2.

[0109] The situation information acquisition unit 62 may analyze at least one of the first moving image data captured by the first camera 12-1 and the second moving image data captured by the second camera 12-2 to detect the degree of congestion and the presence or absence of an obstacle at a given time. Alternatively, for example, the situation information acquisition unit 62 may detect the degree of congestion and the presence or absence of an obstacle at a given time via a network from an external server device that provides road conditions and the like. Alternatively, for example, the situation information acquisition unit 62 may detect weather information at a given time via a network from an external server device that provides weather information. The situation information acquisition unit 62 may detect the imaging direction of the first camera 12-1 and the imaging direction of the second camera 12-2 by referring to settings set by an administrator or the like.

[0110] In the second embodiment, the person information generation unit 46 generates person information including one or more relevant situations and movement information for a target person included in the associated first person image and second person image. When there are multiple target people, the person information generation unit 46 generates person information for each of the multiple target people.

[0111] In addition, the person information generation unit 46 may generate person information for a person included in a first person image to which no second person image is associated, including one or more relevant situations and information indicating that the person has left the imaging range of the first camera 12-1 but has not entered the imaging range of the second camera 12-2.

[0112] In addition, the person information generation unit 46 may generate person information for a person included in a second person image to which no first person image is associated, including one or more relevant situations and information indicating that the person has entered the imaging range of the second camera 12-2 but has not exited the imaging range of the first camera 12-1.

[0113] The association information generation unit 50 statistically processes the movement information according to the situation information and generates inter-camera association information related to the moving object. More specifically, the association information generation unit 50 generates inter-camera association information for a pair of the first camera 12-1 and the second camera 12-2 for each of a plurality of situations based on the person information of each of a plurality of people stored in the person information storage unit 48.

[0114] For example, the association information generation unit 50 performs statistical calculations for each of the multiple situations on the values ​​of each of the items, namely, movement time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, contained in the movement information of multiple target persons contained in the associated first person image and second person image, and generates inter-camera association information for each of the multiple situations.

[0115] For example, the association information generating unit 50 selects one or more target persons who include any one of a first situation among multiple situations of a plurality of target persons as one or more relevant situations. Next, the association information generating unit 50 calculates the average value, median, standard deviation, or quartile deviation of the values ​​of each of the items, i.e., travel time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, included in the travel information of the selected one or more target persons. Then, the association information generating unit 50 sets the average value, median, standard deviation, or quartile deviation of the values ​​of each item as the values ​​of each of the items, i.e., travel time, exit position, entry position, exit angle, entry angle, exit direction, and entry direction, in the inter-camera association information of the first situation.

[0116] Furthermore, the association information generating unit 50 may calculate, for each of a plurality of situations, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who exited the imaging range of first camera 12-1, as the value of the transition ratio item in the inter-camera association information. Furthermore, the association information generating unit 50 may calculate, for each of a plurality of situations, the ratio of the number of people who entered the imaging range of second camera 12-2 from the imaging range of first camera 12-1 to the number of people who entered the imaging range of second camera 12-2, as the value of the transition ratio item in the inter-camera association information.

[0117] Then, the association information generating unit 50 stores the inter-camera association information for each of the plurality of situations in the association information storage unit 52.

[0118] In the second embodiment, the output unit 54 writes the inter-camera association information for each of the plurality of situations stored in the association information storage unit 52 as inter-camera association information for each situation into the information storage unit 26. In the second embodiment, the display control unit 56 displays the generated inter-camera association information for each of the plurality of situations on a display device or the like in response to, for example, a user operation or the like.

[0119] 8 is a flowchart showing the flow of processing by the generating device 30 according to the second embodiment. The generating device 30 according to the second embodiment executes processing, for example, according to the flow shown in FIG.

[0120] Compared to the generation device 30 according to the first embodiment, the generation device 30 according to the second embodiment executes the process of S21 instead of the process of S14. Therefore, the processes from S21 onwards will be described for the generation device 30 according to the second embodiment.

[0121] The generating device 30 according to the second embodiment executes the process of S13 and then executes the process of S21.

[0122] In S21, the generating device 30 acquires one or more relevant situations during movement of a target person included in the associated first person image and second person image from among a plurality of predetermined situations. The generating device 30 also acquires one or more relevant situations for a person included in a first person image that is not associated with a second person image and a person included in a second person image that is not associated with a first person image.

[0123] Following S21, the generation device 30 executes the process of S15. In S15, the generation device 30 generates movement information for the target person included in the associated first person image and second person image.

[0124] Next, in S16, the generation device 30 generates person information including one or more relevant situations and movement information for a target person included in the associated first person image and second person image. Also, the generation device 30 generates person information including one or more relevant situations and information indicating that the target person has not moved from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2 for a person included in the first person image not associated with the second person image and a person included in the second person image not associated with the first person image.

[0125] Next, in S17, generation device 30 generates inter-camera association information for a pair of first camera 12-1 and second camera 12-2 for each of a plurality of situations based on the person information of each of a plurality of people.

[0126] Next, in S18, the generating device 30 outputs the inter-camera association information for each of the plurality of situations to the information storage unit 26 as the inter-camera association information for each situation. Furthermore, the generating device 30 may display the generated inter-camera association information for each of the plurality of situations on a display device or the like.

[0127] When the generation device 30 completes the process of S18, it ends this flow.

[0128] As described above, the generation device 30 according to the second embodiment generates inter-camera association information for each situation, which represents the relationship between the first camera 12-1 and the second camera 12-2 when a person moves from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2. The movement speed and route of a person vary significantly depending on the situation, such as the path along which the person moves. By generating inter-camera association information for each situation, the generation device 30 according to the second embodiment can generate inter-camera association information for each situation in which movement trends, such as movement speed and movement route, are similar. As a result, the information processing system 10 according to the second embodiment can generate inter-camera association information that accurately represents the relationship between the first camera 12-1 and the second camera 12-2. Furthermore, the information processing system 10 according to the second embodiment uses the inter-camera association information for each situation generated by the generation device 30, thereby enabling accurate analysis of a person's movement.

[0129] (Third embodiment) Next, an information processing system 10 according to a third embodiment will be described. The information processing system 10 according to the third embodiment has substantially the same components and functions as the information processing system 10 according to the first embodiment described with reference to Figures 1 to 4, and therefore the substantially same components will be described in the same manner, and detailed description will be omitted except for the differences.

[0130] FIG. 9 is a diagram showing the configuration of an information processing system 10 according to the third embodiment.

[0131] The generating device 30 generates inter-camera association information for each attribute and situation for each pair of two cameras 12 included in the plurality of cameras 12. The generating device 30 stores the generated inter-camera association information for each attribute and situation in the information storage unit 26. Furthermore, the generating device 30 may output the generated inter-camera association information for each attribute and situation to an external device or display it on a display device.

[0132] The information storage unit 26 stores, for each pair of two cameras 12 included in the plurality of cameras 12, inter-camera association information for each attribute and each situation.

[0133] Here, analysis device 22 identifies the attributes of the person to be analyzed and the situation around the pair of two cameras 12, and performs analysis of the movement of the person to be analyzed using the inter-camera association information of the identified attributes and situations among the inter-camera association information for each attribute and situation stored in information storage unit 26. For example, analysis device 22 performs association processing using the inter-camera association information of the identified attributes and situations among the inter-camera association information for each attribute and situation corresponding to the pair of first camera 12-1 and second camera 12-2.

[0134] The analysis device 22 according to the third embodiment performs the association process using the inter-camera association information for each attribute and each situation, and therefore can perform the association process with high accuracy. Therefore, the analysis device 22 according to the third embodiment can perform an analysis of the movement of a person with high accuracy or in more detail.

[0135] FIG. 10 is a diagram showing an example of inter-camera association information for each attribute and each situation according to the third embodiment.

[0136] The inter-camera association information for each attribute and for each situation is generated for each pair of ordered two cameras 12 included in the plurality of cameras 12. For example, the inter-camera association information for each attribute and for each situation includes a value for at least one item of travel time, transition rate, exit position, entry position, exit angle, entry angle, exit direction, and entry direction.

[0137] Furthermore, the inter-camera association information for each attribute and for each situation includes a value of each item for each of the multiple attributes and the multiple situations. That is, the inter-camera association information for each attribute and for each situation includes a value of each item for each combination of at least one attribute from the multiple attributes and at least one situation from the multiple situations.

[0138] For example, each of the multiple attributes represents any one of gender, age group, clothing, social role, presence or absence of specific belongings, belongings, presence or absence of other people acting together, movement, posture, and facial expression. For example, each of the multiple situations represents any one of time of day, date, month, day of the week, congestion level, weather information, presence or absence of obstacles, imaging direction of first camera 12-1, and imaging direction of second camera 12-2.

[0139] FIG. 11 is a diagram showing the configuration of a generating device 30 according to the third embodiment.

[0140] Compared to the generation device 30 according to the first embodiment, the generation device 30 according to the third embodiment further includes a situation information acquisition unit 62. The situation information acquisition unit 62 has the same functions as those in the second embodiment.

[0141] In the third embodiment, the person information generation unit 46 generates person information including one or more relevant attributes, one or more relevant situations, and movement information for a target person included in the associated first person image and second person image. When there are multiple target people, the person information generation unit 46 generates person information for each of the multiple target people.

[0142] In addition, the person information generation unit 46 may generate person information for a person included in a first person image to which no second person image is associated, the person information including one or more relevant attributes, one or more relevant situations, and information indicating that the person has left the imaging range of the first camera 12-1 but has not entered the imaging range of the second camera 12-2.

[0143] In addition, the person information generation unit 46 may generate person information for a person included in a second person image to which no first person image is associated, the person information including one or more relevant attributes, one or more relevant situations, and information indicating that the person has entered the imaging range of the second camera 12-2 but has not exited the imaging range of the first camera 12-1.

[0144] The association information generation unit 50 statistically processes the movement information according to the attribute information and situation information, and generates inter-camera association information related to the moving object. More specifically, the association information generation unit 50 generates inter-camera association information for a pair of the first camera 12-1 and the second camera 12-2 for each of one or more relevant attributes and multiple situations based on the personal information of each of the multiple people stored in the personal information storage unit 48. The association information generation unit 50 then stores the inter-camera association information for each of the multiple attributes and multiple situations in the association information storage unit 52.

[0145] In the third embodiment, the output unit 54 writes the inter-camera association information for each of the multiple attributes and multiple situations stored in the association information storage unit 52 as inter-camera association information for each attribute and for each situation in the information storage unit 26. In the third embodiment, the display control unit 56 displays the generated inter-camera association information for each of the multiple attributes and multiple situations on a display device or the like in response to, for example, a user operation or the like.

[0146] 12 is a flowchart showing the flow of processing performed by the generating device 30 according to the third embodiment. The generating device 30 according to the third embodiment executes processing according to the flow shown in FIG.

[0147] The generation device 30 according to the third embodiment further executes the process of S21 when compared with the generation device 30 according to the first embodiment. Therefore, the processes from S21 onwards will be described for the generation device 30 according to the third embodiment.

[0148] The generation device 30 according to the third embodiment executes the process of S13, and then additionally executes the process of S21.

[0149] In S21, the generating device 30 acquires one or more relevant situations during movement of a target person included in the associated first person image and second person image from among a plurality of predetermined situations. The generating device 30 also acquires one or more relevant situations for a person included in a first person image that is not associated with a second person image and a person included in a second person image that is not associated with a first person image.

[0150] Following S21, in S14, the generation device 30 acquires one or more relevant attributes representing the nature or characteristics of the target person included in the associated first person image and second person image from among a plurality of predetermined attributes. The generation device 30 also acquires one or more relevant attributes for the person included in the first person image not associated with the second person image and the person included in the second person image not associated with the first person image.

[0151] The generating device 30 may execute the process of S21 after S14, or may execute S14 and S21 in parallel.

[0152] Next, in S15, the generation device 30 generates movement information for the target person included in the associated first person image and second person image.

[0153] Next, in S16, the generation device 30 generates person information including one or more relevant situations and movement information for a target person included in the associated first person image and second person image. Also, the generation device 30 generates person information including one or more relevant situations and information indicating that the target person has not moved from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2 for a person included in the first person image not associated with the second person image and a person included in the second person image not associated with the first person image.

[0154] Next, in S17, generation device 30 generates inter-camera association information for a pair of first camera 12-1 and second camera 12-2 for each of a plurality of attributes and a plurality of situations based on the person information of each of a plurality of people.

[0155] Next, in S18, the generating device 30 outputs the inter-camera association information for each of the multiple attributes and multiple situations to the information storage unit 26 as inter-camera association information for each attribute and each situation. Furthermore, the generating device 30 may display the generated inter-camera association information for each of the multiple attributes and multiple situations on a display device or the like.

[0156] When the generation device 30 completes the process of S18, it ends this flow.

[0157] As described above, the generation device 30 according to the third embodiment generates inter-camera association information that represents the relationship between the first camera 12-1 and the second camera 12-2 when a person moves from the imaging range of the first camera 12-1 to the imaging range of the second camera 12-2 for each attribute and for each situation. This allows the information processing system 10 according to the third embodiment to generate inter-camera association information that accurately represents the relationship between the first camera 12-1 and the second camera 12-2. Furthermore, the information processing system 10 according to the third embodiment uses the inter-camera association information for each attribute and for each situation generated by the generation device 30, thereby enabling accurate analysis of the movement of a person.

[0158] (Fourth embodiment) Next, an information processing system 10 according to a fourth embodiment will be described. The information processing system 10 according to the fourth embodiment has substantially the same components and functions as the information processing system 10 according to the first embodiment described with reference to Figures 1 to 4, and therefore the substantially same components will be described in the same manner, and detailed description will be omitted except for the differences.

[0159] 13 is a flowchart showing the flow of processing of the generating device 30 according to the fourth embodiment. The generating device 30 according to the fourth embodiment executes processing, for example, according to the flow shown in FIG.

[0160] The generation device 30 according to the fourth embodiment first executes the processes from S11 to S17, similarly to the generation device 30 according to the first embodiment. After S17, the generation device 30 advances the process to S41.

[0161] In S41, the generation device 30 determines whether a preset termination condition has been reached. For example, the generation device 30 determines that the termination condition has been reached when the process of S17 has been executed a preset number of times. Alternatively, the generation device 30 may determine that the termination condition has been reached when a preset time has elapsed. If the termination condition has not been reached (No in S41), the generation device 30 returns the process to S13.

[0162] In the second or subsequent times of processing S13, the generation device 30 re-associates each of the multiple first person images with one of the multiple second person images that is estimated to include the same person as the corresponding first person image, based on the inter-camera association information generated in the immediately preceding processing S17. That is, in the second or subsequent times of processing S13, the generation device 30 re-executes the association process using the inter-camera association information generated in the immediately preceding processing S17. This allows the generation device 30 to perform the association process more accurately in the second or subsequent times of processing S13 than in the first time of processing S13.

[0163] In steps S14 to S16 from the second time onwards, the generation device 30 executes the process using the results of the re-association. Then, in step S17 from the second time onwards, the generation device 30 generates new inter-camera association information and rewrites the inter-camera association information generated in the process of the immediately preceding step S17.

[0164] The generation device 30 according to the fourth embodiment can improve the accuracy of the inter-camera association information. Note that the generation device 30 according to the second embodiment and the generation device 30 according to the third embodiment may also perform the association process again based on the generated inter-camera association information, similar to the fourth embodiment.

[0165] (Display example) Next, a display example of the inter-camera association information by the information processing system 10 according to the first to fourth embodiments will be described.

[0166] FIG. 14 is a diagram showing a first display image 70 by the information processing system 10 according to the first to fourth embodiments.

[0167] When analyzing the movement of a person, for example, to track a specific person, the information processing device 20 may display the first display image 70 on the display device.

[0168] The first display image 70 includes a person image 72 at a time including a specific person in the video data, for each of the multiple cameras 12. For example, the first display image 70 includes a first person image 72-1 captured by the first camera 12-1, a second person image 72-2 captured by the second camera 12-2, and a third person image 72-3 captured by the third camera 12-3. This allows the user to visually identify the specific person from the first display image 70.

[0169] Furthermore, first display image 70 includes, on a map image including the route of movement of the specific person, imaging range images 74 showing the imaging range of each of the multiple cameras 12. For example, first display image 70 includes a first imaging range image 74-1 showing the imaging range of first camera 12-1, a second imaging range image 74-2 showing the imaging range of second camera 12-2, and a third imaging range image 74-3 showing the imaging range of third camera 12-3, all at corresponding positions on the map image. This allows the user to visually confirm the location on the map of the specific person included in person image 72.

[0170] Furthermore, the first display image 70 includes the value of at least one item included in the inter-camera association information.

[0171] For example, the first display image 70 includes, in the region between the first imaging range image 74-1 and the second imaging range image 74-2, first time information 76-1 indicating a travel time included in the inter-camera association information for the pair of the first camera 12-1 and the second camera 12-2. The first display image 70 also includes a first arrow image 78-1 indicating an exit position and an exit angle and an entry position and an entry angle included in the inter-camera association information for the pair of the first camera 12-1 and the second camera 12-2. The first arrow image 78-1 is positioned at a corresponding position and angle on the map image. The first display image 70 also includes, in the region between the first imaging range image 74-1 and the second imaging range image 74-2, first ratio information 80-1 indicating a transition ratio included in the inter-camera association information for the pair of the first camera 12-1 and the second camera 12-2.

[0172] The first display image 70 also includes, in the region between the second imaging range image 74-2 and the third imaging range image 74-3, second time information 76-2 indicating a travel time included in the inter-camera association information for the pair of the second camera 12-2 and the third camera 12-3. The first display image 70 also includes a second arrow image 78-2 indicating an exit position and an exit angle and an entry position and an entry angle included in the inter-camera association information for the pair of the second camera 12-2 and the third camera 12-3. The second arrow image 78-2 is positioned at a corresponding position and angle on the map image. The first display image 70 also includes, in the region between the second imaging range image 74-2 and the third imaging range image 74-3, second ratio information 80-2 indicating a transition ratio included in the inter-camera association information for the pair of the second camera 12-2 and the third camera 12-3.

[0173] By displaying such a first display image 70, the information processing device 20 can, for example, allow the user to recognize the movement of a specific person included in the person image 72 while visually checking the inter-camera association information.

[0174] FIG. 15 is a diagram showing a second display image 90 by the information processing system 10 according to the first to fourth embodiments.

[0175] When analyzing the movement of a person, for example, to track a specific person, the information processing device 20 may display the second display image 90 on the display device.

[0176] 14, the second display image 90 includes person images 72 at times when the specific person is included in the video data for each of the multiple cameras 12. This allows the user to visually confirm the specific person from the second display image 90.

[0177] The second display image 90 includes an image representing a directed graph in which each of the multiple cameras 12 is represented by a node 92, and the connection relationship between pairs of two cameras 12 represented in the inter-camera association information is represented by an edge 94. In this case, the size or shape of the nodes 92 may be changed depending on, for example, the size or shape of the imaging range of the corresponding camera 12.

[0178] The edge 94 may also include, as an edge weight, the value of a predetermined item included in the corresponding inter-camera association information. The thickness, color, and line style of the edge 94 may be changed for each attribute or situation. Line styles include solid, dashed, and double lines. The edge 94 may also include text information representing the attribute or situation associated with the edge weight.

[0179] By displaying such a second display image 90, the information processing device 20 can, for example, allow the user to recognize the movement of a specific person included in the person image 72 while visually checking the inter-camera association information for each attribute or each attribute.

[0180] (Hardware configuration of information processing device 20, etc.) Fig. 16 is a diagram showing an example of the hardware configuration of the information processing device 20. The information processing device 20 is realized by, for example, a computer having the hardware configuration shown in Fig. 16. The information processing device 20 includes a CPU (Central Processing Unit) 901, a RAM (Random Access Memory) 902, a ROM (Read Only Memory) 903, a storage device 904, and a communication interface device 905. These components are connected via a bus.

[0181] The CPU 901 is one or more processors that execute arithmetic processing, control processing, etc. according to a program. The CPU 901 uses a predetermined area of ​​the RAM 902 as a working area and executes various processes in cooperation with programs stored in the ROM 903, the storage device 904, etc.

[0182] The RAM 902 is a memory such as an SDRAM (Synchronous Dynamic Random Access Memory), and functions as a work area for the CPU 901. The ROM 903 is a memory that stores programs and various information in a non-rewritable manner.

[0183] The storage device 904 is a device that writes and reads data to a semiconductor storage medium such as a flash memory, or a magnetically or optically recordable storage medium, etc. The storage device 904 writes and reads data to the storage medium in response to control from the CPU 901. The communication interface device 905 communicates with external devices via a network in response to control from the CPU 901.

[0184] A program executed by the computer causes the computer to function as the information processing device 20. This program is loaded onto the RAM 902 by the CPU 901 (processor) and executed.

[0185] In addition, the program to be executed by a computer is provided as a file in a format that can be installed on a computer or in a format that can be executed by a computer, and is recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk, a CD-R, or a DVD (Digital Versatile Disk).

[0186] This program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. This program may also be provided or distributed via a network such as the Internet. The program executed by the information processing device 20 may also be provided by being pre-installed in the ROM 903 or the like.

[0187] The program for causing a computer to function as the information processing device 20 includes, for example, a first acquisition module, a second acquisition module, a first person detection module, a second person detection module, a matching module, an attribute information acquisition module, a movement information generation module, a person information generation module, an association information generation module, an output module, and a display control module. The program may further include a situation information acquisition module. When executed by the CPU 901, each module is loaded into the RAM 902, causing the CPU 901 to function as the first acquisition unit 32, the second acquisition unit 34, the first person detection unit 36, the second person detection unit 38, the matching unit 40, the attribute information acquisition unit 42, the movement information generation unit 44, the person information generation unit 46, the association information generation unit 50, the output unit 54, and the display control unit 56. The program may also function as a situation information acquisition unit 62. If the CPU 901 has multiple processors, these units may be divided among the multiple processors. It should be noted that these components may be configured partially or entirely by hardware. This program also causes the RAM 902 and the storage device 904 to function as the person information storage unit 48 and the association information storage unit 52.

[0188] Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be embodied in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as set forth in the claims. In particular, while the above embodiments have been described focusing on people as moving objects, the moving object may also be, for example, an animal, a self-propelled robot, an object operated by a person, or other moving object. [Explanation of symbols]

[0189] 10 Information Processing Systems 12 Camera 20 Information processing equipment 22 Analyzer 24 Video memory unit 26 Information storage section 30 Generator 32 First acquisition part 34 Second acquisition part 36 First person detection unit 38 Second person detection unit 40 Mapping section 42 Attribute information acquisition section 44 Movement information generation section 46 Person information generation section 48 Person information storage section 50 Association information generation unit 52 Association information storage unit 54 Output section 56 Display control unit 62 Status information acquisition unit

Claims

1. An information processing device that processes information from images captured by a plurality of cameras, a correspondence unit that, when a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are estimated to be the same, associates the moving object detected from the first image with the moving object detected from the second image; an attribute information acquisition unit that acquires attribute information relating to the properties or characteristics of the moving object from the image; a movement information generating unit that generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; an association information generation unit that statistically processes the movement information according to the attribute information and generates inter-camera association information related to the moving object; An information processing device comprising:

2. the inter-camera association information and the movement information include a value representing at least one of a movement time, an exit position, an entry position, an exit angle, an entry angle, an exit direction, and an entry direction; the travel time represents a time from when a moving object leaves the imaging range of the first camera to when it enters the imaging range of the second camera, the exit position represents a position at which a moving object exits from an imaging range of the first camera and enters an imaging range of the second camera, the entry position represents a position at which a moving object exiting an imaging range of the first camera and entering an imaging range of the second camera enters the imaging range of the second camera, the exit angle represents an angle or orientation of a moving object, which exits an imaging range of the first camera and enters an imaging range of the second camera, with respect to the first camera at the time of exiting the imaging range of the first camera; the entry angle represents an angle or orientation of a moving object, which leaves an imaging range of the first camera and enters an imaging range of the second camera, with respect to the second camera at the time of the moving object entering the imaging range of the second camera; the exit direction represents a moving direction of a moving object at a time when the moving object exits the imaging range of the first camera and enters the imaging range of the second camera, and The entry direction represents the moving direction of a moving object that leaves the imaging range of the first camera and enters the imaging range of the second camera at the time of entering the imaging range of the second camera. The information processing device according to claim 1 .

3. The association information generation unit performs a statistical calculation on the movement information of each of a plurality of moving objects for each of a plurality of pieces of attribute information, and generates the inter-camera association information for each of the plurality of pieces of attribute information. The information processing device according to claim 2 .

4. The association information generation unit includes an average value, a median value, a standard deviation, or a quartile deviation for each of the plurality of pieces of attribute information of values ​​included in the movement information of each of the plurality of moving bodies in the inter-camera association information for each of the plurality of pieces of attribute information. The information processing device according to claim 3 .

5. the inter-camera association information includes a transition rate; The transition rate represents a rate of the number of moving objects that have entered the imaging range of the second camera from the imaging range of the first camera to the number of moving objects that have exited the imaging range of the first camera. The information processing device according to claim 1 .

6. the inter-camera association information includes a transition rate; The transition rate represents a rate of the number of moving objects that have entered the imaging range of the second camera from the imaging range of the first camera to the number of moving objects that have entered the imaging range of the second camera. The information processing device according to claim 1 .

7. The moving object is a person, The attribute information indicates any of the following: gender, age group, clothing, social role, presence or absence of specific belongings, belongings, presence or absence of other moving objects acting together, movement, posture, and facial expression. The information processing device according to claim 1 .

8. After the inter-camera association information is generated, Furthermore, the association unit again associates the moving object detected from the first image with the moving object detected from the second image based on the generated inter-camera association information, the attribute information acquisition unit acquires the attribute information again for the re-associated moving object; the movement information generation unit regenerates the movement information for the re-associated moving object; The association information generation unit statistically processes the regenerated movement information in accordance with the regenerated attribute information, and regenerates the inter-camera association information for the re-associated moving object. The information processing device according to claim 1 .

9. a display control unit that displays the inter-camera association information on a display device; The information processing device according to claim 1 .

10. An information processing device that processes information from images captured by a plurality of cameras, a correspondence unit that, when a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are estimated to be the same, associates the moving object detected from the first image with the moving object detected from the second image; a situation information acquisition unit that acquires situation information that indicates a situation around the associated moving object; a movement information generating unit that generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; an association information generation unit that statistically processes the movement information according to the situation information and generates inter-camera association information related to the moving object; An information processing device comprising:

11. The situation information indicates any of a time period, a date and a month, a day of the week, a congestion level, weather information, the presence or absence of an obstacle, an imaging direction of the first camera, and an imaging direction of the second camera. The information processing device according to claim 10.

12. An attribute information acquisition unit that acquires attribute information relating to the properties or characteristics of the moving object from the image, The association information generating unit generates the inter-camera association information based on the attribute information, the situation information, and the movement information. The information processing device according to claim 10.

13. Multiple cameras and an analysis device that analyzes the movement of a moving object based on the video image data captured by each of the plurality of cameras; An information processing device according to any one of claims 1 to 12; Equipped with the information processing device generates the inter-camera association information by designating any one of the plurality of cameras as the first camera and any one of the plurality of cameras other than the first camera as the second camera; The analysis device uses the inter-camera association information to analyze a moving object that moves from the imaging range of the first camera to the imaging range of the second camera. Information processing system.

14. An information processing method for processing information from images captured by a plurality of cameras by an information processing device, comprising: when the information processing device estimates that a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are identical, the information processing device associates the moving object detected from the first image with the moving object detected from the second image; the information processing device acquires attribute information relating to the nature or characteristics of the moving object from the image; the information processing device generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; The information processing device statistically processes the movement information according to the attribute information and generates inter-camera association information related to the moving object. Information processing methods.

15. An information processing method for processing information from images captured by a plurality of cameras by an information processing device, comprising: when the information processing device estimates that a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are identical, the information processing device associates the moving object detected from the first image with the moving object detected from the second image; the information processing device acquires situation information that indicates a situation around the associated moving object; the information processing device generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; The information processing device statistically processes the movement information according to the situation information and generates inter-camera association information related to the moving object. Information processing methods.

16. Computer, a correspondence unit that, when a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are estimated to be the same, associates the moving object detected from the first image with the moving object detected from the second image; an attribute information acquisition unit that acquires attribute information relating to the properties or characteristics of the moving object from the image; a movement information generating unit that generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; an association information generation unit that statistically processes the movement information according to the attribute information and generates inter-camera association information related to the moving object; A program that makes it work.

17. Computer, a correspondence unit that, when a moving object detected from a first image captured by a first camera and a moving object detected from a second image captured by a second camera are estimated to be the same, associates the moving object detected from the first image with the moving object detected from the second image; a situation information acquisition unit that acquires situation information that indicates a situation around the associated moving object; a movement information generating unit that generates movement information regarding movement of the associated moving object from an imaging range of the first camera related to the first image to an imaging range of the second camera related to the second image; an association information generation unit that statistically processes the movement information according to the situation information and generates inter-camera association information related to the moving object; A program that makes it work.

Images