Information processing apparatus, information processing method, and information processing program

Abstract

To highly accurately predict a trajectory of an object.SOLUTION: The information processing apparatus includes a trajectory prediction unit that predicts a trajectory of an object included in a target image, a coordinate conversion unit that converts the trajectory into a reference coordinate, a trajectory correlation unit that calculates a degree of correlation with another trajectory, a trajectory integration unit that integrates trajectories having a high degree of correlation, and a space-time synchronization unit that calculates a parameter for spatially and temporally synchronizing an image, in which the coordinate conversion unit converts the trajectory into the reference coordinate with reference to the parameter.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] There are known techniques for predicting the trajectory of a moving object. For example, a technique for predicting the trajectory of an object using radar is known. However, this technique has problems such as difficulty in distinguishing between objects with different appearances and an increase in noise due to the influence of diffused radio waves.

[0003] As another example, a technique using images captured by a camera is known. For example, Patent Document 1 discloses a multiple object tracking device that selects a camera to be used to update the tracking result from multiple cameras, taking into consideration the degree of occlusion of the object, and updates the tracking result based on the association between the detection result of the object in the image of the selected camera and the predicted position. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-177391 Summary of the Invention [Problem to be solved by the invention]

[0005] In the multiple object tracking device described in Patent Document 1, the multiple cameras are not synchronized, so the detection result for the position of an object in an image taken by one camera may differ from the detection result in an image taken by another camera. As a result, the multiple object tracking device described in Patent Document 1 has a problem in that the accuracy of the predicted tracking results is low.

[0006] The present disclosure has been made in consideration of the above-described problems, and an exemplary purpose thereof is to provide a technique for predicting the trajectory of an object with high accuracy. [Means for solving the problem]

[0007] An information processing device according to an exemplary aspect of the present disclosure includes: a trajectory prediction means that, for each of a plurality of cameras, predicts a trajectory of an object included in at least one of a plurality of target images by referring to the plurality of target images captured by the camera; a coordinate conversion means that converts each of the plurality of trajectories predicted by the trajectory prediction means into reference coordinates; a trajectory correlation means that calculates a degree of correlation with other trajectories for each of the plurality of trajectories converted into the reference coordinates; a trajectory integration means that integrates trajectories whose degree of correlation is higher than a predetermined value; and a time-space synchronization means that, by referring to the trajectories integrated by the trajectory integration means, calculates parameters for spatially and temporally synchronizing the images captured by each of the plurality of cameras; and the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters.

[0008] An information processing method according to an exemplary aspect of the present disclosure includes: a trajectory prediction process in which at least one processor, for each of a plurality of cameras, refers to a plurality of target images captured by the camera and predicts a trajectory of an object included in at least one of the plurality of target images; a coordinate transformation process in which each of the plurality of trajectories predicted by the trajectory prediction process is converted into reference coordinates; a trajectory correlation process in which, for each of the plurality of trajectories converted into the reference coordinates, a degree of correlation with other trajectories; a trajectory integration process in which trajectories having a degree of correlation higher than a predetermined value are integrated; and a spatiotemporal synchronization process in which, by referring to the trajectories integrated by the trajectory integration process, parameters are calculated to spatially and temporally synchronize the images captured by each of the plurality of cameras; and in the coordinate transformation process, the at least one processor converts each of the plurality of trajectories into the reference coordinates by referring to the parameters.

[0009] An information processing program according to an exemplary aspect of the present disclosure is a program that causes a computer to function as an information processing device, and causes the computer to function as: a trajectory prediction means that, for each of a plurality of cameras, refers to a plurality of target images captured by the camera, and predicts a trajectory of an object included in at least one of the plurality of target images; a coordinate conversion means that converts each of the plurality of trajectories predicted by the trajectory prediction means into reference coordinates; a trajectory correlation means that calculates a degree of correlation with other trajectories for each of the plurality of trajectories converted into the reference coordinates; a trajectory integration means that integrates trajectories whose degree of correlation is higher than a predetermined value; and a time-space synchronization means that refers to the trajectories integrated by the trajectory integration means and calculates parameters that spatially and temporally synchronize the images captured by the plurality of cameras, and the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters. [Effects of the Invention]

[0010] According to an exemplary aspect of the present disclosure, an exemplary effect is achieved in that a technique for predicting the trajectory of an object with high accuracy can be provided. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 3] FIG. 10 is a diagram illustrating an example of an outline of a process in which an information processing device according to the present disclosure predicts a trajectory and calculates parameters by referring to the predicted trajectory. [Figure 4] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 5] FIG. 1 is a flow diagram showing the flow of an information processing method according to the present disclosure. [Figure 6] 10A and 10B are diagrams illustrating an example of a process in which an information processing device according to the present disclosure generates an integrated trajectory. [Figure 7]FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technical means employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0013] First Exemplary Embodiment A first exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. Note that the scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in the drawings referred to in describing this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure to the extent that no particular technical obstacles arise.

[0014] (Configuration of information processing device 1) The configuration of the information processing device 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes a trajectory prediction unit 11, a coordinate transformation unit 12, a trajectory correlation unit 13, a trajectory integration unit 14, and a space-time synchronization unit 15. In this exemplary embodiment, the trajectory prediction unit 11, the coordinate transformation unit 12, the trajectory correlation unit 13, the trajectory integration unit 14, and the space-time synchronization unit 15 respectively realize a trajectory prediction means, a coordinate transformation means, a trajectory correlation means, a trajectory integration means, and a space-time synchronization means.

[0015] (Trajectory Prediction Unit 11) The trajectory prediction unit 11 refers to a plurality of target images captured by each of a plurality of cameras, and predicts the trajectory of an object included in at least one of the plurality of target images. The trajectory prediction unit 11 supplies information indicating the predicted trajectory to the coordinate conversion unit 12.

[0016] (Coordinate conversion unit 12) The coordinate transformation unit 12 transforms each of the plurality of trajectories predicted by the trajectory prediction unit 11 into reference coordinates. The coordinate transformation unit 12 supplies the trajectory correlation unit 13 with information indicating the trajectories after the transformation.

[0017] Furthermore, the coordinate transformation unit 12 transforms each of the multiple trajectories into reference coordinates by referring to parameters calculated by the time-space synchronization unit 15, which will be described later.

[0018] (Trajectory correlation unit 13) The trajectory correlation unit 13 calculates the degree of correlation with other trajectories for each of the multiple trajectories converted into the reference coordinates. The trajectory correlation unit 13 supplies the calculated degrees to the trajectory integration unit 14.

[0019] (Trajectory integration part 14) The trajectory integration unit 14 integrates trajectories having a degree of correlation higher than a predetermined value, and supplies information indicating the integrated trajectory to the space-time synchronization unit 15.

[0020] (Space-time synchronization unit 15) The time-space synchronization unit 15 calculates parameters for spatially and temporally synchronizing the images captured by the multiple cameras, with reference to the trajectories integrated by the trajectory integration unit 14. The time-space synchronization unit 15 supplies the calculated parameters to the coordinate transformation unit 12.

[0021] (Effects of information processing device 1) As described above, the information processing device 1 employs a configuration including a trajectory prediction unit 11 that, for each of a plurality of cameras, refers to a plurality of target images captured by the camera and predicts the trajectory of an object included in at least one of the plurality of target images; a coordinate conversion unit 12 that converts each of the plurality of trajectories predicted by the trajectory prediction unit 11 into reference coordinates; a trajectory correlation unit 13 that calculates the degree of correlation with other trajectories for each of the plurality of trajectories converted into reference coordinates; a trajectory integration unit 14 that integrates trajectories having a degree of correlation higher than a predetermined value; and a spatiotemporal synchronization unit 15 that refers to the trajectories integrated by the trajectory integration unit 14 and calculates parameters for spatially and temporally synchronizing the images captured by the plurality of cameras.

[0022] Furthermore, in the information processing device 1, the coordinate transformation unit 12 refers to the parameters calculated by the time-space synchronization unit 15 and transforms each of the multiple trajectories into reference coordinates.

[0023] Therefore, according to the information processing device 1, trajectories are integrated based on the target images captured by each of the multiple cameras, so even if an object is in an occlusion state in an image captured by one camera, where the object is hidden by another object, the trajectory is predicted based on the object included in the image captured by the other camera. Therefore, according to the information processing device 1, it is possible to predict the trajectory of an object with high accuracy.

[0024] Furthermore, the information processing device 1 refers to the integrated trajectory, calculates parameters for spatially and temporally synchronizing the images captured by the multiple cameras, and converts the trajectory based on the target images captured by each of the multiple cameras into reference coordinates using the parameters. Therefore, the information processing device 1 predicts the trajectory of an object after temporally and spatially synchronizing the trajectories based on the target images captured by each of the multiple cameras, so that even if an object is in an occlusion state in an image captured by a certain camera, where the object is hidden by another object, the trajectory of the object can be predicted. In other words, the information processing device 1 can predict the trajectory of an object with high accuracy.

[0025] (Flow of information processing method S1) The flow of information processing method S1 will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of information processing method S1. As shown in Fig. 2, information processing method S1 includes trajectory prediction processing S11, coordinate conversion processing S12, trajectory correlation processing S13, trajectory integration processing S14, and space-time synchronization processing S15.

[0026] (Trajectory prediction process S11) In the trajectory prediction process S11, the trajectory prediction unit 11 refers to a plurality of target images captured by each of a plurality of cameras, and predicts the trajectory of an object included in at least one of the plurality of target images. The trajectory prediction unit 11 supplies information indicating the predicted trajectory to the coordinate conversion unit 12.

[0027] (Coordinate conversion process S12) In coordinate conversion processing S12, the coordinate conversion unit 12 converts each of the multiple trajectories predicted by the trajectory prediction unit 11 into reference coordinates. The coordinate conversion unit 12 supplies information indicating the converted trajectories to the trajectory correlation unit 13.

[0028] (Trajectory correlation processing S13) In the trajectory correlation process S13, the trajectory correlation unit 13 calculates the degree of correlation with other trajectories for each of the multiple trajectories converted into the reference coordinates. The trajectory correlation unit 13 supplies the calculated degrees to the trajectory integration unit 14.

[0029] (Trajectory integration process S14) In the trajectory integration process S14, the trajectory integration unit 14 integrates trajectories having a degree of correlation higher than a predetermined value. The trajectory integration unit 14 supplies information indicating the integrated trajectory to the time-space synchronization unit 15.

[0030] (Space-time synchronization processing S15) In the time-space synchronization process S15, the time-space synchronization unit 15 calculates parameters for spatially and temporally synchronizing the images captured by the multiple cameras, with reference to the trajectories integrated by the trajectory integration unit 14. The time-space synchronization unit 15 supplies the calculated parameters to the coordinate transformation unit 12.

[0031] After the time-space synchronization process S15 is performed, the coordinate conversion process S12 is performed again. Here, in the coordinate conversion process S12, the coordinate conversion unit 12 converts each of the multiple trajectories into reference coordinates by referring to the parameters calculated by the time-space synchronization unit 15.

[0032] (Effect of information processing method S1) As described above, the information processing method S1 employs a configuration including: a trajectory prediction process S11 in which the trajectory prediction unit 11, for each of a plurality of cameras, refers to a plurality of target images captured by the camera, and predicts the trajectory of an object included in at least one of the plurality of target images; a coordinate conversion process S12 in which the coordinate conversion unit 12 converts each of the plurality of trajectories predicted by the trajectory prediction unit 11 into reference coordinates; a trajectory correlation process S13 in which the trajectory correlation unit 13 calculates the degree of correlation with other trajectories for each of the plurality of trajectories converted into reference coordinates; a trajectory integration process S14 in which the trajectory integration unit 14 integrates trajectories having a degree of correlation higher than a predetermined value; and a space-time synchronization process S15 in which the space-time synchronization unit 15 refers to the trajectories integrated by the trajectory integration unit 14, and calculates parameters for spatially and temporally synchronizing the images captured by the plurality of cameras.

[0033] In the information processing method S1, in the coordinate conversion process S12, the coordinate conversion unit 12 refers to the parameters calculated by the space-time synchronization unit 15 and converts each of the multiple trajectories into reference coordinates.

[0034] Therefore, according to the information processing method S1, the same effects as those of the information processing device 1 described above can be obtained.

[0035] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technical means employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technical means employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technical means shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0036] (Overview of information processing device 2) The information processing device 2 is a device that predicts the trajectory of an object included in at least one of a plurality of target images captured by each of a plurality of cameras. Furthermore, the information processing device 2 refers to the predicted trajectory and calculates parameters for spatially and temporally synchronizing the images captured by each of the plurality of cameras.

[0037] An example of an outline of a process in which the information processing device 2 predicts a trajectory and calculates parameters by referring to the predicted trajectory will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of an outline of a process in which the information processing device 2 predicts a trajectory and calculates parameters by referring to the predicted trajectory.

[0038] 3 shows a state in which a plurality of cameras CA1 to CA4 capture images of an intersection. Information processing device 2 acquires target images captured by each of the plurality of cameras CA1 to CA4 and predicts the trajectory of an object included in at least one of the target images. For example, information processing device 2 predicts the trajectory of person OB1 included in the target images captured by each of the plurality of cameras CA1 to CA4.

[0039] 3, in the image captured by camera CA3, the position of car OB4 is in an occlusion state, hidden behind building OB5. Even in such a case, information processing device 2 acquires target images captured by each of the multiple cameras CA1 to CA4 and predicts the trajectory of car OB4.

[0040] Furthermore, the information processing device 2 refers to the predicted trajectory and calculates parameters for spatially and temporally synchronizing the images captured by the cameras CA1 to CA4. As an example, the information processing device 2 calculates parameters to be referenced in order to spatially and temporally synchronize the images captured by the cameras CA1 to CA4.

[0041] For example, the information processing device 2 calculates, as a parameter for temporal synchronization, a parameter for referring to a plurality of target images taken at the same time among the plurality of target images taken by each of the cameras CA1 to CA4. As an example, the information processing device 2 calculates a parameter for referring to identify a plurality of target images taken at the same time among the plurality of target images taken by each of the cameras CA1 to CA4.

[0042] In addition, when person OB1 is included in multiple target images taken at the same time, the information processing device 2 calculates parameters for spatially synchronizing the positions of person OB1 included in each of the multiple target images as the same position.

[0043] Examples of the calculated parameters include, but are not limited to, rotation, translation, focal length, camera center coordinates, and camera lens distortion.

[0044] (Configuration of information processing device 2) The configuration of the information processing device 2 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the information processing device 2. As shown in Fig. 4, the information processing device 2 includes a control unit 20, a storage unit 30, an input / output unit 40, and a communication unit 50.

[0045] (Storage unit 30) The storage unit 30 stores data referenced by the control unit 20. Examples of the storage unit 30 include, but are not limited to, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0046] Examples of data stored in the storage unit 30 include, but are not limited to, a target image TP, a state transition model ODM, trajectory information LCI, converted trajectory information C_LCI, integrated trajectory information I_CLI, parameters PA, and identification information ID. The target image TP and parameters PA are as described above. The trajectory information LCI, converted trajectory information C_LCI, integrated trajectory information I_CLI, and identification information ID will be described later.

[0047] The ODM is a machine-learned model that takes an image as input and predicts the state of an object contained in the image. As an example, the ODM is a model that takes multiple images as input and is trained to predict the trajectory of an object contained in the image.

[0048] More specifically, the state transition model ODM receives an image as input, detects an object contained in the image, and predicts the trajectory of the detected object.

[0049] (Input / output section 40) The input / output unit 40 is an interface with an input device that accepts input of data and an output device that outputs data. Examples of input devices include, but are not limited to, a microphone, a camera, an eye-gaze input device, a keyboard, and a touchpad. Examples of output devices include, but are not limited to, a speaker and a liquid crystal display.

[0050] (Communication unit 50) The communication unit 50 is an interface for transmitting and receiving data via a network. Examples of the communication unit 50 include, but are not limited to, communication chips for various communication standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), and wireless communication standards for mobile data communication networks, and a USB-compliant connector.

[0051] Furthermore, the specific configuration of the network is not particularly limited, but examples include a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination of these networks.

[0052] (Control unit 20) The control unit 20 controls each component included in the information processing device 2. As shown in Fig. 4, the control unit 20 also includes an acquisition unit 21, a trajectory prediction unit 11, a coordinate conversion unit 12, a trajectory correlation unit 13, a trajectory integration unit 14, a time-space synchronization unit 15, a learning unit 23, an identification information assignment unit 24, and an output unit 25. In this exemplary embodiment, the acquisition unit 21, the trajectory prediction unit 11, the coordinate conversion unit 12, the trajectory correlation unit 13, the trajectory integration unit 14, the time-space synchronization unit 15, the learning unit 23, and the identification information assignment unit 24 respectively realize an acquisition means, a trajectory prediction means, a coordinate conversion means, a trajectory correlation means, a trajectory integration means, a time-space synchronization means, a learning means, and an identification information assignment means.

[0053] (Acquisition part 21) The acquisition unit 21 acquires data supplied from the input / output unit 40 or the communication unit 50. The acquisition unit 21 stores the acquired data in the storage unit 30.

[0054] As an example, the acquisition unit 21 acquires a plurality of target images TP. For example, in FIG. 3 described above, a plurality of images captured by the camera CA1 over a predetermined period are acquired as the plurality of target images TP. Hereinafter, a plurality of images captured by a certain camera CA over a predetermined period will also be referred to as a moving image. Similarly, the acquisition unit 21 acquires a moving image captured by each of the cameras CA2 to CA4 as the plurality of target images TP.

[0055] (Trajectory Prediction Unit 11) The trajectory prediction unit 11 predicts a trajectory lc of an object included in an image, and stores in the storage unit 30 trajectory information LCI indicating one or more predicted trajectories lc.

[0056] As an example, the trajectory prediction unit 11 refers to a plurality of target images TP and predicts a trajectory lc of an object included in at least one of the plurality of target images TP. For example, in the above-mentioned Fig. 3, the trajectory prediction unit 11 refers to a plurality of target images TP (moving images) captured by each of cameras CA1 to CA4 over a predetermined period and predicts a trajectory lc of an object (person OB1, person OB2, car OB3, and car OB4) included in at least one of the plurality of target images TP.

[0057] For example, if a person OB2 and a car OB3 are included in a video 1 captured by a camera CA1 from time tn (n is 2 or more) to time t-1, the trajectory prediction unit 11 refers to the video 1 and predicts the trajectory lc_OB2 of the person OB2 and the trajectory lc_OB3 of the car OB3 from time t onwards.

[0058] Similarly, if a person OB1 and a car OB3 are included in the video 2 captured by the camera CA2 from time tn to time t-1, the trajectory prediction unit 11 refers to the video 2 and predicts the trajectory lc_OB1 of the person OB1 and the trajectory lc_OB3 of the car OB3 from time t onwards.

[0059] That is, the trajectory prediction unit 11 predicts, for each camera CA, the trajectory lc of an object included in the video captured by that camera CA.

[0060] The trajectory prediction unit 11 may also predict the trajectory lc of an object by inputting the target image TP into a state transition model ODM that takes an image as input and predicts the state of an object included in the image.

[0061] As shown in FIG. 4, the trajectory prediction unit 11 includes an object detection unit 111, a trajectory candidate prediction unit 112, a correlation unit 113, and a trajectory determination unit 114.

[0062] The object detection unit 111 detects an object included in an image. As an example, the object detection unit 111 detects an object included in the target image TP by inputting the target image TP into a state transition model ODM. As another example, the object detection unit 111 detects an object included in the target image TP using an object detection model (e.g., YOLOX) or a generation AI (e.g., ChatGPT4, Gemini).

[0063] The object detection unit 111 supplies information indicating the detected object to the trajectory candidate prediction unit 112. As an example, the object detection unit 111 supplies an image in which an object included in the target image TP is surrounded by a rectangle to the trajectory candidate prediction unit 112 as information indicating the detected object.

[0064] The trajectory candidate prediction unit 112 predicts one or more trajectory candidates of an object included in an image. The trajectory candidate prediction unit 112 supplies the predicted one or more trajectory candidates to the correlation unit 113. As an example, the trajectory candidate prediction unit 112 inputs, into the state transition model ODM, a plurality of target images TP (moving images) captured from time tn to time t-1 and information indicating the object detected by the object detection unit 111 for each of the plurality of target images TP, thereby predicting one or more trajectory candidates of the object detected by the object detection unit 111 from time t onwards.

[0065] The correlation unit 113 calculates the degree of correlation between the position of the detected object and the position of the object in one or more trajectory candidates. The correlation unit 113 supplies the calculated degree of correlation to the trajectory determination unit 114. As an example, the correlation unit 113 calculates, as the degree of correlation, the difference between a rectangle surrounding the object OB detected by the object detection unit 111 and included in the target image TP captured at time t and the position of the object OB at time t based on one or more trajectory candidates of the object OB predicted by the trajectory candidate prediction unit 112 from the video captured from time tn to time t−1.

[0066] The trajectory determination unit 114 determines one or more trajectories lc of an object included in the image. The trajectory determination unit 114 stores trajectory information LCI indicating the determined one or more trajectories lc in the storage unit 30. For example, the trajectory determination unit 114 determines one or more trajectories for which the degree of correlation calculated by the correlation unit 113 is equal to or greater than a threshold as one or more trajectories lc of an object included in the image.

[0067] Furthermore, the trajectory determination unit 114 determines the trajectory of the object by referring to the identification information ID assigned by the identification information assignment unit 24, which will be described later. An example of the process in which the trajectory determination unit 114 refers to the identification information ID will be described later.

[0068] (Coordinate conversion unit 12) The coordinate conversion unit 12 converts the trajectory lc into reference coordinates. As an example, the coordinate conversion unit 12 converts each of the multiple trajectories lc predicted by the trajectory prediction unit 11 into reference coordinates using a parameter PA calculated by the spatiotemporal synchronization unit 15 (described later). The coordinate conversion unit 12 stores converted trajectory information C_LCI ​​indicating the converted trajectory c_cl in the storage unit 30.

[0069] An example of a method by which the coordinate transformation unit 12 transforms the trajectory cl into the reference coordinates is to use the parameter PA to transform the trajectory cl into the reference coordinates by affine transformation or homography transformation, but is not limited to these.

[0070] The reference coordinates are not particularly limited. As an example, if the multiple cameras CA include a camera CA that captures an object OB from above, the coordinate conversion unit 12 converts each of the multiple trajectories lc into reference coordinates using the target image TP captured by the camera CA that captures from above as a reference. In other words, the coordinate conversion unit 12 sets the coordinate system of the target image TP captured by the camera CA that captures from above as a reference coordinate system, and converts the trajectory lc predicted with reference to the target images TP captured by the other cameras CA into coordinates in the reference coordinate system. With this configuration, the information processing device 2 can convert the trajectory lc predicted from the target images TP captured by the other cameras CA into reference coordinates using the target image TP that captures the entire movement of the object as a reference.

[0071] Furthermore, at least a portion of the imaging range of each of the multiple cameras CA may overlap at least a portion of the imaging range of the other cameras.

[0072] (Trajectory correlation unit 13) The trajectory correlation unit 13 calculates the degree of correlation between the trajectories lc. As an example, the trajectory correlation unit 13 calculates the degree of correlation between each of the multiple transformed trajectories c_lc converted into the reference coordinates by the coordinate conversion unit 12 and other transformed trajectories c_cl. The trajectory correlation unit 13 supplies the calculated degree of correlation to the trajectory integration unit 14.

[0073] For example, the trajectory correlation unit 13 calculates the degree of similarity between the trajectories as the degree of correlation. As an example, the trajectory correlation unit 13 calculates the distance between the trajectories as the degree of correlation. In this case, the trajectory correlation unit 13 calculates the degree of correlation so that the shorter the distance between the trajectories, the higher the degree of correlation.

[0074] Furthermore, the trajectory correlation unit 13 may calculate the similarity of state variables such as the velocity of an object as the degree of correlation between the trajectories l and l. In this case, the trajectory correlation unit 13 calculates the degree of correlation such that the higher the similarity of the state variables, the higher the degree of correlation between the trajectories l and l.

[0075] (Trajectory integration part 14) The trajectory integration unit 14 integrates multiple trajectories lc. As an example, the trajectory integration unit 14 integrates trajectories lc whose degree of correlation calculated by the trajectory correlation unit 13 is higher than a predetermined value. The trajectory integration unit 14 stores integrated trajectory information I_LCI indicating the integrated trajectory i_lc in the storage unit 30.

[0076] An example of a method for the trajectory integration unit 14 to integrate the trajectories lc is a method using optimization by a greedy method. Another example of a method for the trajectory integration unit 14 to integrate the trajectories is a method using discrete optimization.

[0077] (Space-time synchronization unit 15) The space-time synchronization unit 15 synchronizes the multiple trajectories lc in time and space. As an example, the space-time synchronization unit 15 refers to the integrated trajectory i_lc integrated by the trajectory integration unit 14, and calculates a parameter PA that spatially and temporally synchronizes the images captured by the multiple cameras CA. The space-time synchronization unit 15 stores the calculated parameter PA in the storage unit 30.

[0078] As described above, examples of the parameters PA calculated by the time-space synchronization unit 15 include, but are not limited to, rotation, translation, focal length, camera center coordinates, and camera lens distortion.

[0079] As an example of processing performed by the time-space synchronization unit 15, assume that the integrated trajectory i_lc integrated by the trajectory integration unit 14 is a trajectory lc_1 predicted from the image captured by camera CA1 and a trajectory lc_2 predicted from the image captured by camera CA2. In this case, the time-space synchronization unit 15 projects the trajectories lc_1 and lc_2 onto a certain plane. Next, the time-space synchronization unit 15 calculates the spatial and temporal differences between the trajectories lc_1 and lc_2 based on the respective positions and directions of the cameras CA1 and CA2. Then, the time-space synchronization unit 15 calculates a parameter PA in three-dimensional reference coordinates from the calculated differences.

[0080] Furthermore, as described above, when at least a portion of the shooting range of each of the multiple cameras CA overlaps with at least a portion of the shooting range of another camera, the spatiotemporal synchronization unit 15 can easily calculate the parameter PA for spatial and temporal synchronization by referring to the position of the object in the overlapping area.

[0081] (Study Section 23) The learning unit 23 trains the machine learning model. As an example, the learning unit 23 trains the state transition model ODM using the integrated trajectory i_lc integrated by the trajectory integration unit 14.

[0082] As an example of processing performed by the learning unit 23, the learning unit 23 models a linear or nonlinear state update equation based on the integrated trajectory i_lc, with the position and velocity of the object as conditions, and calculates parameters of the update equation by regression from accumulated data. For example, the learning unit 23 learns a conditional state transition model ODM, such as that a certain object moves in a certain direction at a certain location.

[0083] With this configuration, the learning unit 23 can learn a state transition model ODM that uses an image as input and predicts the trajectory of an object included in the image.

[0084] (Identification information assigning unit 24) The identification information assigning unit 24 associates an object included in an image with an identification information ID for distinguishing the object from other objects. As an example, the identification information assigning unit 24 associates an object included in at least one of a plurality of target images TP with an identification information ID for distinguishing the object from other objects.

[0085] For example, the identification information assigning unit 24 identifies an object included in the target image TP using an individual identification method based on biometric authentication such as face authentication or gait authentication. Furthermore, the identification information assigning unit 24 identifies an object included in the target image TP using a mechanical authentication method such as RFID (Radio Frequency Identification). The identification information assigning unit 24 associates identification information ID indicating the identified object with the identified object.

[0086] Furthermore, information related to a trajectory lc may be associated with the identification information ID associated with an object by the identification information assigning unit 24. For example, the identification information ID of a certain object may be associated with a trajectory lc along which the certain object has moved in the past.

[0087] In this case, the trajectory determination unit 114 determines the trajectory lc of the object by referring to the identification information ID. For example, if the similarity between one or more trajectories lc whose degree of correlation calculated by the correlation unit 113 is equal to or greater than a threshold and the trajectory lc associated with the identification information ID is equal to or greater than a threshold, the trajectory determination unit 114 determines the one or more trajectories lc as the trajectory of the object.

[0088] With this configuration, the trajectory determination unit 114 can determine the trajectory lc of the object with higher accuracy.

[0089] (Output section 25) The output unit 25 outputs data to the input / output unit 40 or the communication unit 50. As an example, the output unit 25 outputs the integrated trajectory i_lc to the input / output unit 40 or the communication unit 50.

[0090] (Flow of information processing method S2) The flow of the information processing method S2 executed by the information processing device 2 will be described with reference to Fig. 5. Fig. 5 is a flow diagram showing the flow of the information processing method S2.

[0091] (Step S21) In step S21, the acquisition unit 21 acquires a plurality of target images TP and stores the plurality of target images TP in the storage unit 30.

[0092] (Step S22) In step S22, the identification information assigning unit 24 associates the identification information ID of an object included in each of the target images TP with the object.

[0093] (Step S23) In step S23, the trajectory prediction unit 11 refers to the plurality of target images TP and predicts a trajectory lc of an object included in at least one of the plurality of target images TP. The trajectory prediction unit 11 stores trajectory information LCI indicating the predicted trajectory lc in the storage unit 30.

[0094] (Step S24) In step S24, the coordinate transformation unit 12 refers to the parameter PA and transforms each of the multiple trajectories lc predicted by the trajectory prediction unit 11 into reference coordinates. The coordinate transformation unit 12 stores in the storage unit 30 transformed trajectory information C_LCI ​​indicating the transformed trajectory c_cl.

[0095] An example of the trajectory c_cl after conversion will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of processing in which the information processing device 2 generates an integrated trajectory i_lc.

[0096] As shown in FIG. 6, it is assumed that in the target images TP captured by the cameras CA1 to CA3, an object is in an occlusion state where it is hidden behind an obstacle OBS1, an obstacle OBS2, or an obstacle OBS3.

[0097] 6, the trajectory prediction unit 11 predicts trajectories lc1_1 and lc1_2 with reference to a plurality of target images TP1 captured by camera CA1. Similarly, the trajectory prediction unit 11 predicts trajectories lc2 with reference to a plurality of target images TP2 captured by camera CA2. Furthermore, the trajectory prediction unit 11 predicts trajectories lc3_1 and lc3_2 with reference to a plurality of target images TP3 captured by camera CA3.

[0098] In this case, as shown in the upper part of Figure 6, the coordinate transformation unit 12 generates a trajectory c_lc1_1, a trajectory c_cl1_2, a trajectory c_lc2, a trajectory c_lc3_1, and a trajectory c_lc3_2 after converting each of the multiple trajectories lc into reference coordinates.

[0099] (Step S25) In step S25, the trajectory correlation unit 13 calculates the degree of correlation with other trajectories for each of the multiple transformed trajectories c_lc converted into the reference coordinates by the coordinate transformation unit 12. The trajectory correlation unit 13 supplies the calculated degree of correlation to the trajectory integration unit 14.

[0100] As an example, the trajectory correlation unit 13 first executes a first selection process to select one converted trajectory c_lc from among the multiple converted trajectories c_lc. Next, the trajectory correlation unit 13 executes a second selection process to select one or more converted trajectories c_lc that exist within a predetermined range from the converted trajectory c_lc selected in the first selection process. The predetermined range is not particularly limited, but an example is a range within which the trajectory correlation unit 13 determines that the trajectories are similar to each other.

[0101] Subsequently, the trajectory correlation unit 13 executes a degree calculation process to calculate the degree of correlation between the converted trajectory c_lc selected in the first selection process and each of one or more converted trajectories c_lc selected in the second selection process. The trajectory correlation unit 13 repeats the first selection process, the second selection process, and the degree calculation process until all converted trajectories c_lc have been selected in the first selection process.

[0102] For example, in the upper diagram of FIG. 6, the trajectory correlation unit 13 selects the converted trajectory c_lc1_1 in the first selection process.

[0103] Next, in a second selection process, the trajectory correlation unit 13 selects the converted trajectory c_lc3_1 and the converted trajectory c_lc2 as one or more converted trajectories c_lc that exist within a predetermined range from the converted trajectory c_lc1_1 selected in the first selection process.

[0104] Next, in a degree calculation process, the trajectory correlation unit 13 calculates the degree of correlation between the converted trajectory c_lc1_1 selected in the first selection process and each of the converted trajectories c_lc3_1 and c_lc2 selected in the second selection process.

[0105] As an example, assume that the trajectory correlation unit 13 calculates the degree of correlation between the converted trajectory c_lc1_1 and the converted trajectory c_lc3_1. In this case, the trajectory correlation unit 13 determines whether the converted trajectory c_lc3_1 exists within a predetermined range from a start point sp1_1, which is the point where the converted trajectory c_lc1_1 starts, and an end point ep1_1, which is the point where the converted trajectory c_lc1_1 ends.

[0106] As shown in the upper diagram of Fig. 6, the converted trajectory c_lc3_1 exists within a predetermined range from each of the start point sp1_1 and the end point ep1_1. Therefore, the trajectory correlation unit 13 calculates the degree of correlation between the converted trajectory c_lc1_1 and the converted trajectory c_lc3_1 in the section from the start point sp1_1 to the end point ep1_1. As shown in the upper diagram of Fig. 6, the degree of correlation between the converted trajectory c_lc1_1 and the converted trajectory c_lc3_1 is high in the section from the start point sp1_1 to the end point ep1_1.

[0107] As another example, assume that the trajectory correlation unit 13 calculates the degree of correlation between a converted trajectory c_lc1_1 and a converted trajectory c_lc2. In this case, too, the trajectory correlation unit 13 determines whether the converted trajectory c_lc2 exists within a predetermined range from each of the start point sp1_1 and the end point ep1_1 of the converted trajectory c_lc1_1.

[0108] Here, the transformed trajectory c_lc2 (start point sp2) exists within a predetermined range of the end point ep1_1, but the transformed trajectory c_lc2 does not exist within the predetermined range of the start point sp1_1. Therefore, the trajectory correlation unit 13 determines whether the transformed trajectory c_lc1_1 exists within a predetermined range from each of the start point sp2 and end point ep2 of the transformed trajectory c_lc2. In this case, too, the transformed trajectory c_lc1_1 (end point ep1_1) exists within the predetermined range of the start point sp2, but the transformed trajectory c_lc1_1 does not exist within the predetermined range of the end point ep2. In other words, in the transformed trajectory c_lc1_1 and the transformed trajectory c_lc2, the end point ep1_1 of the transformed trajectory c_lc1_1 and the start point sp2 of the transformed trajectory c_lc2 exist within a predetermined range of each other, but there are no sections in which the trajectories are similar. Therefore, the degree of correlation between the transformed trajectory c_lc1_1 and the transformed trajectory c_lc2 is low.

[0109] As another example, assume that the trajectory correlation unit 13 calculates the degree of correlation between the converted trajectory c_lc2 and the converted trajectory c_lc3_1. In this case, the trajectory correlation unit 13 also determines whether the converted trajectory c_lc2 exists within a predetermined range from each of the start point sp2 and the end point ep2 of the converted trajectory c_lc2.

[0110] Here, the transformed trajectory c_lc3_1 exists within a predetermined range from the start point sp2, but the transformed trajectory c_lc3_1 does not exist within a predetermined range from the end point ep2. Therefore, as in the above example, the trajectory correlation unit 13 determines whether the transformed trajectory c_lc2 exists within a predetermined range from each of the start point sp3_1 and the end point ep3_1 of the transformed trajectory c_lc3_1. In this case, the transformed trajectory c_lc2 does not exist within the predetermined range from the start point sp3_1, but does exist within a predetermined range from the end point ep3_1. In other words, the trajectories are similar in the section from the start point sp2 of the transformed trajectory c_lc2 to the end point ep3_1 of the transformed trajectory c_lc3_1. Therefore, the trajectory correlation unit 13 calculates the degree of correlation between the converted trajectory c_lc2 and the converted trajectory c_lc3_1 in the section from the start point sp2 of the converted trajectory c_lc2 to the end point ep3_1 of the converted trajectory c_lc3_1. As shown in the upper diagram of Fig. 6, the degree of correlation between the converted trajectory c_lc2 and the converted trajectory c_lc3_1 is high in the section from the start point sp2 of the converted trajectory c_lc2 to the end point ep3_1 of the converted trajectory c_lc3_1.

[0111] By performing the above processing, in the upper diagram of FIG. 6, the degree of correlation between the converted trajectory c_lc1_1 and the converted trajectory c_lc3_1 increases. Similarly, the degree of correlation between the converted trajectory c_lc2 and the trajectory c_lc3_1 increases. Furthermore, the degree of correlation between the converted trajectory c_lc2 and the converted trajectory c_lc3_2 increases. Furthermore, the degree of correlation between the converted trajectory c_lc1_2 and the converted trajectory c_lc3_2 increases.

[0112] (Step S26) In step S26, the trajectory integration unit 14 integrates the trajectories lc whose degree of correlation calculated by the trajectory correlation unit 13 is higher than a predetermined value. The trajectory integration unit 14 stores, in the storage unit 30, integrated trajectory information I_LCI indicating the integrated trajectory i_lc.

[0113] For example, as described above, if (1) the degree of correlation between the converted trajectory c_lc1_1 and the converted trajectory c_lc3_1, (2) the degree of correlation between the converted trajectory c_lc2 and the trajectory c_lc3_1, (3) the degree of correlation between the converted trajectory c_lc2 and the converted trajectory c_lc3_2, and (4) the degree of correlation between the converted trajectory c_lc1_2 and the converted trajectory c_lc3_2 are higher than a predetermined value, the trajectory integration unit 14 integrates the trajectories with a high degree of correlation and generates an integrated trajectory i_lc, as shown in the lower part of Figure 6.

[0114] (Step S27) In step S27, the space-time synchronization unit 15 calculates parameters PA for spatially and temporally synchronizing the images captured by the multiple cameras CA, with reference to the integrated trajectory i_lc integrated by the trajectory integration unit 14. The space-time synchronization unit 15 stores the calculated parameters PA in the storage unit 30.

[0115] (Step S28) In step S28, the learning unit 23 uses the integrated trajectory i_lc integrated by the trajectory integration unit 14 to learn the state transition model ODM.

[0116] (Step S29) In step S29, the output unit 25 outputs the integrated trajectory i_lc to the input / output unit 40 or the communication unit 50.

[0117] After executing step S29, the information processing device 2 returns to step S21 again and repeats the processes of steps S21 to S29.

[0118] Here, in step S23, the trajectory prediction unit 11 may be configured to sequentially predict the trajectory of the object by referring to a plurality of target images sequentially captured by each of a plurality of cameras CA.

[0119] That is, in step S21, the acquisition unit 21 acquires a plurality of target images TP sequentially captured by each of the plurality of cameras CA. For example, in step S21, the acquisition unit 21 acquires a plurality of target images TP1 captured during a period from time t-1 to time t. In addition, in step S23, the trajectory prediction unit 11 predicts a trajectory lc1 of an object by referring to the plurality of target images TP1.

[0120] After step S29 is executed, in step S21, the acquisition unit 21 again acquires a plurality of target images TP2 captured during the period from time t to time t+1. In addition, in step S23, the trajectory prediction unit 11 predicts a trajectory lc2 of the object by referring to the plurality of target images TP2.

[0121] With this configuration, the information processing device 2 can predict the trajectory of an object included in the target images TP that are sequentially captured.

[0122] Furthermore, the information processing device 2 may repeatedly execute steps S21 to S29 using a plurality of target images captured during a predetermined period.

[0123] That is, in step S21, the acquisition unit 21 acquires a plurality of target images TP captured during a predetermined period and stored in the storage unit 30. Then, using the plurality of target images TP, the information processing device 2 repeatedly executes the following processes: trajectory prediction processing by the trajectory prediction unit 11; conversion processing to reference coordinates with reference to the parameters PA by the coordinate conversion unit 12; calculation processing of the degree of correlation by the trajectory correlation unit 13; integration processing of trajectories by the trajectory integration unit 14; and parameter calculation processing by the time-space synchronization unit 15.

[0124] With this configuration, the information processing device 2 repeatedly executes the process, and the coordinate transformation unit 12 transforms the trajectory lc into the reference coordinates using the updated parameters, thereby enabling the information processing device 2 to predict the integrated trajectory i_lc with higher accuracy.

[0125] (Effects of information processing device 2) As described above, the information processing device 2 can generate the integrated trajectory i_lc of the object even if the object is occluded by obstacles OBS1 to OBS3 as shown in Fig. 6. Therefore, the information processing device 2 can predict the trajectory of the object with high accuracy.

[0126] [Software implementation example] Some or all of the functions of the information processing devices 1 and 2 (hereinafter also referred to as "the above-mentioned devices") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0127] In the latter case, each of the above devices is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 7. Figure 7 is a block diagram showing the hardware configuration of computer C that functions as each of the above devices.

[0128] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as each of the above-mentioned devices. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing the functions of each of the above-mentioned devices.

[0129] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0130] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0131] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0132] [Additional Notes] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0133] (Appendix A1) a trajectory prediction means for predicting, for each of the plurality of cameras, a trajectory of an object included in at least one of the plurality of target images by referring to the plurality of target images captured by the camera; a coordinate conversion means for converting each of the plurality of trajectories predicted by the trajectory prediction means into a reference coordinate; a trajectory correlation means for calculating a degree of correlation between each of the plurality of trajectories converted into the reference coordinates and other trajectories; a trajectory integration means for integrating trajectories having a degree of correlation higher than a predetermined value; a time-space synchronization means for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated by the trajectory integration means; Equipped with the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters; Information processing device.

[0134] (Appendix A2) the trajectory prediction means predicts a trajectory of an object by inputting the target image into a state transition model that receives an image and predicts a state of an object included in the image; The information processing device further includes a learning means for learning the state transition model using the trajectories integrated by the trajectory integration means. 10. The information processing device according to claim 1,

[0135] (Appendix A3) further comprising an identification information assigning means for associating, with an object, identification information for distinguishing the object included in at least one of the plurality of target images from other objects; the trajectory prediction means predicts the trajectory of the object by referring to the identification information; An information processing device according to appendix A1 or A2.

[0136] (Appendix A4) the plurality of cameras includes a camera that photographs the object from above, the coordinate conversion means converts each of the plurality of trajectories into a reference coordinate based on the target image captured by the camera capturing the image from above. An information processing device according to any one of appendices A1 to A3.

[0137] (Appendix A5) At least a portion of the imaging range of each of the plurality of cameras overlaps with at least a portion of the imaging range of the other cameras. An information processing device according to any one of appendices A1 to A4.

[0138] (Appendix A6) the trajectory prediction means sequentially predicts the trajectory of the object by referring to a plurality of target images sequentially captured by each of the plurality of cameras; An information processing device according to any one of appendices A1 to A5.

[0139] (Appendix A7) Using a plurality of target images taken by the plurality of cameras over a predetermined period of time, a trajectory prediction process by the trajectory prediction means; a conversion process to the reference coordinates by the coordinate conversion means, the conversion process referring to the parameters; a calculation process of the degree of correlation by the trajectory correlation means; a process of integrating trajectories by the trajectory integration means; A calculation process of the parameters by the space-time synchronization means; Repeatedly execute An information processing device according to any one of appendices A1 to A5.

[0140] (Appendix B1) At least one processor a trajectory prediction process for predicting a trajectory of an object included in at least one of a plurality of target images captured by each of a plurality of cameras, with reference to the plurality of target images captured by the camera; a coordinate transformation process for transforming each of the plurality of trajectories predicted in the trajectory prediction process into a reference coordinate; a trajectory correlation process for calculating a degree of correlation with other trajectories for each of the plurality of trajectories converted into the reference coordinates; a trajectory integration process for integrating trajectories having a degree of correlation higher than a predetermined value; a spatiotemporal synchronization process for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated in the trajectory integration process; Including, In the coordinate conversion process, the at least one processor converts each of the plurality of trajectories into the reference coordinates by referring to the parameters. Information processing methods.

[0141] (Appendix C1) A program that causes a computer to function as an information processing device, The computer a trajectory prediction means for predicting, for each of the plurality of cameras, a trajectory of an object included in at least one of the plurality of target images by referring to the plurality of target images captured by the camera; a coordinate conversion means for converting each of the plurality of trajectories predicted by the trajectory prediction means into a reference coordinate; a trajectory correlation means for calculating a degree of correlation between each of the plurality of trajectories converted into the reference coordinates and other trajectories; a trajectory integration means for integrating trajectories having a degree of correlation higher than a predetermined value; a time-space synchronization means for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated by the trajectory integration means; It functions as the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters; Information processing program. [Explanation of symbols]

[0142] 1, 2 Information processing device 11 Trajectory prediction section 12 Coordinate conversion section 13 Trajectory correlation section 14 Trajectory integration part 15 Space-time synchronization unit 21 Acquisition Department 23 Learning Department 24 Identification information assignment section lc locus c_lc converted trajectory i_lc integrated locus TP target image ODM State Transition Model LCI trajectory information C_LCI ​​Transformed trajectory information I_LCI Integrated Trajectory Information PA parameters ID Identification Information

Claims

1. a trajectory prediction means for predicting, for each of the plurality of cameras, a trajectory of an object included in at least one of the plurality of target images by referring to the plurality of target images captured by the camera; a coordinate conversion means for converting each of the plurality of trajectories predicted by the trajectory prediction means into a reference coordinate; a trajectory correlation means for calculating a degree of correlation between each of the plurality of trajectories converted into the reference coordinates and other trajectories; a trajectory integration means for integrating trajectories having a degree of correlation higher than a predetermined value; a time-space synchronization means for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated by the trajectory integration means; Equipped with the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters; Information processing device.

2. the trajectory prediction means predicts a trajectory of an object by inputting the target image into a state transition model that receives an image and predicts a state of an object included in the image; The information processing device further includes a learning means for learning the state transition model using the trajectories integrated by the trajectory integration means. The information processing device according to claim 1 .

3. further comprising an identification information assigning means for associating, with an object, identification information for distinguishing the object included in at least one of the plurality of target images from other objects; the trajectory prediction means predicts the trajectory of the object by referring to the identification information; 3. The information processing device according to claim 1.

4. the plurality of cameras includes a camera that photographs the object from above, the coordinate conversion means converts each of the plurality of trajectories into a reference coordinate based on the target image captured by the camera capturing the image from above.

3. The information processing device according to claim 1.

5. At least a portion of the imaging range of each of the plurality of cameras overlaps with at least a portion of the imaging range of the other cameras.

3. The information processing device according to claim 1.

6. the trajectory prediction means sequentially predicts the trajectory of the object by referring to a plurality of target images sequentially captured by each of the plurality of cameras; 3. The information processing device according to claim 1.

7. Using a plurality of target images taken by the plurality of cameras over a predetermined period of time, a trajectory prediction process by the trajectory prediction means; a conversion process to the reference coordinates by the coordinate conversion means, the conversion process referring to the parameters; a calculation process of the degree of correlation by the trajectory correlation means; a process of integrating trajectories by the trajectory integration means; A calculation process of the parameters by the space-time synchronization means; Repeatedly execute 3. The information processing device according to claim 1.

8. At least one processor a trajectory prediction process for predicting a trajectory of an object included in at least one of a plurality of target images captured by each of a plurality of cameras, with reference to the plurality of target images captured by the camera; a coordinate transformation process for transforming each of the plurality of trajectories predicted in the trajectory prediction process into a reference coordinate; a trajectory correlation process for calculating a degree of correlation with other trajectories for each of the plurality of trajectories converted into the reference coordinates; a trajectory integration process for integrating trajectories having a degree of correlation higher than a predetermined value; a spatiotemporal synchronization process for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated in the trajectory integration process; Including, In the coordinate conversion process, the at least one processor converts each of the plurality of trajectories into the reference coordinates by referring to the parameters. Information processing methods.

9. A program that causes a computer to function as an information processing device, The computer a trajectory prediction means for predicting, for each of the plurality of cameras, a trajectory of an object included in at least one of the plurality of target images by referring to the plurality of target images captured by the camera; a coordinate conversion means for converting each of the plurality of trajectories predicted by the trajectory prediction means into a reference coordinate; a trajectory correlation means for calculating a degree of correlation between each of the plurality of trajectories converted into the reference coordinates and other trajectories; a trajectory integration means for integrating trajectories having a degree of correlation higher than a predetermined value; a time-space synchronization means for calculating parameters for spatially and temporally synchronizing the images captured by the plurality of cameras by referring to the trajectories integrated by the trajectory integration means; It functions as the coordinate conversion means converts each of the plurality of trajectories into the reference coordinates by referring to the parameters; Information processing program.

Images