A device for predicting the position of a moving object, and a method for predicting the position of a moving object.

Abstract

This technology allows for more accurate prediction of the movement position of an object outside the camera's field of view. [Solution] The mobile object position prediction device acquires first detection data detected by a first sensor or second detection data detected by a second sensor, generates multiple particles that predict the position of the mobile object based on the first or second detection data and field particle data that has been collected in advance and stores the movement information of the mobile object moving around the area for each position coordinate, calculates the probability of the mobile object being present in the area based on each of the multiple particles, and generates and outputs a probability distribution that predicts the position of the mobile object.

Description

Technical Field

[0001] The present disclosure relates to a position prediction device for a moving body and a position prediction method for a moving body.

Background Art

[0002] Patent Document 1 discloses a predicted steering angle calculation device that calculates a predicted steering angle of a vehicle using a forward viewing point set in front of the vehicle and a course point group composed of a plurality of course coordinate points arranged along the center of the lane on a map. The predicted steering angle calculation device recognizes the position of the vehicle on the map, and based on the position of the vehicle on the map and the course point group, identifies the point near the vehicle position closest to the vehicle from the course point group, extracts the self-lane point group continuous with the point near the vehicle position from the course point group, identifies the point near the viewing point closest to the forward viewing point from the self-lane point group, and calculates the predicted steering angle of the vehicle based on the forward viewing point and the point near the viewing point.

[0003] Patent Document 2 discloses a system for monitoring the movement of an object in a transit terminal. The system includes a plurality of imaging devices arranged in the transit terminal, the imaging devices providing a continuous field of view for a moving object in any area of the transit terminal, and one or more object monitoring / tracking hosts communicating with one or more of the plurality of imaging devices, at least one host having a processor configured to perform repetitive processing of visual-based scene analysis to monitor the movement of an object within the transit terminal in real time.

Prior Art Documents

Patent Documents

[0004] <\

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present disclosure has been devised in view of the above-described conventional circumstances, and an object thereof is to provide a moving body position prediction device that more accurately predicts the moving position of a moving body outside the shooting angle of a camera, and a moving body position prediction method.

Means for Solving the Problems

[0006] The present disclosure includes an acquisition unit that acquires first detection data detected by a first sensor that detects the moving body from a first area within an area where the moving body moves, or second detection data detected by a second sensor that detects the moving body from a second area by a method different from that of the first sensor; a particle data generation unit that generates a plurality of particles for predicting the position of the moving body on the area based on the first detection data or the second detection data and field particle data that has been previously collected and stores the movement information of the moving body moving in the area for each position coordinate; and a prediction unit that calculates the existence probability of the moving body on the area based on each of the plurality of particles, and generates and outputs a probability distribution predicting the position of the moving body based on the calculated existence probability of the moving body. A moving body position prediction device is provided.

[0007] Furthermore, this disclosure provides a method for predicting the position of a moving object performed by at least one processor, comprising: acquiring first detection data detected by a first sensor that detects the moving object from a first area in the area in which the moving object is moving, or second detection data detected by a second sensor that detects the moving object from a second area using a method different from that of the first sensor; generating a plurality of particles that predict the position of the moving object on the area based on the first detection data or the second detection data and field particle data that has been collected in advance and stores the movement information of the moving object moving in the area for each position coordinate; calculating the probability of the presence of the moving object on the area based on each of the plurality of particles; and generating and outputting a probability distribution that predicts the position of the moving object based on the calculated probability of the presence of the moving object. [Effects of the Invention]

[0008] According to this disclosure, the position of a moving object outside the camera's field of view can be predicted with greater accuracy. [Brief explanation of the drawing]

[0009] [Figure 1] A top view showing an example of the arrangement of cameras and sensors installed in the yard in Embodiment 1. [Figure 2] This figure shows an example of the system configuration and internal configuration of the vehicle management system according to Embodiment 1. [Figure 3] Flowchart showing an example of the learning data generation procedure for the terminal device in Embodiment 1. [Figure 4] Figure showing an example of field particle data. [Figure 5] This figure shows an example of the system configuration and internal configuration of the vehicle management system according to Embodiment 2. [Figure 6] Flowchart showing an example of the procedure for predicting the vehicle position of the terminal device in Embodiment 2. [Figure 7] A diagram showing an example of vehicle particle data. [Figure 8]Flowchart showing an example of the passage sensor processing procedure for the terminal device in Embodiment 2. [Figure 9] A flowchart showing an example of the procedure for relearning field particle data for the terminal device in Embodiment 2. [Figure 10] Figure showing an example of the first output of the vehicle probability distribution. [Figure 11] Figure showing an example of the second output of the vehicle probability distribution. [Figure 12] A flowchart showing an example of the procedure for predicting the vehicle position of the terminal device in a modified example 1 of Embodiment 2. [Figure 13] A flowchart showing an example of the procedure for relearning field particle data of a terminal device in a modified example 1 of Embodiment 2. [Figure 14] A flowchart showing an example of the procedure for predicting the vehicle position of the terminal device in a modified example 2 of Embodiment 2. [Figure 15] A flowchart showing an example of the procedure for relearning field particle data of a terminal device in a modified example 2 of Embodiment 2. [Figure 16] A flowchart showing an example of the procedure for predicting the vehicle position of the terminal device in a modified example 3 of Embodiment 2. [Modes for carrying out the invention]

[0010] (Background leading to this disclosure) This disclosure assumes a scenario where the moving object whose temporal position changes are to be recorded is a trailer, within a large business premises (hereinafter sometimes referred to as a "yard") of a transportation company or the like, where tractors (so-called transport trucks) towing trailers (an example of vehicles) loaded with many goods are concentrated. In such a case, when a transport truck (in other words, a tractor) enters the yard (checks in) driven by the tractor driver, the trailer is parked in a designated parking space within the yard. However, the parking space where the trailer is actually parked may differ from the designated parking space. Conventionally, trailers that have entered the yard have been photographed with a camera, and each trailer has been tracked based on the position of the trailer captured in the image (see Patent Document 2).

[0011] However, tracking the movement of each trailer required installing many cameras within the yard, which presented challenges in terms of camera installation costs and maintenance. Therefore, there was a need for technology that could predict the trailer's position outside the camera's field of view.

[0012] Patent Document 1 recognizes the vehicle's position and azimuth angle obtained by a GPS receiver mounted on the vehicle, and calculates the predicted steering angle of the vehicle based on the recognized vehicle's position on a map and a course point cloud. However, Patent Document 1 only calculates the predicted steering angle of a single vehicle and does not envision calculating the predicted steering angle of each of the numerous tractors that have entered the yard and applying this to the management of the movement position of each tractor.

[0013] Hereinafter, with reference to the drawings as appropriate, each embodiment specifically disclosing the mobile body position prediction device and mobile body position prediction method according to this disclosure will be described in detail. However, unnecessarily detailed explanations may be omitted. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The accompanying drawings and the following explanation are provided to enable those skilled in the art to fully understand this disclosure and are not intended to limit the subject matter described in the claims.

[0014] (Embodiment 1) First, an example of a yard YRD will be described with reference to Figure 1. Figure 1 is a top view showing an example of the arrangement of cameras C1, C2 and sensors S1, S2, S3, S4 installed within the yard YRD. The configuration of the yard YRD, the number of cameras and sensors, and their arrangement shown in Figure 1 are examples only and are not limited thereto.

[0015] The yard YRD shown in Figure 1 is an example of the arrangement of cameras C1, C2 and sensors S1, S2, S3, S4 installed in an actual yard YRD where the vehicle management system 100A according to Embodiment 2 (see Figure 5) is introduced, in order to make the explanation of this disclosure easier to understand. In the vehicle management system 100 according to Embodiment 1 (see Figure 2), cameras are installed in place of sensors S1,... in the yard YRD shown in Figure 1, and based on the images captured by each camera, training data is collected and a training model is generated. Alternatively, a simulation is performed in which multiple vehicles TR move through a yard where cameras are installed in place of sensors S1,..., and based on the simulation results, training data is collected and a training model is generated.

[0016] Note that the number and arrangement of cameras used to collect training data in Embodiment 1 may differ from the number and arrangement of cameras in the actual yard YRD where Embodiment 2 is implemented. In the following description of Embodiment 1, an example will be described in which cameras (not shown) are placed at the positions of sensors S1,... in the yard YRD shown in Figure 1. In the following descriptions of Figures 1 and 2, sensors S1 to S4 can be read as "camera".

[0017] As shown in Figure 1, the yard (YRD) is a large plot of land owned by a transportation company or other business operator. Specifically, the yard (YRD) includes a gate (GT) for checking in and out of vehicles (TR) to the yard (YRD), parking areas (PAR1, PAR2) with multiple parking spaces, and a warehouse (WH) for unloading or loading goods onto the vehicles (TR).

[0018] During the generation of the learning model, the yard YRD is equipped with at least one sensor S1 to S4 for detecting vehicles TR moving on the roads within the yard YRD, and at least one camera C2 for capturing images of vehicles TR moving on the roads within the yard YRD.

[0019] The field of view ARC1 shown in Figure 1 represents the field of view of camera C1. The field of view ARC2 represents the field of view of camera C2. Furthermore, the detection area ARS1 represents the area in which sensor S1 can detect the passage of vehicle TR. The detection area ARS2 represents the area in which sensor S2 can detect the passage of vehicle TR. The detection area ARS3 represents the area in which sensor S3 can detect the passage of vehicle TR. The detection area ARS4 represents the area in which sensor S4 can detect the passage of vehicle TR. Note that each of the detection areas ARS1 to ARS4 shown in Figure 1 is replaced by the field of view of the camera in the vehicle management system 100 according to Embodiment 1.

[0020] Gate GT is through which vehicles TR entering the yard YRD and vehicles TR exiting the yard YRD pass. At least one camera C1 is installed at Gate GT. Camera C1 captures images of vehicles TR entering the yard YRD and vehicles TR exiting the yard YRD. Gate GT may also be equipped with a sensor (not shown) capable of reading an IC card or driver's license, etc., that can identify the driver of the vehicle TR that has entered.

[0021] Each of the multiple parking areas PAR1 and PAR2 has a parking space for parking one or more vehicles TR or towed objects (e.g., trailers) pulled by vehicles TR.

[0022] Warehouse WH stores cargo loaded onto or towed by a vehicle TR. Warehouse WH is equipped with at least one dock for loading or unloading cargo to or from the vehicle TR or towed by a vehicle TR.

[0023] Next, with reference to Figure 2, the vehicle management system 100 according to Embodiment 1 will be described. Figure 2 is a diagram showing an example of the system configuration and internal configuration of the vehicle management system 100 according to Embodiment 1.

[0024] The vehicle management system 100 according to Embodiment 1 generates a learning model in advance for predicting the movement position of a vehicle TR moving within the yard YRD. The vehicle management system 100 includes at least one camera C1,..., at least one sensor S1,..., and a terminal device P1.

[0025] Furthermore, when performing a simulation to move the vehicle TR in yard YRD to collect training data (field particle data) and generate a training model, the vehicle management system 100 may consist only of the terminal device P1, and the various detection data obtained by the cameras C1,... and sensors S1,... shown in Figures 2 to 4 may be replaced with the various detection data obtained from the simulation results.

[0026] Cameras C1,... capture images within their own field of view. Cameras C1,... generate vehicle TR detection data by associating the captured images with camera identification information that allows them to identify themselves, and transmit it to terminal device P1. If cameras C1,... have an image analysis function, cameras C1,... may detect vehicle TR from the captured images, associate the time the vehicle TR was detected (i.e., the time the image was captured), the position of the detected vehicle TR, and the camera identification information, and transmit this data to terminal device P1.

[0027] Sensors S1,... detect vehicles TR passing through their detection area. Sensors S1,... generate vehicle TR detection data by associating the time of detection of vehicle TR passage with identification information of sensors capable of identifying themselves, and transmit it to terminal device P1.

[0028] The sensors S1,... in this disclosure may be implemented as, for example, reflective or shielded near-infrared sensors, pyroelectric sensors, vibration sensors, ultrasonic sensors, thermographic sensors, radar sensors, Time of Flight (TOF) sensors, Light Detection and Ranging (LiDAR) sensors, or reflective or shielded paired near-infrared sensors.

[0029] Furthermore, the sensors S1,... of this disclosure have a passage detection function capable of detecting the passage of at least an object (vehicle TR) within a detection area. The sensors S1,... are installed to arbitrarily implement functions such as direction detection capable of detecting the direction of passage of an object (vehicle TR) passing through the detection area, dwell detection capable of detecting an object (vehicle TR) that is lingering (stopped) within the detection area, or speed detection capable of detecting the moving speed of an object (vehicle TR) passing through the detection area. In addition, the sensors S1,... may be configured to be unable or capable of implementing any detection function that one would like to achieve with each sensor, depending on the combination of multiple sensors or the installation method.

[0030] The terminal device P1 generates a learning model to predict the position of a vehicle TR moving in the actual yard YRD, based on detection data of a vehicle TR based on images captured by cameras C1,... and detection data of a vehicle TR detected by sensors S1,.... The terminal device P1 includes a communication unit 10, a processor 11, and a memory 12.

[0031] The communication unit 10 receives various data or information transmitted from each of the multiple cameras C1, ... and multiple sensors S1, ... and outputs it to the processor 11.

[0032] The processor 11 is configured using, for example, a Central Processing Unit (hereinafter referred to as "CPU"), a Digital Signal Processor (hereinafter referred to as "DSP"), or a Field Programmable Gate Array (hereinafter referred to as "FPGA"). The processor 11 works in cooperation with the memory 12 to perform various processing and control. Specifically, the processor 11 refers to the programs and data held in the memory 12 and executes those programs to realize various functions such as the learning model generation unit 111.

[0033] The learning model generation unit 111 performs image analysis on the captured images transmitted from cameras C1,... and acquires vehicle TR identification information (hereinafter referred to as "ID") and vehicle TR detection data based on the captured images. The learning model generation unit 111 also acquires vehicle TR detection data transmitted from sensors S1,... The learning model generation unit 111 generates a learning model based on the acquired vehicle TR detection data and detection data. The learning model generation unit 111 includes a field particle generation unit 111A and a learning unit 111B.

[0034] The field particle generation unit 111A calculates the speed and acceleration of the vehicle TR for each position of the vehicle TR based on the acquired detection data of the vehicle TR. The field particle generation unit 111A generates vehicle TR trajectory data obtained by arranging the position of the vehicle TR and the acceleration of the vehicle TR at that position in a time series. Based on the vehicle TR trajectory data, the field particle generation unit 111A generates field particle data indicating the state of the vehicle TR (vehicle TR speed, acceleration) for each position of the vehicle TR.

[0035] The field particle generation unit 111A divides the yard YRD map data (2D) into grids of predetermined distances (e.g., 2m, 4m, etc.), writes the generated field particle data to the grid positions corresponding to the vehicle TR's position, and generates field particle data Dt11 (see Figure 4). The field particle data Dt11 generated here is stored for each movement scenario in which vehicle TR detection data is acquired (data for "Field: 1", "Field: 2", etc. shown in Figure 2). The field particle data Dt11 will be explained in detail in Figure 4.

[0036] The learning unit 111B uses the field particle data Dt11 generated for each movement scenario as training data to generate a learning model that learns how vehicle TR moves in yard YRD. The field particles used as training data may be vehicle position information from a vehicle model simulated on a computer.

[0037] Memory 12 comprises Read Only Memory (hereinafter referred to as "ROM") and Random Access Memory (hereinafter referred to as "RAM"). ROM stores a program that defines the processing (operation) of the processor 11, and data that is referenced when the program is executed. RAM is work memory used when the processing (operation) of the processor 11, and temporarily stores data or information generated or acquired in each process.

[0038] Furthermore, memory 12 stores information such as map data of yard YRD, the positions and field of view of cameras C1,… installed within yard YRD, and the positions and detection areas of sensors S1,…. The map data of yard YRD may be 2D or 3D data.

[0039] <Method for generating field particle data> Next, with reference to Figures 3 and 4, the method for generating field particle training data and a training model in Embodiment 1 will be described. Figure 3 is a flowchart showing an example of the training data generation procedure for terminal device P1 in Embodiment 1. Figure 4 is a diagram showing an example of field particle data Dt11. In this disclosure, as an example, the speed of the vehicle TR is calculated and field particle data Dt11 including the calculated speed is generated, but the field particle data Dt11 only needs to include information on the acceleration of the vehicle TR and does not need to include information on the speed.

[0040] The learning model generation unit 111 of terminal device P1 generates field particles for all movement scenarios (St10) by combining the movement path pattern of a vehicle TR moving on roads within yard YRD with the movement pattern of roads within yard YRD (for example, the road that the vehicle TR proceeds (selects) from among multiple branched roads, the vehicle TR's position relative to the road width, the speed and acceleration when moving straight, or the speed and centripetal acceleration when turning left or right, etc.) as one movement scenario for the vehicle TR. The field particles referred to here are the training data used to train the learning model.

[0041] The learning model generation unit 111 acquires captured images (frames) taken by all cameras C1,... in a time series for one movement scenario and generates field particle data Dt11 corresponding to this one movement scenario (St20).

[0042] Specifically, the field particle generation unit 111A detects vehicles TR in the captured images (frames) taken by all cameras C1,... as time-series data through image analysis, and reads the vehicle ID of the detected vehicles TR. The vehicle ID is used for vehicle identification (distinguishing) processing. The field particle generation unit 111A also reads the position of the vehicles TR within the yard YRD (St21) based on the installation position (angle of view) of the camera that took the captured image and the position of the vehicles TR in the captured image.

[0043] The field particle generation unit 111A calculates which grid position the detected vehicle TR's position corresponds to (St22). The field particle generation unit 111A also calculates the vehicle TR's speed and acceleration based on the change in the vehicle TR's position (grid position) captured at different times and the time difference between when the vehicle TR was detected (i.e., when the captured image was taken) through identification processing using the vehicle ID (St22).

[0044] The field particle generation unit 111A writes the vehicle TR's velocity and acceleration to the calculated grid position of yard YRD, which is the movement information of the vehicle TR moving in the current scenario (St23).

[0045] The field particle generation unit 111A generates one field particle data Dt11 (for example, the "Field: 1" data shown in Figure 2) by performing the processing in steps St21 to St23 for all frames (imported images) in which one or more vehicles TR moving in the same movement scenario are captured (St20).

[0046] Here, the field particle data Dt11 will be described. The field particle data Dt11 is data generated by writing the data of each field particle (the speed and acceleration of the vehicle TR) to the corresponding grid position. The field particle generation unit 111A calculates the acceleration component (a x , a y ) and the speed component (v x , v y ) of the vehicle TR in the directions along the x-axis and y-axis of the map data (2D) of the yard YRD based on the calculated acceleration and speed of the vehicle TR. The field particle generation unit 111A writes the information of the calculated acceleration (a x , a y ) and speed (v x , v y ) of the vehicle TR to the corresponding grid position.

[0047] Note that the field particle data Dt11 shown in FIG. 4 shows an example where the number of grids (grid number) of the map data of the yard YRD is N G , and the number of field particles (particle number) obtained in one movement scenario is N F . Also, although not shown in the field particle data Dt11, each of the N G grids is associated with grid identification information that enables identification of the position of each grid with respect to the entire yard YRD.

[0048] When the field particle generation unit 111A determines that the generation of the field particle data Dt11 corresponding to each of all the movement scenarios (all the field particle data "Field: 1", "Field: 2",..., see FIG. 2) has been completed by the process of step St20 described above, it ends the loop process of step St10.

[0049] The learning unit 111B generates a learning model that learns the way of movement of the vehicle TR moving within the yard YRD based on the field particle data Dt11 corresponding to each of all the generated movement scenarios.

[0050] As described above, the vehicle management system 100 according to Embodiment 1 can collect field particle data Dt11 for each movement scenario obtained by combining the movement path pattern and movement pattern of the vehicle TR, that is, data on the speed and acceleration of the vehicle TR at each grid position in the yard YRD. Therefore, the vehicle management system 100 can generate a learning model that has learned various ways in which the vehicle TR moves within the yard YRD by learning all the collected field particle data Dt11 as training data.

[0051] (Embodiment 2) The first embodiment describes an example in which the vehicle management system 100 collects learning data (field particle data Dt11) showing various ways in which vehicles TR move within the yard YRD using real cameras C1,... or simulations, and generates a learning model that has learned from the learning data. The second embodiment describes an example in which the vehicle management system 100A predicts the position of each vehicle TR moving within the yard YRD based on the generated learning model and detection data of vehicles TR acquired by cameras C1,... or sensors S1,... installed in the yard YRD.

[0052] The vehicle management system 100A according to Embodiment 2 has the same configuration and functions as the vehicle management system 100 according to Embodiment 1. Therefore, in the description of the vehicle management system 100A according to Embodiment 2, the same reference numerals are assigned to the same components as in the vehicle management system 100 according to Embodiment 1, and the description is omitted.

[0053] First, with reference to Figure 5, the vehicle management system 100A according to Embodiment 2 will be described. Figure 5 is a diagram showing an example of the system configuration and internal configuration of the vehicle management system 100A according to Embodiment 2. Note that the terminal device P1A shown in Figure 5 is illustrated so that the vehicle particle data Dt12 is included in the processor 11A for the sake of clarity, but the vehicle particle data Dt12 may be stored in memory 12.

[0054] The vehicle management system 100A according to Embodiment 2, as shown in Figure 1, tracks the position of each vehicle TR moving within the yard YRD based on detection data detected by at least one camera C1,... and at least one sensor S1,..., and predicts and tracks the movement position of the vehicle TR outside the field of view of the cameras C1,.... The vehicle management system 100A includes at least one camera C1,..., at least one sensor S1,..., a terminal device P1, and an output unit DP. Note that the output unit DP is not a mandatory component and may be omitted.

[0055] The terminal device P1A predicts the movement position of a vehicle TR moving through the actual yard YRD based on a pre-generated learning model, detection data of a vehicle TR based on images captured by cameras C1,..., and detection data of a vehicle TR detected by sensors S1,.... The terminal device P1A includes a communication unit 10, a processor 11A, and a memory 12.

[0056] The processor 11A is configured using, for example, a CPU, DSP, or FPGA. The processor 11A works in cooperation with the memory 12 to perform various processes and controls. Specifically, the processor 11A refers to the programs and data held in the memory 12 and executes those programs to realize various functions such as the learning model generation unit 111, the coordinate transformation unit 112, the particle filter processing unit 113, or the probability distribution generation unit 114.

[0057] The learning model generation unit 111 generates training data used for retraining the learning model and performs retraining based on the generated training data, based on the data of multiple vehicle particles output from the probability distribution generation unit 114 (i.e., data indicating the position and velocity of vehicle TR). The learning model generation unit 111 includes a field particle generation unit 111C and a learning unit 111D.

[0058] The field particle generation unit 111C obtains information on the predicted position of the vehicle TR from the probability distribution generation unit 114. The field particle generation unit 111C calculates a grid position corresponding to the predicted position, and calculates the speed and acceleration of the vehicle TR based on the latest predicted position of the acquired vehicle TR and the previous predicted position of the vehicle TR with the same vehicle ID, and obtains this as field particle data. The field particle generation unit 111C writes the speed and acceleration of the vehicle TR based on the prediction results to the calculated grid position and updates the field particle data Dt11. Field particle data may be generated for each vehicle ID (for example, data for "Field:1", "Field:2", etc.).

[0059] The learning unit 111D uses the generated field particle data Dt11A as training data to retrain the current learning model.

[0060] The coordinate transformation unit 112 analyzes the captured images transmitted from each of the cameras C1, ... and acquires vehicle TR detection data (vehicle ID, vehicle position, etc.) based on the captured images. The learning model generation unit 111 also acquires vehicle TR detection data transmitted from sensors S1, ... The coordinate transformation unit 112 transforms the position of the vehicle TR on the captured image (i.e., on the image coordinate system) to a grid position on the yard YRD (i.e., on the world coordinate system) based on the camera position and field of view in which the vehicle TR was detected and the position of the vehicle TR on the captured image. The coordinate transformation unit 112 outputs the grid position information after coordinate transformation, the time when the vehicle TR was detected (captured) at this grid position, and the vehicle ID to the particle filter processing unit 113, associating them with each other.

[0061] The particle filter processing unit 113 predicts the position of the vehicle TR moving within the yard YRD by performing particle filter processing.

[0062] The particle filter processing unit 113 acquires detection data transmitted from each of the sensors S1,... (identification information and time of the sensor S1,... that detected the vehicle TR). The particle filter processing unit 113 also acquires the vehicle TR detection data output from the coordinate transformation unit 112. In addition, the particle filter processing unit 113 acquires a pre-generated or the latest learning model from the learning model generation unit 111.

[0063] The particle filter processing unit 113 predicts the position of a single vehicle TR based on the vehicle TR detection data from the camera C1 installed at gate GT and a learning model trained using field particle data Dt11. P Data for individual vehicle particles is generated. Each vehicle particle is data that includes the position of the vehicle TR, its velocity (acceleration), and a weight used to calculate the likelihood of the vehicle TR position indicated by this vehicle particle (see Figure 7). The particle filter processing unit 113 generates vehicle particle data Dt12 by writing the generated vehicle particle data for each vehicle TR and stores it in memory 12.

[0064] Furthermore, the particle filter processing unit 113 predicts the position and velocity (acceleration) of a single vehicle TR based on the vehicle TR detection data from camera C2 or sensor S1,..., the learning model, and field particle data Dt11A. P Each individual vehicle particle is updated, and vehicle particle data Dt12 is generated by writing the updated vehicle particles.

[0065] The probability distribution generation unit 114 calculates the likelihood of the vehicle particle data written to the vehicle particle data Dt12, and performs resampling of the vehicle particles based on the calculated likelihood. The probability distribution generation unit 114 calculates the probability of the presence of a vehicle TR for each location (grid) based on the mean particle position and frequency distribution of the vehicle particles after resampling. The probability distribution generation unit 114 generates a probability distribution of the presence of a vehicle TR as a predicted location of the vehicle TR based on the calculated existence probability for each location (grid). The probability distribution generation unit 114 generates screens DP1 and DP2 (see Figures 10 and 11) that visualize the probability distribution of each vehicle TR on the yard YRD, and outputs them to the output unit DP, or outputs the predicted location of the vehicle TR based on the probability distribution of the vehicle TR to the field particle generation unit 111C.

[0066] The output unit DP is configured using, for example, a Liquid Crystal Display (LCD) or an organic electroluminescence (EL). The output unit DP outputs a screen (for example, screen DP1 (see Figure 10) or screen DP2 (see Figure 11)) that includes the probability distribution of each vehicle TR output from the probability distribution generation unit 114.

[0067] <Method for generating probability distributions> Next, the method for predicting the position of the vehicle TR will be explained with reference to Figures 6 to 8. Figure 6 is a flowchart showing an example of the vehicle position prediction procedure of the terminal device P1A in Embodiment 2. Figure 7 is a diagram showing an example of vehicle particle data Dt12. Figure 8 is a flowchart showing an example of the passage sensor processing procedure of the terminal device P1A in Embodiment 2.

[0068] Note that the process in step St30 shown in Figure 6 is executed in a different thread than the process in step St50 shown in Figure 9.

[0069] <Method for predicting vehicle position> Terminal device P1A repeatedly executes the flow of step St30 shown in Figure 6 (i.e., the vehicle TR position prediction flow) at a speed corresponding to the frame rate of cameras C1,... (St30). Terminal device P1A repeatedly executes the process of step St30 until the yard YRD or the administrator of terminal device P1A commands the termination of the vehicle TR position prediction process in yard YRD.

[0070] First, the particle filter processing unit 113 of the terminal device P1A performs passage sensor processing (St31) to determine the weight of each of the multiple vehicle particles that predict the position of the vehicle TR, based on the detection data of the vehicle TR detected by sensors S1,....

[0071] The coordinate transformation unit 112 acquires vehicle TR detection data from camera C2 (i.e., a camera other than camera C1 which is imaging gate GT) (St32). Based on the position and field of view of camera C2 in which vehicle TR was detected, and the position of vehicle TR on the captured image, the coordinate transformation unit 112 converts the position of vehicle TR on the captured image (i.e., on the image coordinate system) to a grid position on the yard YRD (i.e., on the world coordinate system) (St33).

[0072] The particle filter processing unit 113 updates the data of vehicle particles used to predict the position of the vehicle TR and predicts the position of the vehicle TR for each vehicle TR (St40).

[0073] The particle filter processing unit 113 determines whether the vehicle TR with the vehicle ID for which the processing in step St40 is currently being performed has been detected by the camera C2 (St41).

[0074] If the particle filter processing unit 113 determines in step St41 that a vehicle TR has been detected by the camera C2 (St41, YES), it increases the weight of the vehicle particles included in the detection area where the vehicle TR was detected, i.e., the field of view of the camera where the vehicle TR was detected (St42). The weight referred to here is the weight value used to calculate the likelihood of the vehicle particles. The processing in step St42 may be performed only in the passage sensor processing (step St31), or it may be omitted as it is not mandatory.

[0075] On the other hand, if the particle filter processing unit 113 determines in step St41 that the vehicle TR is not detected by the camera C2 (St41, NO), it reduces the weight of the vehicle particles that are included in the camera's field of view and correspond to the detection area where the vehicle TR was detected by the sensor (St43). Note that "reduction" here may include subtraction or division, and is a process that does not result in a negative weight.

[0076] The particle filter processing unit 113 refers to the field particle data Dt11A generated (updated) in step St50 (see Figure 9) and randomly reads out one field particle from among the field particles written to the same grid position as the grid position to which the vehicle particle belongs (St44). The particle filter processing unit 113 performs this field particle reading process N P This is performed for each individual vehicle particle.

[0077] The particle filter processing unit 113 controls the N corresponding to the vehicle TR. P The data for each individual vehicle particle is read out as N P Update the data for each field particle (St45). Note that the "update" in step St45 refers to the read N P The data of each field particle contains the vehicle's acceleration component (a x ,a y ) and velocity component (v x ,v y Based on this, the position of the vehicle TR in the next frame (time (k+1)) is (x k+1 ,y k+1) and the speed of the vehicle TR (v x,k+1 ,v y,k+1 ) and calculate the position of the vehicle particle at the present time (time k) (x k ,y k ) and velocity (v x,k ,v y,k This process overwrites the weight w. k is weight w k+1 It will be updated as is.

[0078] (Equation 1) The position of the vehicle particle at time (k+1), which is the next frame (x k+1 ,y k+1 This is a formula for calculating noise (n). x ,n y ) is noise used to disperse the position of vehicle particles in particle filtering. Time Δt is the frame period corresponding to the camera's frame rate.

[0079]

number

[0080] (Equation 2) is the formula for calculating the velocity of the vehicle particle at time (k+1). Noise (n vx ,n vy ) is noise used to disperse the velocity of vehicle particles in particle filtering. Processor 11A calculates the acceleration of vehicle particles at time (k+1) based on the velocity of vehicle particles calculated using (Equation 2).

[0081]

number

[0082] The particle filter processing unit 113 determines whether or not the vehicle TR corresponding to the vehicle ID for which the processing in step St40 is currently being performed has been detected by the camera C2 (St46).

[0083] If the particle filter processing unit 113 determines in step St46 that a vehicle TR has been detected by camera C2 (St46, YES), it calculates the likelihood of the latest vehicle particle for each vehicle particle based on the distance between the position of the vehicle TR detected by camera C2 and the position of the vehicle particle (St47). The particle filter processing unit 113 calculates the weight w of the j-th vehicle particle. j The position of the vehicle particle (x j ,y j ) and the detection position of the vehicle TR (x c ,y c The distance d between ) j Multiplying by a function using the following, we obtain the likelihood L of the updated vehicle particles. j Calculate.

[0084] Here, the likelihood L j Let's explain how to calculate (Equation 3), which is the likelihood L. j This is the formula for calculating (Equation 4). The position of the jth vehicle particle (x j ,y j ) and the detection position of the vehicle TR (x c ,y c The distance d between ) j This is the formula for calculating it.

[0085]

number

[0086]

number

[0087] On the other hand, if the particle filter processing unit 113 determines in step St46 that the vehicle TR is not detected by the camera C2 (St46, NO), it proceeds to the processing in step St48.

[0088] The particle filter processing unit 113 compares the likelihood of the latest vehicle particles with the likelihood of vehicle particles obtained from the field particle data Dt11A, and performs resampling to overwrite vehicle particles with low likelihood with vehicle particles with high likelihood (St48).

[0089] Specifically, the particle filter processing unit 113 uses (Equation 5) to determine N P Likelihood L of individual vehicle particles j Each of these is normalized. The particle filter processing unit 113 uses a uniform random number to calculate the likelihood L after normalization. j 'Having N P The process of selecting one vehicle particle from among the individual vehicle particles and returning it is called N P The process is repeated several times. The particle filter processing unit 113 processes the N extracted in each process. p Individual vehicle particles, new N of vehicle TR p By acquiring the vehicle particles individually, vehicle particles with a high likelihood are selected instead of those with a low likelihood. This allows the particle filter processing unit 113 to resample the vehicle particles.

[0090]

number

[0091] Furthermore, if the particle filter processing unit 113 determines in step St46 that the vehicle TR is not detected by the camera C2 (St46, NO), it calculates the likelihood based on the weights corresponding to each of the updated vehicle particle data and the likelihood based on the weights corresponding to each of the vehicle particle data before the update, and performs resampling of the vehicle particles (St48). The particle filter processing unit 113 updates the vehicle particle data Dt12 by writing the latest vehicle particle data (position, velocity, and weight) obtained by resampling for each vehicle TR (vehicle ID).

[0092] Here, we will explain the vehicle particle data Dt12. The vehicle particle data Dt12 is the vehicle ID of each vehicle TR (the number of vehicles shown in Figure 7, "1", "2", ..., "N"). V Corresponding to ", N P The position (x,y) and velocity component (v) of each vehicle particle. x ,v yThis data is generated by writing the vehicle ID and weight w. The management of the vehicle ID in the vehicle particle data Dt12 is performed by the particle filter processing unit 113 based on the analysis results of the detection data from the camera C1 that images the gate GT. Specifically, when the particle filter processing unit 113 determines that a new vehicle TR has entered the yard YRD, it reads the vehicle ID of this vehicle TR and generates data for the vehicle particle corresponding to the read vehicle ID. Furthermore, when the particle filter processing unit 113 determines that a vehicle TR has left the yard YRD, it may delete the vehicle ID of this vehicle TR and the data for the vehicle particle corresponding to the vehicle ID.

[0093] The probability distribution generation unit 114 calculates the probability of vehicle TR being present at each location (grid) based on the particle position mean and frequency distribution of vehicle particles. Based on the calculated existence probabilities for each location (grid), the probability distribution generation unit 114 generates a probability distribution (St49) in which vehicle TR is present as the predicted location of vehicle TR. The probability distribution generation unit 114 generates a screen showing the predicted probability distribution of vehicle TR (i.e., the predicted location) and outputs it to the output unit DP. The probability distribution generation unit 114 also further calculates the predicted speed of vehicle TR and outputs information on the predicted location of vehicle TR based on the probability distribution of vehicle TR to the field particle generation unit 111C.

[0094] If the processor 11A determines that it has completed the prediction of the position of each vehicle TR (vehicle ID) through the processing in steps St41 to St49, it terminates the loop processing in step St40.

[0095] The processor 11A receives instructions for prediction processing from the administrator, and if it determines that the prediction processing for each vehicle TR has been completed for all frames through the processing in steps St31 to St33 and step St40, it terminates the flow in step St30 shown in Figure 6.

[0096] <Passage Sensor Processing Method> Next, referring to Figure 8, we will explain the process of step St31 (pass sensor processing) shown in Figure 6.

[0097] The particle filter processing unit 113 acquires detection data of the vehicle TR detected by one of the sensors (St312). Based on the detection data acquired from the sensors, the particle filter processing unit 113 determines whether the vehicle TR has been detected by the sensor and whether there are vehicle particles of the vehicle TR that are included in the detection area of ​​the sensor (St313). Alternatively, the particle filter processing unit 113 may determine whether the vehicle TR has been detected by the sensor based on the current position (estimated position) of the vehicle TR.

[0098] In step St313, if the particle filter processing unit 113 determines that a vehicle TR has been detected by the sensor and that there are vehicle particles of the vehicle TR included in the detection area of ​​the sensor (St313, YES), it increases the weight of the vehicle particles of the vehicle TR included in the detection area (St314).

[0099] On the other hand, in step St313, if the particle filter processing unit 113 determines that the vehicle TR is not detected by the sensor and that there are vehicle particles of the vehicle TR included in the detection area of ​​the sensor (St313, NO), it reduces the weight of the vehicle particles included in the detection area (St315). Here, the weight of the vehicle particles after the reduction process is determined by dividing or subtracting the current weight of the vehicle particles by a predetermined value (for example, a constant such as 2 or 8) so that it does not become a negative value. Note that the processing in step St315 is not mandatory and may be omitted.

[0100] The particle filter processing unit 113 performs the processes described in steps St312 to St315 for all sensors installed in yard YRD. When the particle filter processing unit 113 determines that the processes in steps St312 to St315 have been completed for all sensors, it terminates the loop processing of step St31 (pass sensor processing) shown in Figure 8.

[0101] As described above, the terminal device P1A according to Embodiment 2 updates the information of the acceleration of the grid position corresponding to the position of the vehicle particle based on the pre-generated field particle data Dt11A. This allows the vehicle particle to be assigned the acceleration component when the vehicle TR is moving straight if the grid position is a position where the vehicle TR is moving straight, and the centripetal acceleration component when the vehicle TR is turning if the grid position is a position where the vehicle TR is turning.

[0102] Therefore, each vehicle particle moves (changes) in a frame according to the updated acceleration, so that the position of each vehicle particle in each frame matches the actual movement of the vehicle TR. As a result, terminal device P1A can predict the position of the vehicle TR even when the position of the vehicle TR cannot be detected by camera C2. Thus, a yard YRD in which the vehicle management system 100A is introduced can predict and manage the position of each vehicle TR in the yard YRD in real time without having to install a large number of cameras to capture images of the entire movement route of the vehicle TR within the yard YRD.

[0103] Furthermore, as described above, the vehicle management system 100A according to Embodiment 2 can predict the position of vehicle TR even when vehicle TR is within the field of view of camera C2 but vehicle ID of vehicle TR cannot be detected, or when vehicle TR cannot be detected due to an obstacle (for example, another vehicle).

[0104] <Method 1 for relearning field particle data> Next, with reference to Figure 9, the method for updating the field particle data Dt11A in Embodiment 2 will be described. Figure 9 is a flowchart showing an example of the retraining procedure for the field particle data Dt11A of the terminal device P1A in Embodiment 2.

[0105] The learning model generation unit 111 of terminal device P1A executes the process of step St50 (field particle data update process) in a different thread from the process of step St30 shown in Figure 6.

[0106] The field particle generation unit 111C acquires the predicted position of the vehicle TR, which is detected by camera C2 (i.e., a camera other than camera C1 that is imaging gate GT), from the probability distribution generation unit 114 (St52). Here, the predicted position of the vehicle TR output from the probability distribution generation unit 114 (information on the position of the vehicle TR indicated by the probability distribution) is acquired as a position on world coordinates (on yard YRD).

[0107] The field particle generation unit 111C calculates a grid position corresponding to the predicted position of the vehicle TR (St53). Note that the predicted positions for which grid positions are calculated here may be only those positions whose probability of existence in the probability distribution is calculated to be greater than or equal to a predetermined probability. The field particle generation unit 111C calculates the speed and acceleration of the vehicle TR based on the difference between the grid position of the vehicle TR at the last acquired time k and the grid position of the vehicle TR at the latest time (k+1) (St53). The field particle generation unit 111C generates field particle data based on the calculated grid position and the speed and acceleration of the vehicle TR at this grid position.

[0108] The learning unit 111D writes the generated field particle data to the field particle data Dt11A and updates the field particle data Dt11A (St54). The learning unit 111D then uses the updated field particle data Dt11A to retrain the currently used learning model (St54).

[0109] The learning model generation unit 111 performs the processes described in steps St52 to St54 for all vehicles detected by camera C2 (i.e., cameras other than camera C1 which is imaging gate GT). When the processor 11A determines that the processes in steps St52 to St54 have been completed for all vehicles TR detected by camera C2, it terminates the loop processing of step St50 (field learning process) shown in Figure 13.

[0110] As described above, the vehicle management system 100A according to Embodiment 2 can update field particle data Dt11A to be more suitable for the operation of the yard YRD where the vehicle management system 100A is installed, based on a probability distribution showing the probability of the existence of vehicle TR acquired during operation. By updating the field particle data Dt11A and retraining the learning model using the updated field particle data Dt11A, the vehicle management system 100A can generate a learning model that is more suitable for the actual yard YRD. As a result, the vehicle management system 100A can further improve the accuracy of predicting the position of vehicle TR outside the field of view of camera C2 installed in the yard YRD.

[0111] Next, an example of the output of the probability distribution of vehicle TR will be explained with reference to Figure 10. Figure 10 shows a first example of the output of the probability distribution of vehicle TR. Note that the screen DP1 shown in Figure 10 is just one example and is not limited to this.

[0112] The probability distribution generation unit 114 is N P Based on the likelihood of each individual vehicle particle, N corresponds to the vehicle TR. P A screen DP1 is generated by color-mapping each of the individual vehicle particles onto the yard YRD map data. Note that, as shown in Figure 10, the yard YRD map data may be a 3D map. The number of vehicle particles displayed on screen DP1 is N. P It is not limited to individual particles; for example, only vehicle particles with a predetermined weight or likelihood level or higher may be displayed.

[0113] Screen DP1 displays the camera's viewpoint (field of view) generated by Computer Graphics (CG), or the image captured by the actually installed camera, with N P Each individual vehicle particle contains a set of probability distributions PRT11 to which it is color-mapped. Screen DP1 visualizes the set of probability distributions PRT11 that predict the position of vehicle TR based on the position and color of the vehicle particles. This allows administrators to see at a glance the predicted location where vehicle TR is located and the likelihood (probability) of that predicted location, based on the probability distribution of vehicle TR.

[0114] Next, an example of the output of the probability distribution of vehicle TR will be explained with reference to Figure 11. Figure 11 shows a second example of the output of the probability distribution of vehicle TR. Note that the screen DP2 shown in Figure 11 is just one example and is not limited to this.

[0115] The probability distribution generation unit 114 is N P Based on the likelihood of each individual vehicle particle, N corresponds to the vehicle TR. P A screen DP2 is generated by color-mapping each individual vehicle particle onto the yard YRD map data (2D). Although not shown in Figure 11, screen DP2 may also include vehicle ID information for the vehicle TR corresponding to each probability distribution group in the neighborhood of each probability distribution group.

[0116] Screen DP2 is a screen that predicts the position of vehicle TR if it travels along travel route Rt11 and travel route Rt12 at the branching point shown in Figure 1. As a result of predicting the movement position of vehicle TR at the branching point, screen DP2, as predicted using field particle data (training data) collected in field particle data Dt11A, includes a probability distribution group PRT21 that shows the position of vehicle TR if it travels along travel route Rt11, and a probability distribution group PRT22 that shows the position of vehicle TR if it travels along travel route Rt12.

[0117] Screen DP2 visualizes, based on the color of the vehicle particles, the probability of vehicle TR moving along movement route Rt11 and the predicted position (probability distribution) of vehicle TR when moving along movement route Rt11, as well as the probability of vehicle TR moving along movement route Rt12 and the predicted position (probability distribution) of vehicle TR when moving along movement route Rt12.

[0118] Furthermore, terminal device P1A predicts which of the two routes, Rt11 or Rt12, vehicle TR took, based on the probability that the actual vehicle traveled along either Rt11 or Rt12, when field particles are collected from simulations or executions of all possible travel scenarios. Based on the acceleration information (acceleration component) written to the field particle data Dt11A, terminal device P1A can generate vehicle particles that reflect the probability that vehicle TR took route Rt11 and the probability that it took route Rt12, respectively.

[0119] Furthermore, the vehicle particles included in the probability distribution groups PRT21 and PRT22 visualized on screen DP2 have their weights and likelihoods updated based on the detection results of sensors S1 or S3 installed at the respective destinations of each travel route Rt11 and Rt12. Based on the updates of the weights and likelihoods of the vehicle particles included in the probability distribution groups PRT21 and PRT22, resampling is performed to overwrite vehicle particles with lower likelihoods with vehicle particles with higher likelihoods. As a result, only the vehicle particles included in the probability distribution group along the travel route where the passage of vehicle TR was detected (e.g., travel route Rt11) remain, and the vehicle particles included in the probability distribution group along the travel route where the passage of vehicle TR was not detected (e.g., travel route Rt12) are overwritten and deleted.

[0120] This allows terminal device P1A to predict the position of vehicle TR using detection data detected by the sensor. Terminal device P1A, in the cycle in which the processing of step St30 shown in Figure 6 is executed, receives the latest N P By generating a screen that color-maps each individual vehicle particle, it is possible to visualize both the position of vehicles captured by camera C2 (absolute position) and the position of vehicles outside the field of view of camera C2 (probability distribution).

[0121] (Modification 1 of Embodiment 2) The vehicle management system 100A according to Embodiment 2 shows an example in which the retraining of the learning model using the position of the vehicle TR detected by the camera C2 is performed at any time. The vehicle management system 100A according to Modification 1 of Embodiment 2 describes an example in which the retraining of the learning model using the position of the vehicle TR detected by the camera C2 is performed during the processing of step St30A.

[0122] The vehicle management system 100A according to Modification 1 of Embodiment 2 has the same configuration and functions as the vehicle management system 100A according to Embodiment 2. Therefore, a description of the configuration of the vehicle management system 100A according to Modification 1 of Embodiment 2 will be omitted.

[0123] <Method 2 for relearning field particle data> Referring to Figures 12 and 13, respectively, examples of field particle learning and output of the vehicle TR probability distribution in Modification 1 of Embodiment 2 will be described. Figure 12 is a flowchart of an example of the vehicle position prediction procedure for terminal device P1A in Modification 1 of Embodiment 2. Figure 13 is a flowchart of an example of the retraining procedure for field particle data Dt11A for terminal device P1A in Modifications 1 and 3 of Embodiment 2.

[0124] Note that the processes of steps St31 to St33 and St40 to St49 in step St30A shown in Figure 12 are the same as the processes of steps St31 to St33 and St40 to St49 shown in Figure 6, so their explanation is omitted here.

[0125] The particle filter processing unit 113 transforms the position of the vehicle TR detected by camera C2 (i.e., a camera other than camera C1 which is imaging gate GT) into a grid position on yard YRD (i.e., on the world coordinate system) (St33). The particle filter processing unit 113 outputs information of the predicted position based on the probability distribution of the vehicle TR to the field particle generation unit 111C of the learning model generation unit 111, causing it to execute the processing in step St55 (field learning processing, see Figure 13).

[0126] The field particle generation unit 111C calculates a grid position corresponding to the detected position of the vehicle TR (St53A). The field particle generation unit 111C may calculate only the grid positions where the probability of the vehicle TR's presence is greater than or equal to a predetermined probability in the acquired probability distribution of the vehicle TR. The field particle generation unit 111C calculates the vehicle TR's velocity and acceleration based on the difference between the grid position of the vehicle TR at the last acquired time k and the grid position of the vehicle TR at the most recent time (k+1) (St53A). The field particle generation unit 111C generates field particle data based on the calculated grid position and the vehicle TR's velocity and acceleration at this grid position.

[0127] The learning unit 111D writes the generated field particle data to the field particle data Dt11A and updates the field particle data Dt11A (St54A). The learning unit 111D uses the updated field particle data Dt11A to retrain the currently used learning model (St54A). The learning unit 111D outputs the data of the retrained learning model to the particle filter processing unit 113.

[0128] After the processing in step St55, the particle filter processing unit 113 uses the retrained learning model to perform the processing in step St40 (prediction of the vehicle TR's position).

[0129] As a result, in the modified example 1 of Embodiment 2, the terminal device P1A can update the learning model used to predict the position of the vehicle TR based on the latest detection data of the vehicle TR. The vehicle management system 100A can update the field particle data Dt11A and retrain the learning model using the updated field particle data Dt11A, thereby enabling prediction of the position of the vehicle TR using the latest learning model that has learned the actual movement of the vehicle TR.

[0130] (Modification 2 of Embodiment 2) The vehicle management system 100A according to Embodiment 2 and Modification 1 of Embodiment 2 demonstrates an example in which the learning model is retrained using the position of the vehicle TR detected by the camera C2. The vehicle management system 100A according to Modification 2 of Embodiment 2 describes an example in which the learning model is retrained using the predicted position (probability distribution) of the vehicle TR.

[0131] The vehicle management system 100A according to the modified example 2 of Embodiment 2 has the same configuration and functions as the vehicle management system 100A according to Embodiment 2. Therefore, the explanation of the configuration of the vehicle management system 100A according to the modified example 2 of Embodiment 2 will be omitted.

[0132] <Method 3 for relearning field particle data> Referring to Figures 14 and 15, respectively, examples of field particle learning and output of the probability distribution of the vehicle TR in Modification 2 of Embodiment 2 will be described. Figure 14 is a flowchart showing an example of the vehicle position prediction procedure for terminal device P1A in Modification 2 of Embodiment 2. Figure 15 is a flowchart showing an example of the retraining procedure for field particle data Dt11A of terminal device P1A in Modification 2 of Embodiment 2. Note that the processing of step St30B shown in Figure 14 and the processing of step St50B shown in Figure 15 are executed in different threads.

[0133] Note that the processes of steps St31 to St33 and St40 to St49 in step St30B shown in Figure 14 are the same as the processes of steps St31 to St33 and St40 to St49 shown in Figure 6, so their explanation is omitted here.

[0134] The probability distribution generation unit 114 of processor 11A performs position mean-frequency distribution analysis on each of the resampled vehicle particles and predicts the most likely position for vehicle TR (St49). The probability distribution generation unit 114 outputs the predicted vehicle TR position (probability distribution) information to the field particle generation unit 111C of the learning model generation unit 111, causing it to execute the processing in step St50B (field learning process, see Figure 15).

[0135] The field particle generation unit 111C acquires information on the predicted position (position in world coordinates) of the vehicle TR output from the probability distribution generation unit 114 (St52B). The field particle generation unit 111C calculates the grid position corresponding to the predicted position of the vehicle TR (St53B). The field particle generation unit 111C may calculate only the grid positions in the acquired probability distribution of the vehicle TR where the probability of the vehicle TR's existence is greater than or equal to a predetermined probability. The field particle generation unit 111C calculates the vehicle TR's velocity and acceleration based on the difference between the grid position of the vehicle TR at the last acquired time k and the grid position of the vehicle TR at the most recent time (k+1) (St53B). The field particle generation unit 111C generates field particle data based on the calculated grid position and the vehicle TR's velocity and acceleration at this grid position.

[0136] The learning unit 111D writes the generated field particle data to the field particle data Dt11A and updates the field particle data Dt11A (St54B). The learning unit 111D uses the updated field particle data Dt11A to retrain the currently used learning model (St54B). The learning unit 111D outputs the data of the retrained learning model to the particle filter processing unit 113.

[0137] The learning model generation unit 111 executes the processes of steps St53B to St54B described above, in accordance with the output of the predicted position of the vehicle TR by the probability distribution generation unit 114. When the learning model generation unit 111 determines that the processes of steps St53B to St54B have been completed for all the predicted positions of the vehicle TR output from the probability distribution generation unit 114, it terminates the loop processing of step St50B (field learning process) shown in Figure 15.

[0138] As a result, in the modified example 2 of Embodiment 2, the terminal device P1A can update the learning model used to predict the position of the vehicle TR based on the latest predicted position of the vehicle TR. By updating the field particle data Dt11A using the actually acquired predicted position of the vehicle TR and retraining the learning model using the updated field particle data Dt11A, the terminal device P1A can achieve prediction of the position of the vehicle TR using the latest learning model that is better suited to the actual yard YRD.

[0139] (Modification 3 of Embodiment 2) The vehicle management system 100A according to Modification 2 of Embodiment 2 shows an example in which the retraining of the learning model using the probability distribution (predicted position) of the vehicle TR is performed in separate threads for the vehicle TR prediction process and the field particle data retraining process. The vehicle management system 100A according to Modification 3 of Embodiment 2 describes an example in which the retraining of the learning model using the probability distribution (predicted position) of the vehicle TR is performed in the same thread for the vehicle TR prediction process.

[0140] The vehicle management system 100A according to the modified example 3 of Embodiment 2 has the same configuration and functions as the vehicle management system 100A according to Embodiment 2. Therefore, a description of the configuration of the vehicle management system 100A according to the modified example 3 of Embodiment 2 will be omitted.

[0141] <Method 4 for relearning field particle data> Referring to Figures 13 and 16, respectively, examples of field particle learning and vehicle TR probability distribution output in Modification 3 of Embodiment 2 will be described. Figure 16 is a flowchart showing an example of the vehicle position prediction procedure for terminal device P1A in Modification 3 of Embodiment 2.

[0142] Note that the processes of steps St31 to St33 and St40 to St49 in step St30B shown in Figure 14 are the same as the processes of steps St31 to St33 and St40 to St49 shown in Figure 6, so their explanation is omitted here.

[0143] The probability distribution generation unit 114 of the processor 11A performs position mean-frequency distribution analysis on each of the resampled vehicle particles and predicts the most likely position of the vehicle TR (St49). The probability distribution generation unit 114 outputs the predicted vehicle TR position information to the field particle generation unit 111C of the learning model generation unit 111, causing it to perform the processing in step St55 (field learning process, see Figure 13).

[0144] The field particle generation unit 111C calculates a grid position corresponding to the predicted position of the predicted vehicle TR (St53C). The field particle generation unit 111C may calculate only the grid positions where the probability of the vehicle TR's existence is greater than or equal to a predetermined probability in the acquired probability distribution of the vehicle TR. The field particle generation unit 111C calculates the vehicle TR's velocity and acceleration based on the difference between the grid position of the vehicle TR at the last acquired time k and the grid position of the vehicle TR at the most recent time (k+1) (St53C). The field particle generation unit 111C generates field particle data based on the calculated grid position and the vehicle TR's velocity and acceleration at this grid position.

[0145] The learning unit 111D writes the generated field particle data to the field particle data Dt11A and updates the field particle data Dt11A (St54C). The learning unit 111D uses the updated field particle data Dt11A to retrain the currently used learning model (St54C). The learning unit 111D outputs the data of the retrained learning model to the particle filter processing unit 113.

[0146] As a result, in the modified example 3 of Embodiment 2, the terminal device P1A can update the learning model used to predict the position of the vehicle TR based on the latest predicted position of the vehicle TR. By updating the field particle data Dt11A using the actually acquired predicted position of the vehicle TR and retraining the learning model using the updated field particle data Dt11A, the terminal device P1A can realize the prediction of the position (predicted position) of the vehicle TR using the latest learning model that is better suited to the actual yard YRD.

[0147] (Note) Based on the descriptions of the embodiments described above, the following technologies are disclosed.

[0148] (Technology 1) An acquisition unit (communication unit 10) acquires first detection data detected by a first sensor (camera C2) that detects the moving object (vehicle) from a first area (field of view ARC2) within the area (yard YRD) in which the moving object (vehicle) moves, or second detection data detected by a second sensor (sensors S1 to S4) that detects the moving object from a second area (detection area ARS1 to ARS4) using a method different from that of the first sensor (camera C2), A particle data generation unit (particle filter processing unit 113) generates a plurality of particles (vehicle particles) that predict the position of the moving object on the area (yard YRD) based on the first detection data or the second detection data and field particle (vehicle particle) data that has been collected in advance and stores the movement information of the moving object moving in the area (yard YRD) for each position coordinate, The system includes a prediction unit (probability distribution generation unit 114) that calculates the probability of the moving object being present in the area (yard YRD) based on each of the plurality of particles (vehicle particles), and generates and outputs a probability distribution that predicts the position of the moving object based on the calculated probability of the moving object being present. A device for predicting the position of a moving object (terminal device P1A). As a result, even when the camera C2 cannot constantly detect the position (absolute position) of the vehicle TR, the terminal device P1A can predict and track the location of the vehicle TR using a probability distribution based on the position of the vehicle particles. This allows the terminal device P1A to predict and manage the position of each vehicle TR in the yard YRD in real time without having to install numerous cameras to capture images of the entire movement route of the vehicle TR in the yard YRD.

[0149] (Technology 2) The movement information of the moving body is the acceleration information of the moving body. (Technology 1) A device for predicting the position of a moving object (terminal device P1A). As a result, terminal device P1A can predict the direction and position of vehicle TR in the next frame by generating vehicle particles based on the acceleration (acceleration component) of vehicle TR stored at each grid position on yard YRD.

[0150] (Technology 3) The particle (vehicle particle) includes position information predicting the position of the moving body and acceleration information of the moving body at the said position. The particle data generation unit (particle filter processing unit 113) refers to the field particle data, obtains acceleration information of the moving body collected at the same position coordinates as the position information of the particles (vehicle particles), and updates the acceleration information of the particles (vehicle particles) with the acceleration information of the moving body stored in the field particle data. The prediction unit (probability distribution generation unit 114) calculates the probability of the moving object being present in the area (yard YRD) based on each of the plurality of particles (vehicle particles) whose acceleration information of the moving object has been updated. (Technology 2) A device for predicting the position of a moving object (terminal device P1A). As a result, terminal device P1A can predict the direction and position of the vehicle TR in the next frame by updating the acceleration of the vehicle particle to match the acceleration (acceleration component) of the vehicle TR stored at the same position (grid position).

[0151] (Technology 4) The particle data generation unit (particle filter processing unit 113) is, When the first detection data is obtained, weighting is performed on the particles (vehicle particles) included in the first area (viewing angle ARC2). When the second detection data is obtained, weighting is performed on the particles (vehicle particles) included in the second area (detection area ARS1 to ARS4). (Technology 1) A device for predicting the position of a moving object (terminal device P1A). As a result, terminal device P1A can predict the position of vehicle TR with higher accuracy by increasing the weight of vehicle particles with a high probability of presence based on the detection data.

[0152] (Technology 5) The particle data generation unit (particle filter processing unit 113) samples the plurality of particles (vehicle particles) of the moving body based on the weighted particles (vehicle particles), The prediction unit (probability distribution generation unit 114) calculates the probability of existence of the moving object based on each of the plurality of particles (vehicle particles) after sampling. (Technology 4) A device for predicting the position of a moving object (terminal device P1A). As a result, terminal device P1A can sample vehicle particles based on their weights, overwriting vehicle particles with a low probability (likelihood) of vehicle TR being present at the predicted location with vehicle particles with a high probability. Therefore, terminal device P1A can predict the position of vehicle TR with higher accuracy using vehicle particles.

[0153] (Technology 6) The particle data generation unit (particle filter processing unit 113) calculates the likelihood of each of the plurality of particles (vehicle particles) based on the difference between the detection position of the moving body based on the first detection data and the position of the particles (vehicle particles), and samples the plurality of particles (vehicle particles) of the moving body based on the likelihood. The prediction unit (probability distribution generation unit 114) calculates the probability of existence of the moving object based on each of the plurality of particles (vehicle particles) after sampling. (Technology 1) A device for predicting the position of a moving object (terminal device P1A). As a result, terminal device P1A can calculate the likelihood based on the difference between the detected vehicle's position (absolute position) and the position predicted by each vehicle particle. This allows it to overwrite vehicle particles with a low probability (likelihood) of vehicle TR being present at the predicted position with vehicle particles that have a high likelihood. Therefore, terminal device P1A can predict the position of vehicle TR with higher accuracy using vehicle particles.

[0154] (Technology 7) The system further includes a learning unit (learning model generation unit 111) that learns the movement information of the moving object for each position coordinate based on the aforementioned probability distribution. A mobile object position prediction device (terminal device P1A) described in any one of (Technology 1) to (Technology 6). As a result, terminal device P1A can retrain the vehicle TR's movement information (position and acceleration) using the actually acquired predicted position of the vehicle TR, thereby enabling prediction of the vehicle TR's position (predicted position) using the latest learning model.

[0155] (Technology 8) The first sensor (camera C2) is a camera, The first detection data includes the identification information (vehicle ID) of the moving object and the detection position of the moving object. A mobile object position prediction device (terminal device P1A) as described in any one of (Technology 1) to (Technology 7). As a result, terminal device P1A can acquire vehicle identification and absolute position of vehicle TR using camera C2. Therefore, terminal device P1A can generate (update) vehicle particles based on the position of vehicle TR predicted by vehicle particles and the actual position of vehicle TR.

[0156] (Technology 9) The second sensors (sensors S1 to S4) are detection devices that detect the presence or absence of the moving object. A mobile object position prediction device (terminal device P1A) as described in any one of (Technology 1) to (Technology 8). As a result, even when the camera C2 cannot constantly detect the position (absolute position) of the vehicle TR, the terminal device P1A can generate (update) vehicle particles based on the detection results of a less expensive sensor than the camera. This allows the terminal device P1A to predict and manage the position of each vehicle TR in the yard YRD in real time without having to install numerous cameras to capture images of the entire movement route of the vehicle TR in the yard YRD.

[0157] (Technology 10) A method for predicting the position of a moving object (vehicle) performed by at least one processor 11A, First detection data is obtained by a first sensor (camera C2) that detects the moving object (vehicle) from a first area (field of view ARC2) within the area (yard YRD) in which the moving object (vehicle) moves, or second detection data is obtained by a second sensor (sensors S1 to S4) that detects the moving object from a second area (detection area ARS1 to ARS4) using a method different from that of the first sensor (camera C2). Based on the first detection data or the second detection data and field particle data collected in advance, which stores the movement information of the moving object moving within the area (yard YRD) for each position coordinate, a plurality of particles (vehicle particles) are generated to predict the position of the moving object within the area (yard YRD). Based on each of the plurality of particles (vehicle particles), the probability of the moving object being present in the area (yard YRD) is calculated, and based on the calculated probability of the moving object being present, a probability distribution predicting the position of the moving object is generated and output. A method for predicting the position of a moving object. As a result, even when the camera C2 cannot constantly detect the position (absolute position) of the vehicle TR, the processor 21A can predict and track the location of the vehicle TR using a probability distribution based on the position of the vehicle particles. This allows the processor 21A to predict and manage the position of each vehicle TR in the yard YRD in real time without having to install a large number of cameras to capture images of the entire movement route of the vehicle TR in the yard YRD.

[0158] Although various embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It will be clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these will also be understood to fall within the technical scope of this disclosure. Furthermore, the components of the various embodiments described above can be combined arbitrarily without departing from the spirit of the invention. [Industrial applicability]

[0159] This disclosure is useful as a device for predicting the position of a moving object that can predict the position of a moving object outside the camera's field of view with higher accuracy, and as a method for predicting the position of a moving object. [Explanation of Symbols]

[0160] 10 Communications Department 11 processors 11A Processor 12 memory 100,100A Vehicle Management System 111 Learning Model Generation Unit 111A, 111C Field Particle Generation Unit 111B, 111D Learning Department 112 Coordinate Transformation Unit 113 Particle Filter Processing Unit 114 Probability Distribution Generation Unit ARC1, ARC2 field of view ARS1, ARS2, ARS3, ARS4 detection area C1, C2 Camera DP output section DP1, DP2 screen Dt11, Dt11A Field Particle Data Dt12 Vehicle Particle Data P1, P1A Terminal Devices PRT11, PRT21, PRT22 Probability Distribution Group S1, S2, S3, S4 sensors TR Vehicle YRD Yard

Claims

1. An acquisition unit that acquires first detection data detected by a first sensor that detects the moving object from a first area within the area in which the moving object moves, or second detection data detected by a second sensor that detects the moving object from a second area using a method different from that of the first sensor, A particle data generation unit generates a plurality of particles that predict the position of the moving object on the area, based on the first detection data or the second detection data and field particle data that has been collected in advance and stores the movement information of the moving object moving in the area for each position coordinate; The system includes a prediction unit that calculates the probability of the moving object being present in the area based on each of the plurality of particles, and generates and outputs a probability distribution predicting the position of the moving object based on the calculated probability of the moving object being present. A device for predicting the position of a moving object.

2. The movement information of the moving body is the acceleration information of the moving body. A device for predicting the position of a moving object according to claim 1.

3. The particle includes position information predicting the position of the moving body and acceleration information of the moving body at the position, The particle data generation unit refers to the field particle data, obtains acceleration information of the moving body collected at the same position coordinates as the position information of the particle, and updates the acceleration information of the particle with the acceleration information of the moving body stored in the field particle data. The prediction unit calculates the probability of the moving body being present in the area based on each of the plurality of particles whose acceleration information has been updated. The device for predicting the position of a moving object according to claim 2.

4. The particle data generation unit, When the first detection data is obtained, weighting is performed on the particles included in the first area. When the second detection data is obtained, weighting is performed on the particles included in the second area. A device for predicting the position of a moving object according to claim 1.

5. The particle data generation unit samples the plurality of particles of the moving body based on the weighted particles, The prediction unit calculates the probability of the existence of the moving body based on each of the plurality of particles after sampling. A device for predicting the position of a moving object according to claim 4.

6. The particle data generation unit calculates the likelihood of each of the plurality of particles based on the difference between the detection position of the moving body based on the first detection data and the position of the particles, and samples the plurality of particles of the moving body based on the likelihood. The prediction unit calculates the probability of the existence of the moving body based on each of the plurality of particles after sampling. A device for predicting the position of a moving object according to claim 1.

7. The system further includes a learning unit that learns the movement information of the moving object for each position coordinate based on the aforementioned probability distribution. A device for predicting the position of a moving object according to claim 1.

8. The first sensor is a camera, The first detection data includes the identification information of the moving body and the detection position of the moving body. A device for predicting the position of a moving object according to claim 1.

9. The second sensor is a detection device that detects the presence or absence of the moving object. A device for predicting the position of a moving object according to claim 1.

10. A method for predicting the position of a moving object performed by at least one processor, First detection data is obtained by a first sensor that detects the moving object from a first area within the area in which the moving object is moving, or second detection data is obtained by a second sensor that detects the moving object from a second area using a method different from that of the first sensor. Based on the first detection data or the second detection data and field particle data collected in advance, which stores the movement information of the moving object moving in the area for each position coordinate, a plurality of particles are generated to predict the position of the moving object in the area. Based on each of the plurality of particles, the probability of the moving object being present in the area is calculated, and based on the calculated probability of the moving object being present, a probability distribution predicting the position of the moving object is generated and output. A method for predicting the position of a moving object.

Images