Trailer tracking device, trailer tracking system, and trailer tracking method

Abstract

To efficiently support easy visualization of a position of a target trailer for a worker in the yard.SOLUTION: A trailer tracking device comprises: an acquisition unit for acquiring a picked-up image of a camera from at least one camera which images a predetermined movement path of a trailer in a yard; a database for storing identification information of the trailer and information on a position of the trailer in the yard based on the picked-up image of the camera in association with each other; and a visualization processing unit for outputting, on the basis of the database, to an external browser device information on an identified present position of the trailer in accordance with a visualization instruction from the external browser device.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present disclosure relates to a trailer tracking device, a trailer tracking system, and a trailer tracking method. [Background technology]

[0002] Patent Document 1 discloses a moving object tracking device that includes one or more target area cameras that capture an image of a target area, a recognition means that recognizes a unique identification code displayed on a moving object moving in the target area based on an image captured by the target area camera, an image position acquisition means that acquires the position of the moving object in the target area based on the captured image, and a position processing means that determines the identity of the moving object based on the recognized identification code and records the time-dependent change in the position of the moving object in a recording unit. The target area camera includes, for example, an entry camera that captures an image when entering the target area. The moving object tracking device tracks the behavior of a worker wearing a helmet with an identification code attached at the work site, with a work site being exemplified as the target area and a worker being exemplified as the moving object. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-98590 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, it is assumed that the moving object whose change in position over time is to be recorded is a human (a worker moving within a work site). Therefore, it is easy to track the change in the worker's position over time. However, if the moving object whose change in position over time is to be recorded is something other than a human (for example, an object such as a load), tracking it may be difficult.

[0005] For example, assume that the moving object to be recorded for change in position over time is a trailer in a vast business office (hereinafter sometimes referred to as a "yard") of a transportation company or the like where tractors (so-called transportation trucks) towing trailers (an example of a towed object) loaded with many packages are accumulated. In this case, when the transportation truck (in other words, the tractor) enters (checks in) the yard by the driver who drives the tractor, the trailer is parked in a designated parking space in the yard. After that, a yard worker (hereinafter sometimes referred to as a "yard jockey") who is a person different from the tractor driver may get on the same tractor and tow another designated tractor, and then leave (check out) the yard. When the driver and the yard jockey are different people, the parking space where the driver thinks the trailer is parked may not match the parking space where the yard jockey is heading. In other words, there is a problem that the yard jockey cannot find the desired trailer even if he gets on the tractor and heads to the designated parking space. Especially when the yard has a vast area, it is often difficult for the yard jockey to find the desired trailer by visual inspection. Therefore, there is a need for a technology that can visualize the location of each trailer parked in such a yard at any time.

[0006] The present disclosure has been devised in consideration of the above-mentioned conventional circumstances, and aims to provide a trailer tracking device, a trailer tracking system, and a trailer tracking method that efficiently assist workers in a yard in easily visualizing the location of a target trailer. [Means for solving the problem]

[0007] The present disclosure provides a method for controlling a trailer-reading system, comprising: a memory and a processor; and the processor uses the memory to acquire images captured by at least one camera that captures a predetermined moving path of a trailer within a yard, the camera being a first camera that is provided at an entrance to the yard and that reads identification information of the trailer. and , at least one second camera that is provided in the yard and captures an image of the travel path; , and 1 monththe camera periodically captures an image having a front portion of the trailer as a subject; The processor uses the memory to: The identification information of the trailer and information on the position of the trailer in the yard based on the image captured by the camera are stored in a database in association with each other, information identifying a first trailer is acquired, and based on the information identifying the first trailer, position information of the first trailer corresponding to the first trailer is read from the database and output. a position of the trailer within the yard is estimated based on an image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; if it is determined that a first position indicating the position of the trailer within the yard based on the image captured by the second camera is within a predicted range of a position of the trailer that may move within the yard, the flow line to the first position is added to a previously extracted flow line that has been extracted most recently; if it is determined that the first position indicating the position of the trailer within the yard based on the image captured by the second camera is outside the predicted range, the flow line to the first position is added as a new flow line that has not been extracted most recently. The present invention provides a trailer tracking device.

[0008] The present disclosure also relates to a trailer tracking method executed by a trailer tracking device communicably connected to at least one camera that captures an image of a predetermined movement path of the trailer within a yard, the method comprising: acquiring an image captured by at least one camera that captures an image of the predetermined movement path of the trailer within the yard, the camera being a first camera that is provided at an entrance to the yard and that reads identification information of the trailer; and , at least one second camera that is provided in the yard and captures an image of the travel path; , and 1 month The camera periodically captures images of the front portion of the trailer, associates identification information of the trailer with information on the position of the trailer in the yard based on the captured images of the camera, stores the information in a database, obtains information that identifies a first trailer, and reads out the position information of the first trailer corresponding to the first trailer from the database based on the information that identifies the first trailer, and outputs the information. a position of the trailer within the yard is estimated based on an image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; if it is determined that a first position indicating the position of the trailer within the yard based on the image captured by the second camera is within a predicted range of a position of the trailer that may move within the yard, the flow line to the first position is added to a previously extracted flow line that has been extracted most recently; if it is determined that the first position indicating the position of the trailer within the yard based on the image captured by the second camera is outside the predicted range, the flow line to the first position is added as a new flow line that has not been extracted most recently. A trailer tracking method is provided.

[0009] The present disclosure also provides a method for detecting a trailer having a trailer-reading device, the method comprising: acquiring, by using the memory, an image captured by at least one camera that captures an image of a predetermined moving path of a trailer within a yard; and a first camera that is provided at an entrance to the yard and that reads identification information of the trailer. and , at least one second camera that is provided in the yard and captures an image of the travel path; , and 1 month the camera periodically captures an image having a front portion of the trailer as a subject; The processor uses the memory to:The identification information of the trailer and information on the position of the trailer in the yard based on the image captured by the camera are stored in a database in association with each other, information identifying a first trailer is acquired, and based on the information identifying the first trailer, position information of the first trailer corresponding to the first trailer is read from the database and output. a position of the trailer within the yard is estimated based on an image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; if it is determined that a first position indicating the position of the trailer within the yard based on the image captured by the second camera is within a predicted range of a position of the trailer that may move within the yard, the flow line to the first position is added to a previously extracted flow line that has been extracted most recently; if it is determined that the first position indicating the position of the trailer within the yard based on the image captured by the second camera is outside the predicted range, the flow line to the first position is added as a new flow line that has not been extracted most recently. The present invention provides a trailer tracking system. Effect of the Invention

[0010] According to the present disclosure, it is possible to efficiently assist workers in a yard in easily visualizing the location of a desired trailer. [Brief description of the drawings]

[0011] [Figure 1] Bird's-eye view of the yard's atmosphere [Diagram 2] Aerial view showing an example of camera placement within the yard [Diagram 3] FIG. 13 is a diagram showing an example of an image captured by a license plate reading camera. [Figure 4] A diagram showing an example of an image captured by a trailer ID reading camera. [Diagram 5] A block diagram showing a schematic example of the system configuration of a trailer tracking system. [Figure 6A] A diagram showing an example of an ID database configuration [Figure 6B] A diagram showing an example of the configuration of a flow line database [Figure 6C] A diagram showing an example of the pairing database configuration [Figure 6D] A diagram showing an example of the expected value database configuration [Figure 7] Bird's-eye view showing an example use case of trailer movement trajectory (traffic line) within a yard [Figure 8] A diagram showing specific examples of each database for each situation corresponding to the use cases in Figure 7. [Figure 9] A diagram showing specific examples of each database for each situation corresponding to the use cases in Figure 7. [Figure 10]A diagram showing specific examples of each database for each situation corresponding to the use cases in Figure 7. [Figure 11] A diagram showing specific examples of each database for each situation when an abnormal situation occurs during the situation corresponding to the use case in Figure 7. [Figure 12] A bird's-eye view of an example of the first traffic flow of a trailer checked into the yard. [Figure 13] FIG. 13 shows an example of a mapping display screen displayed on a display device, visualizing that the trailer in FIG. 12 has been parked in the correct position as instructed. [Figure 14] FIG. 13 is a diagram showing an example of a mapping display screen displayed on a display device, which visualizes that the trailer in FIG. 12 has been parked in an incorrect position different from the instruction. [Figure 15] A bird's-eye view of an example of the second traffic flow of trailers checked into the yard. [Figure 16] FIG. 16 shows an example of a mapping display screen displayed on a display device, visualizing that the trailer in FIG. 15 has been parked in the correct position as instructed. [Figure 17] FIG. 16 shows an example of a mapping display screen displayed on a display device, visualizing that the trailer in FIG. 15 has been parked in an incorrect position different from the one instructed. [Figure 18] A sequence diagram showing the operation procedure from trailer check-in to trailer information pairing processing of the trailer tracking system according to the present embodiment in chronological order. [Figure 19] A sequence diagram showing the operation procedure for visualizing the trailer position and traffic line of the trailer tracking system according to the present embodiment in a chronological order. [Figure 20A] FIG. 11 is a block diagram showing a detailed functional configuration example of a trailer flow line extraction unit according to a modified example. [Figure 20B] FIG. 13 is a diagram showing an example of the configuration of a flow line database according to a modified example; [Figure 20C] FIG. 13 is a diagram showing an example of the configuration of a pairing database according to a modified example; [Figure 21] FIG. 13 is a first visualization example of the positions of trailers checked into a yard, which is displayed on a display device according to a modified example. [Figure 22]FIG. 13 is a diagram showing a second visualization example of the positions of trailers checked into a yard, which is displayed on the display device according to the modified example. [Figure 23] FIG. 13 is a third visualization example of the positions of trailers checked into a yard, which is displayed on a display device according to a modified example. [Figure 24] FIG. 13 is a fourth visualization example of the positions of trailers checked into a yard, which is displayed on the display device according to the modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, with reference to the drawings as appropriate, an embodiment specifically disclosing a trailer tracking system and a trailer tracking method according to the present disclosure will be described in detail. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0013] 1. Definitions First, the terms used in the following embodiments will be defined. (1) Trailer: A container-like object that has a rectangular parallelepiped case carrying one or more cargo items and is towed by a tractor. The trailer has identification information ("trailer ID (IDentification)") to identify the trailer. (2) Tractor: An object that is a vehicle part of a transport truck that can be docked (coupled) with a trailer or detached from the trailer. A tractor has identification information ("Tractor ID") to identify the tractor. (3) Yard: A large office or site of a transportation company or the like where tractors pulling trailers (so-called transport trucks) are gathered. (4) Yard jockey: A worker who performs tasks within the yard, such as driving a tractor in the yard to dock a desired trailer in a designated parking space with the tractor and moving it to the dock. The yard jockey may also perform tasks such as taking out or carrying in cargo from a trailer that has been moved to the dock. In this embodiment, the yard jockey is treated as being different from the driver of a transport truck that checks in to the yard or the driver of a transport truck that checks out to the yard.

[0014] (Embodiment 1) 2. Schematic description of the yard First, with reference to Figures 1 to 4, a yard YRD in which a trailer tracking system 100 (see Figure 5) according to this embodiment is installed will be generally described. Figure 1 is a bird's-eye view of the atmosphere of the yard YRD. Figure 2 is an overhead view showing an example of the arrangement of cameras installed in the yard YRD. Figure 3 is a diagram showing an example of an image captured by the license plate reading camera LC1. Figure 4 is a diagram showing an example of an image captured by the trailer ID reading camera IC1.

[0015] As shown in Figures 1 and 2, the yard YRD is a vast site (e.g., a rectangular site when viewed from above the yard YRD) owned by a business entity such as a transportation company. Specifically, the yard YRD at least includes a check-in gate CIG, a check-out gate COG, a plurality of parking spaces (e.g., parking spaces PKN, PKS, PKE, PKW), and a dock station (so-called dock DCK) within the site. The check-in gate CIG is a location through which a transportation truck TK1 (e.g., a tractor TRC1 towing a trailer TRA1) passes when entering the yard YRD. The check-out gate COG is a location through which a transportation truck TK2 (e.g., a tractor TRC2 towing a trailer TRA2) passes when exiting the yard YRD.

[0016] Each of the parking spaces PKN, PKS, PKE, and PKW is a space for parking one or more trailers, and is arranged at a distance from one another. In the example of FIG. 1, the parking space PKN parks a total of five trailers, the parking space PKS parks a total of five trailers, and the parking space PKE parks a total of three trailers. The number of trailers that can be parked in each parking space may be predetermined, or may be increased as appropriate depending on actual operation. Each parking space is provided with white lines according to the number of trailers that can be parked, to distinguish where one trailer can be parked.

[0017] A dock DCK is a place where a yard jockey drives a tractor to tow a desired trailer in a parking space and stops the trailer, or where a yard jockey drives a tractor to move and stop the tractor alone without towing a trailer. At a dock DCK, the yard jockey, for example, removes cargo from a trailer and carries it into the dock DCK, or carries new cargo from the dock DCK into the trailer. Also, at a dock DCK, the yard jockey may dock a trailer parked at the dock DCK to the tractor.

[0018] As shown in FIG. 2, a license plate reading camera LC1 and a trailer ID reading camera IC1 (an example of a first camera) are disposed near a check-in gate CIG (an example of an entrance) of the yard YRD. Each of the license plate reading camera LC1 and the trailer ID reading camera IC1 is connected to a trailer tracking device 20 (see FIG. 5) so as to be able to perform data communication. The license plate reading camera LC1 captures an image of the license plate LPct1 (see FIG. 3) of the transport truck TK1 that is about to enter the yard YRD, and transmits data of the captured image IMG1 (see FIG. 3) to the trailer tracking device 20 (see FIG. 5). Meanwhile, the trailer ID reading camera IC1 captures an image of the identification information IDct1 (see FIG. 4) of the trailer TRA1 of the transport truck TK1 that is about to enter the yard YRD, and transmits the captured image IMG2 (see FIG. 4) to the trailer tracking device 20 (see FIG. 5).

[0019] A license plate reading camera LC2 and a trailer ID reading camera IC2 (an example of a first camera) are disposed near the check-out gate COG (an example of an exit) of the yard YRD. Each of the license plate reading camera LC2 and the trailer ID reading camera IC2 is connected to a trailer tracking device 20 (see FIG. 5) so as to be able to perform data communication. The license plate reading camera LC2 captures an image of the license plate of a transport truck TK2 about to leave the yard YRD, and transmits the captured image data to the trailer tracking device 20 (see FIG. 5). Meanwhile, the trailer ID reading camera IC2 captures an image of the identification information of the trailer of the transport truck TK2 about to leave the yard YRD, and transmits the captured image data to the trailer tracking device 20 (see FIG. 5).

[0020] At least one trailer tracking camera C1, C2, C3, C4, C5, C6, C7, C8 (an example of a second camera) is disposed on the zenith of the roof or the like of the building of the dock DCK, capturing an image of a predetermined moving path of the trailer in the yard YRD. Each of the trailer tracking cameras C1, C2, C3, C4, C5, C6, C7, C8 is connected to the trailer tracking device 20 (see FIG. 5) so as to be capable of data communication. Each of the trailer tracking cameras C1, C2, C3, C4, C5, C6, C7, C8 captures an image of a subject (e.g., a trailer) within the field of view of the vehicle itself, and transmits the captured image data to the trailer tracking device 20 (see FIG. 5).

[0021] As shown in FIG. 3, the license plate reading camera LC1 periodically captures an image of a front part of a tractor TRC1 of a transport truck TK1 approaching the check-in gate CIG. The captured image IMG1 obtained by this capture shows a license plate LPct1, which is identification information of the transport truck TK1. The captured image IMG1 is sent from the license plate reading camera LC1 to the trailer tracking device 20. The trailer tracking device 20 detects the license plate LPct1 by analyzing the captured image IMG1 from the license plate reading camera LC1, and extracts information on the license plate LPct1. The method of detecting the license plate LPct1 may be a known technique, and a detailed description will be omitted here, but may be performed by a learning model formed by machine learning using a large amount of learning data and AI (Artificial Intelligence), for example.

[0022] As shown in FIG. 4, the trailer ID reading camera IC1 periodically captures an image of the front part of the trailer TRA1 of the transport truck TK1 approaching the check-in gate CIG. The captured image IMG2 obtained by this capture shows the identification information IDct1 of the trailer TRA1 towed by the tractor TRC1 of the transport truck TK1. The captured image IMG2 is sent from the trailer ID reading camera IC1 to the trailer tracking device 20. The trailer tracking device 20 detects the identification information IDct1 of the trailer TRA1 by analyzing the captured image IMG2 from the trailer ID reading camera IC1, and extracts the identification information IDct1. Hereinafter, the trailer identification information may be abbreviated as "trailer ID". Note that the method of detecting this identification information may be a publicly known technology, and a detailed description will be omitted here, but it may be executed by a learning model formed by machine learning using a large amount of learning data and AI, for example.

[0023] 3. System configuration of trailer tracking system Next, a system configuration example of the trailer tracking system 100 according to this embodiment will be described in detail with reference to Fig. 5. Fig. 5 is a block diagram that shows a schematic system configuration example of the trailer tracking system 100. The trailer tracking system 100 according to this embodiment includes a camera 10, a trailer tracking device 20, and a display device 30. The camera 10 and the trailer tracking device 20, and the trailer tracking device 20 and the display device 30 are connected to each other via a network so as to be able to perform data communication.

[0024] The network referred to here is either a wired network, a wireless network, or a combination of a wired network and a wireless network. The wired network corresponds to at least one of a wired LAN (Local Area Network), a wired WAN (Wide Area Network), and a power line communication (PLC), and may be other network configurations capable of wired communication. On the other hand, the wireless network corresponds to at least one of a wireless LAN such as Wi-Fi (registered trademark), a wireless WAN, and a mobile communication network such as 4G or 5G, and may be other network configurations capable of wireless communication.

[0025] The camera 10 captures an image of a predetermined movement path of the trailer within the yard YRD. The camera 10 includes, for example, license plate reading cameras LC1 and LC2, trailer ID reading cameras IC1 and IC2, and trailer tracking cameras C1 to C8. Note that cameras other than these may be disposed as the camera 10. For example, the camera 10 may include a drone (not shown) capable of taking an aerial photograph of the inside of the yard YRD from the sky, or a device capable of taking a satellite photograph from the sky.

[0026] The trailer tracking device 20 detects (tracks) a temporal change in the position of a trailer that is scheduled to move along a predetermined moving path in the yard YRD using the captured image data from the camera 10, and stores the detection result data. The trailer tracking device 20 may be configured, for example, as a personal computer or a high-performance server computer. The trailer tracking device 20 includes a memory 21, a communication interface 22, a trailer ID extraction unit 23, an ID database DB1, a trailer flow line extraction unit 24, a flow line database DB2, a pairing processing unit 25, a pairing database DB3, a visualization application processing unit 26, an expected value database DB4, and a communication interface 27. In FIG. 5, the interface is abbreviated as "I / F".

[0027] The memory 21 includes a ROM (Read Only Memory) and a RAM (Random Access Memory). The ROM stores programs that define the processes (operations) of the trailer ID extraction unit 23, the trailer flow line extraction unit 24, the pairing processing unit 25, and the visualization application processing unit 26, and data that is referenced when the programs are executed. The RAM is a work memory that is used when executing the processes (operations) of the trailer ID extraction unit 23, the trailer flow line extraction unit 24, the pairing processing unit 25, and the visualization application processing unit 26, and temporarily stores data or information generated or acquired in each process.

[0028] The communication interface 22 is an example of an acquisition unit, and is configured using a communication circuit that communicates data (e.g., images, videos) with the various cameras 10. When the communication interface 22 receives captured image data from the various cameras 10, it sends the data to the trailer ID extraction unit 23 or to the trailer flow line extraction unit 24. The communication interface 22 may distribute the destination of the captured image data from the various cameras 10 according to the camera 10 that sent the data. For example, the communication interface 22 may send captured image data from each of the trailer ID reading cameras IC1, IC2 and the license plate reading cameras LC1, LC2 to the trailer ID extraction unit 23, and may send captured image data from each of the trailer tracking cameras C1 to C8 to the trailer flow line extraction unit 24.

[0029] Each of the trailer ID extraction unit 23, the trailer traffic line extraction unit 24, the pairing processing unit 25, and the visualization application processing unit 26 is configured using a processor such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array).

[0030] The trailer ID extraction unit 23 periodically acquires data from the communication interface 22 (i.e., captured image data from each of the trailer ID reading cameras IC1 and IC2 and the license plate reading cameras LC1 and LC2). The trailer ID extraction unit 23 periodically extracts a trailer ID, which is identification information of the trailer shown in the captured image, by analyzing the captured image data from the trailer ID reading camera (for example, the trailer ID reading camera IC1). The trailer ID extraction unit 23 stores (registers) the trailer ID extraction result data obtained by the analysis in the ID database DB1 and sends it to the pairing processing unit 25. As described above, the method of extracting the trailer ID by analyzing the captured image data may be executed by a learning model formed by machine learning using a large amount of learning data and AI, or may be executed by using a publicly known technique such as existing pattern matching.

[0031] The ID database DB1 is a database that stores (registers) a timestamp (TM) indicating the time when the trailer ID was extracted and the extracted trailer ID (TID) in association with each other (see FIG. 6A). FIG. 6A is a diagram showing an example of the configuration of the ID database DB1. As shown in FIG. 6A, the ID database DB1 stores the timestamp (TM) and the trailer ID (TID) in association with each other. Specific examples of the ID database DB1 will be described later with reference to use cases shown in FIG. 7 to FIG. 11.

[0032] The trailer flow line extraction unit 24 periodically acquires data from the communication interface 22 (i.e., data of captured images from each of the trailer tracking cameras C1 to C8). The trailer flow line extraction unit 24 periodically extracts position information indicating the position of the trailer shown in the captured image by performing an analysis of the captured image data from the trailer tracking camera (e.g., the trailer tracking camera C1). The trailer flow line extraction unit 24 determines whether the position indicated by the extracted position information constitutes the same or different flow line between the position indicated by the most recently (previously) extracted position information and the position indicated by the position information based on a predetermined determination criterion, and assigns a flow line ID based on the determination result. When the trailer flow line extraction unit 24 determines that the extracted position information is the same as the existing flow line (i.e., the same as the existing flow line), the trailer flow line extraction unit 24 associates the extracted position information with the flow line ID (FID), which is identification information of the same flow line, and stores (registers) the FID in the flow line database DB2 and sends it to the pairing processing unit 25. On the other hand, if the trailer flow line extraction unit 24 determines that it is a different flow line (i.e. a new flow line), it associates the flow line ID (FID), which is the identification information of the new flow line, with the extracted location information, stores (registers) it in the flow line database DB2, and sends it to the pairing processing unit 25.

[0033] Here, the predetermined criterion is whether or not the position indicated by the most recently extracted position information is within a predictable range from the position indicated by the most recently extracted position information. The predictable range is, for example, when assuming that the trailer is moving in a uniform linear motion, a range that radially includes a position to which the trailer can be estimated to move when it is next extracted (i.e., the most recent) from the most recently (i.e., previous) extracted position information based on the speed of the trailer moving in a uniform linear motion. The predictable range may also be derived by the trailer flow line extraction unit 24 through calculation processing using a predetermined function (e.g., a Kalman filter) with the most recently extracted position information as input.

[0034] Here, the trailer position information is information indicating a relative position from the origin (0.0) when a two-dimensional xy coordinate system is established within the site of the yard YRD shown in Fig. 7 described later. The trailer traffic flow extraction unit 24 stores (registers) data of the extraction result of the trailer position information obtained by the analysis in the ID database DB1. As described above, the method of extracting the trailer position information by analyzing the captured image data may be performed by a learning model formed by machine learning using a large amount of learning data and AI, or may be performed by using a publicly known technique such as existing pattern matching.

[0035] The flow line database DB2 is a database that stores (registers) a time stamp (TM) indicating the time when the trailer position information is extracted by the trailer flow line extraction unit 24, a trailer position (POS) indicated by the extracted position information (e.g., two-dimensional coordinates), and a flow line ID (FID) indicating whether the trailer position constitutes a new flow line or an existing flow line, in association with each other (see FIG. 6B). FIG. 6B is a diagram showing an example of the configuration of the flow line database DB2. As shown in FIG. 6B, the flow line database DB2 stores the time stamp (TM), the flow line ID (FID), and the trailer position (POS) in association with each other. A specific example of the flow line database DB2 will be described later with reference to use cases shown in FIG. 7 to FIG. 11.

[0036] The pairing processing unit 25 acquires data of the extraction result of the trailer ID from the trailer ID extraction unit 23 and data of the extraction result of the trailer position information from the trailer flow line extraction unit 24. Furthermore, the pairing processing unit 25 determines whether or not predetermined conditions (e.g., [Condition 1], [Condition 2], and [Condition 3], see FIG. 18) are satisfied based on instructions from the trailer flow line extraction unit 24. When the pairing processing unit 25 determines that any one of [Condition 1], [Condition 2], and [Condition 3] is satisfied, it pairs (associates) the data of the extraction result of the trailer ID with the data of the extraction result of the trailer position information and stores (registers) them in the pairing database DB3. In other words, the pairing processing unit 25 integrates different information (specifically, trailer ID (TID), trailer position (POS), and traffic flow ID (FID)) extracted using image data captured by different cameras 10 (specifically, trailer ID reading camera IC1 and at least one of trailer tracking cameras C1 to C8) into information about a single trailer based on the time stamp (TM) information common to the different information.

[0037] The pairing database DB3 is a database that associates and stores (registers) a time stamp (TM) indicating the time when the trailer ID was extracted and the time when the trailer position information was extracted, the extracted trailer ID (TID), the trailer position (POS) indicated by the extracted position information, and a flow line ID (FID) indicating whether the trailer position constitutes a new flow line or an existing flow line (see FIG. 6C). FIG. 6C is a diagram showing a configuration example of the pairing database DB3. As shown in FIG. 6C, the pairing database DB3 associates and stores the time stamp (TM), the trailer ID (TID), the flow line ID (FID), and the trailer position (POS). A specific example of the pairing database DB3 will be described later with reference to use cases shown in FIG. 7 to FIG. 11.

[0038] The visualization application processing unit 26 acquires a visualization request (an example of a visualization instruction) for visualizing the current trailer position (POS) from the display device 30 via the communication interface 27. For example, when the yard jockey heads to the position of a target trailer specified by an operator in the yard YRD but cannot find the target trailer, the visualization request is generated by the yard jockey browser device 32 through the operation of the yard jockey, or is generated by the operator browser device 31 through the operation of the operator who has received a notification from the yard jockey. The visualization request includes, for example, a trailer ID for identifying the target trailer. When the visualization application processing unit 26 acquires a visualization request from various display devices 30, the visualization application processing unit 26 inquires of the pairing processing unit 25 to acquire information on the current position of the trailer specified by the visualization request, or directly accesses and refers to the pairing database DB to acquire the information. Based on the acquisition result of the current position of the trailer in response to the visualization request, the visualization application processing unit 26 generates at least a mapping display screen (see, for example, FIG. 13 or FIG. 14) showing the current position of the trailer, and outputs (transmits) it to the display device 30. This mapping display screen (see, for example, FIG. 13 or FIG. 14) is displayed on the display device 30 so that the operator or the yard jockey can visually confirm it. Note that, detailed operation procedures of the visualization application processing unit 26 will be described later with reference to FIG.

[0039] The expected value database DB4 is a database that specifies the position on the movement route in the yard YRD for each trailer that should be essentially observed when the operator, who is responsible for the operation and management of the dock DCK, manages the movement of the trailer, such as the arrival and departure from the appropriate dock DCK (see FIG. 6D). FIG. 6D is a diagram showing a configuration example of the expected value database DB4. As shown in FIG. 6D, the pairing database DB3 stores a time stamp (TM), a trailer ID (TID), a trailer parking position (PPOS), and a trailer dock position (DPOS) in association with each other. The time stamp (TM), the trailer ID (TID), the trailer parking position (PPOS), and the trailer dock position (DPOS) may be stored (registered) by, for example, inputting the time stamp (TM), the trailer ID (TID), the trailer parking position (PPOS), and the trailer dock position (DPOS) through the operation of the display device 30 (for example, the operator browser device 31) that can refer to the pairing database DB3 via the visualization application processing unit 26. A specific example of the expected value database DB4 will be described later with reference to the use cases shown in FIG. 7 to FIG. 11. The trailer parking position (PPOS) is the position information (e.g., two-dimensional coordinates) of the parking space where the checked-in trailer should be parked. The trailer dock position (DPOS) is the position information (e.g., two-dimensional coordinates) of the dock DCK to which the trailer should next move from the parking space.

[0040] The communication interface 27 is an example of an acquisition unit, and is configured using a communication circuit that communicates data (e.g., visualization request, matching display screen) with various display devices 30. When the communication interface 27 receives visualization request data from various display devices 30, it sends the data to the visualization application processing unit 26. When the communication interface 27 receives mapping display screen data from the visualization application processing unit 26, it sends the data to various display devices 30. Note that the communication interface 27 may distribute the corresponding mapping display screen data according to the sender of the visualization request data from various display devices 30, or may send the mapping display screen data corresponding to all display devices 30 regardless of the sender of the visualization request data.

[0041] The display device 30 generates data of a visualization request for the position of a target trailer input by a user operation, sends it to the trailer tracking device 20, and displays a mapping display screen (see, for example, FIG. 13 or FIG. 14 ) showing the position of the target trailer generated by the trailer tracking device 20. The display device 30 includes an operator browser device 31 and a yard jockey browser device 32.

[0042] The operator browser device 31 is, for example, a notebook PC or a tablet terminal, and is used by an administrator (operator) who is in charge of the yard YRD that manages the trailer tracking system 100. Note that the operator browser device 31 may be a computer device other than a notebook PC or a tablet terminal.

[0043] The yard jockey browser device 32 is used by a yard jockey who is a worker working in the yard YRD, and is, for example, a notebook PC or a tablet terminal. Note that the yard jockey browser device 32 may be a computer device other than a notebook PC or a tablet terminal.

[0044] 4. Use Cases of Trailer Tracking System Next, with reference to Fig. 7 to Fig. 11, a use case example when the trailer tracking system 100 according to the present embodiment is actually operated will be described in detail. Fig. 7 is a bird's-eye view showing a use case example of a movement trajectory (traffic line) of a trailer in a yard. Figs. 8, 9, and 10 are diagrams showing specific examples of each database for each situation corresponding to the use case of Fig. 7. Fig. 11 is a diagram showing specific examples of each database for each situation when an abnormal situation occurs in the situation corresponding to the use case of Fig. 7. In the description of Figs. 8 to 10, the same configurations and contents are given the same reference numerals, and the description of the overlapping contents is simplified or omitted, and different contents will be described.

[0045] In explaining a use case of the trailer tracking system 100 according to this embodiment, two-dimensional coordinate axes (specifically, the origin, the x-axis, and the y-axis) for defining the position of a trailer within the yard YRD are shown in FIG.

[0046] For the use case shown in Figure 7, assume that the following events occur in a chronological order:

[0047] (Event 1) At time 0:00, a trailer with trailer ID "ID0001" passes through check-in gate CIG (see FIG. 1) located at coordinates (16, 50) and moves toward a parking space (e.g., parking space PKS) designated by the operator. Note that the movement of the trailer is performed in conjunction with the movement of the tractor (see FIG. 1) towing the trailer, and so on.

[0048] (Event 2) At time 0:02, a trailer having trailer ID "ID0001" is located at coordinates (15, 37) and is moving toward a parking space specified by the operator (e.g., parking space PKS).

[0049] (Event 3) At time 0:05, a trailer having trailer ID "ID0001" is parked in a parking space provided at coordinates (5, 31) (eg, any one of the parking spaces within parking space PKS).

[0050] (Event 4) At time 1:00, a trailer having a trailer ID "ID0001" is parked (docked in) in the parking space of dock DCK located at coordinates (30, 30).

[0051] (Event 5) At time 1:30, a trailer having trailer ID "ID0001" is parked in a parking space provided at coordinates (37, 39) (eg, any one of the parking spaces within parking space PKE).

[0052] (Event 6) At time 2:00, a trailer having a trailer ID "ID0001" passes through the checkout gate COG (see FIG. 1) located at the position of coordinates (13, 50) and exits the yard YRD.

[0053] The temporal change in the position of the trailer having the same trailer ID "ID0001" from the series of events (event 1) to (event 6) is continuously extracted. Therefore, the trailer tracking device 20 assigns and manages the flow line ID "A" as the flow line ID (FID) that identifies the flow line of the trailer having the trailer ID "ID0001" from (event 1) to (event 6).

[0054] 8, "1. Before check-in" shows the state immediately before a trailer having trailer ID "ID0001" passes through check-in gate CIG of yard YRD at the timing "before check-in" before (event 1). In this case, no data has been written by trailer tracking device 20 to ID database DB1, flow line database DB2, pairing database DB3, and expected value database DB4.

[0055] In "2. Check-in" in Fig. 8, a trailer having a trailer ID "ID0001" passes through the check-in gate CIG of the yard YRD. At this time, the trailer tracking device 20 detects the trailer ID "ID0001" based on an image captured by the trailer ID reading camera IC1 installed near the check-in gate CIG, and writes the trailer ID "ID0001" and a timestamp "0:00" indicating the time when the trailer ID "ID0001" was detected into the ID database DB1. The trailer tracking device 20 also detects the position of the trailer (e.g., coordinates (16, 50)) based on an image captured by the trailer tracking camera (see Fig. 5), and writes the timestamp "0:00" indicating the time when the trailer position was detected, the trailer position (e.g., coordinates (16, 50)), and a flow line ID "A" indicating that the flow line of the trailer is a new flow line into the flow line database DB2 in association with each other. Furthermore, the trailer tracking device 20 writes the timestamp "0:00", trailer ID "ID0001", flow line ID "A", and trailer position (e.g., coordinates (16, 50)) corresponding to the same trailer into the pairing database DB3 as a pairing process result using data having a timestamp "0:00" common to the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 receives and stores the timestamp "0:00" indicating the time when the trailer was detected for the trailer with trailer ID "ID0001" by the operation of the operator using the operator browser device 31, the trailer parking position (e.g., coordinates (5, 31)) indicating the position where the trailer should be parked next, and the trailer dock position (e.g., coordinates (30, 30)) indicating the position of the dock DCK where the trailer should be moved after parking.

[0056] In "3. Moving to parking lot" in FIG. 8, a trailer having a trailer ID "ID0001" is shown moving to a parking lot in the yard YRD specified by the operator (for example, the trailer parking position corresponding to the trailer ID "ID0001" in the expected value database DB4). At this time, the contents of the ID database DB1 are the same as those at "2. Check-in" because a new trailer ID has not been detected. The trailer tracking device 20 detects the trailer's position (for example, coordinates (15, 37)) based on the captured image taken by the trailer tracking camera (see FIG. 5), associates the timestamp "0:02" indicating the time when the trailer's position was detected with the trailer's position (for example, coordinates (15, 37)) and the flow line ID "A" indicating that the trailer's flow line is an existing flow line, and writes them to the flow line database DB2 at the time of "2. Check-in". Furthermore, the trailer tracking device 20 additionally writes the timestamp "0:02", trailer ID "ID0001", flow line ID "A", and trailer position (e.g., coordinates (15, 37)) corresponding to the same trailer to the pairing database DB3 at the time of "2. Check-in" as a pairing process result using data having the common timestamp "0:02" in both the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 contains the same contents as those input at the time of "2. Check-in".

[0057] In "4. 1st Parking" in FIG. 9, a trailer having a trailer ID "ID0001" is parked at a position of a dock DCK in a yard YRD specified by an operator (for example, a trailer parking position corresponding to the trailer ID "ID0001" in the expected value database DB4). At this time, the contents of the ID database DB1 are the same as those at "2. Check-in" because a new trailer ID has not been detected. The trailer tracking device 20 detects the trailer's position (for example, coordinates (5, 31)) based on an image captured by the trailer tracking camera (see FIG. 5), associates the timestamp "0:05" indicating the time when the trailer's position was detected with the trailer's position (for example, coordinates (5, 31)) and the flow line ID "A" indicating that the trailer's flow line is an existing flow line, and writes it to the flow line database DB2 for "3. Moving to Parking Lot". Furthermore, the trailer tracking device 20 additionally writes the timestamp "0:05", trailer ID "ID0001", flow line ID "A", and trailer position (e.g. coordinates (5, 31)) corresponding to the same trailer to the pairing database DB3 at "3. Moving to parking lot" as a pairing process result using data having a common timestamp "0:05" in both the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 contains the same contents as those input at "2. Check-in".

[0058] In "5. Docking in" in FIG. 9, a trailer with trailer ID "ID0001" stops at a dock DCK in the yard YRD specified by the operator (for example, the trailer dock position corresponding to trailer ID "ID0001" in the expected value database DB4). At this time, the contents of the ID database DB1 are the same as those at "2. Checking in" because a new trailer ID has not been detected. The trailer tracking device 20 detects the trailer's position (for example, coordinates (30, 30)) based on the captured image captured by the trailer tracking camera (see FIG. 5), associates the timestamp "1:00" indicating the time when the trailer's position was detected with the trailer's position (for example, coordinates (30, 30)) and the flow line ID "A" indicating that the trailer's flow line is an existing flow line, and writes them additionally to the flow line database DB2 at "4. 1st parking". Furthermore, the trailer tracking device 20 additionally writes the timestamp "1:00", trailer ID "ID0001", flow line ID "A", and trailer position (e.g., coordinates (30, 30)) corresponding to the same trailer to the pairing database DB3 at "4. 1st parking" as a pairing process result using data having a common timestamp "1:00" in both the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 contains the same contents as those input at "2. Check-in".

[0059] In "6. 2nd Parking" in FIG. 9, a trailer with trailer ID "ID0001" is parked in a parking lot (e.g., one of the parking spaces in parking space PKE) in yard YRD that is suddenly designated by the operator. At this time, the contents of ID database DB1 are the same as those at "2. Check-in" because a new trailer ID has not been detected. Trailer tracking device 20 detects the position of the trailer (e.g., coordinates (37, 39)) based on the captured image captured by the trailer tracking camera (see FIG. 5), associates the timestamp "1:30" indicating the time when the trailer position was detected with the trailer position (e.g., coordinates (37, 39)) and the flow line ID "A" indicating that the flow line of the trailer is an existing flow line, and writes it to flow line database DB2 at "5. Dock-in". Furthermore, the trailer tracking device 20 additionally writes the timestamp "1:30", trailer ID "ID0001", flow line ID "A", and trailer position (e.g. coordinates (37, 39)) corresponding to the same trailer to the pairing database DB3 at the time of "5. Docking in" as a pairing process result using data having a common timestamp "1:30" in both the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 contains the same contents as those input at the time of "2. Check-in".

[0060] In "7. Check-out" in Fig. 10, a trailer having a trailer ID "ID0001" passes through the check-out gate COG of the yard YRD. At this time, the contents of the ID database DB1 are the same as those at "2. Check-in" because a new trailer ID has not been detected. The trailer tracking device 20 detects the position of the trailer (e.g., coordinates (13, 50)) based on the image captured by the trailer tracking camera (see Fig. 5), associates the timestamp "2:00" indicating the time when the trailer position was detected with the trailer position (e.g., coordinates (13, 50)) and the flow line ID "A" indicating that the flow line of the trailer is an existing flow line, and writes it additionally to the flow line database DB2 at "6. 2nd parking". Furthermore, the trailer tracking device 20 additionally writes the timestamp "2:00", trailer ID "ID0001", flow line ID "A", and trailer position (e.g., coordinates (13, 50)) corresponding to the same trailer to the pairing database DB3 at "6. 2nd parking" as a pairing process result using data having a common timestamp "2:00" in each of the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 has the same contents as those input at "2. Check-in".

[0061] 10, "8. Archive and delete data of the ID", the trailer tracking device 20 deletes various data corresponding to the trailer having the trailer ID "ID0001" checked out in "7. Check out" from each of the ID database DB1, the flow line database DB2, the pairing database DB3, and the expected value database DB4. In this way, the trailer tracking device 20 can suppress an increase in the storage capacity of various databases by deleting various data corresponding to trailers that have left the yard YRD.

[0062] On the other hand, as shown in FIG. 11, for example, when "3. Moving to the parking lot", a trailer is misdetected, and the flow line that was generated until the misdetection occurred is cut off (in other words, it is determined to be a different flow line). The cause of the cut off of the flow line may be, for example, an obstacle (for example, a tree object blown in by a tornado or a gust of wind) in the image captured by the trailer tracking camera, or the image quality may temporarily deteriorate, causing the trailer tracking device 20 to mistakenly detect that it is a different trailer. It goes without saying that the cause of the cut off of the flow line is not limited to the above-mentioned reasons. In this case, the trailer tracking device 20 assigns the trailer ID "UN0001" to the trailer, which is different from the trailer ID "0001" detected at the timestamp "0:00". This trailer ID "UN0001" is a trailer ID that is provisionally assigned when, for example, a trailer that was detected until just before suddenly becomes undetectable, and is treated as the identification information of a floating flow line that does not specify which other trailer it is. The trailer tracking device 20 additionally writes the timestamp "0:02" indicating the time when the trailer ID "UN0001" was detected and the trailer ID "UN0001" to the ID database DB1 at the time of "2. Check-in". The trailer tracking device 20 newly assigns a flow line ID "B" different from the existing flow line ID "A" to the trailer with the trailer ID "UN0001" detected at the timestamp "0:02". That is, the trailer tracking device 20 detects the position of the trailer (for example, coordinates (15, 37)) based on the captured image captured by the trailer tracking camera (see FIG. 5), and additionally writes the timestamp "0:02" indicating the time when the trailer position was detected, the trailer position (for example, coordinates (15, 37)), and the flow line ID "B" indicating that the trailer's flow line is a new flow line to the flow line database DB2 at the time of "2. Check-in".Furthermore, the trailer tracking device 20 additionally writes the timestamp "0:02", trailer ID "UN0001", flow line ID "B", and trailer position (e.g., coordinates (15, 37)) to the pairing database DB3 at the time of "2. Check-in" as a pairing process result using data having the timestamp "0:02" common to both the ID database DB1 and the flow line database DB2. Furthermore, the expected value database DB4 contains the same contents as those input at the time of "2. Check-in".

[0063] Thereafter, as shown in the second to fifth lines of the ID database DB1, the flow line database DB2, and the pairing database DB3 in Figure 11, the trailer (flow line ID "B") detected during the event "3. Moving to parking lot" continued to be detected normally until "6. 2nd parking".

[0064] As shown in Fig. 11, at the time of "7. Checking out", the trailer tracking device 20 detects the position of the trailer (e.g., coordinates (13, 50)) detected during the event "3. Moving to the parking lot" based on the captured image captured by the trailer tracking camera (see Fig. 5), associates the timestamp "2:00" indicating the time when the trailer position was detected with the trailer position (e.g., coordinates (13, 50)) and the flow line ID "B" indicating that the flow line of the trailer is an existing flow line, and adds and writes them to the flow line database DB2 at the time of "6. 2nd parking". In addition, the trailer tracking device 20 adds and writes the timestamp "2:00", trailer ID "UN0001", flow line ID "B", and trailer position (e.g., coordinates (13, 50)) corresponding to the same trailer to the pairing database DB3 at the time of "6. 2nd parking" as a pairing process result using data having the timestamp "2:00" common to the ID database DB1 and the flow line database DB2.

[0065] In "8. Archive and delete data of the ID" in Fig. 11, the trailer tracking device 20 deletes various data corresponding to the trailer with trailer ID "ID0001" that was detected until the event "3. Moving to the parking lot" was detected from each of the ID database DB1, the flow line database DB2, the pairing database DB3, and the expected value database DB4. Various data D1 and D2 corresponding to the trailer ID "UN0001" of the flow line ID "B" remain as floating flow line data. However, the trailer tracking device 20 (for example, the pairing processing unit 25 or the visualization application processing unit 26) performs a comparison judgment as to whether or not there is floating flow line data that matches the trailer parking position (PPOS) and trailer dock position (DPOS) in the expected value database DB4. As shown in FIG. 11, the trailer parking position (PPOS) "5, 31" in the expected value database DB4 matches the trailer position (POS) "5, 31" of the floating flow line, and the trailer dock position (DPOS) "30, 30" in the expected value database DB4 matches the trailer position (POS) "30, 30" of the floating flow line. Therefore, the trailer tracking device 20 determines that the flow line of the flow line ID [B] is the flow line of the trailer ID "ID0001", and archives various data D1 and D2 corresponding to the trailer ID "UN0001" of the flow line ID "B" as data corresponding to the trailer of the flow line ID "ID0001". The trailer tracking device 20 may perform a collation determination with the expected value database DB4 during the pairing process.

[0066] FIG. 11 shows an example in which a trailer is misdetected during "3. Moving to parking lot", and the flow line that was generated until the misdetection occurred is cut off. A trailer that has been checked into the yard YRD moves along a predetermined route, that is, in the order of check-in, 1st parking, docking-in, 2nd parking, and check-out. Therefore, if the flow line of a trailer with trailer ID "0001" is cut off while the trailer is moving along a predetermined route, the trailer tracking device 20 can estimate the approximate position of the trailer with trailer ID "0001" and display it on at least one of the operator browser device 31 and the yard jockey browser device 32. For example, if the trailer's flow line is cut off between check-in and 1st parking, the trailer tracking device 20 can estimate that the trailer has been checked in, and display "Trailer has been checked in" on at least one of the operator browser device 31 and the yard jockey browser device 32. Also, if the trailer's traffic line is interrupted between 1st parking and docking, the trailer tracking device 20 can estimate that the trailer has completed its movement to 1st parking and display "Trailer has completed its movement to 1st parking" or the like on at least one of the operator browser device 31 or the yard jockey browser device 32. Also, if the trailer's traffic line is interrupted between docking and 2nd parking, the trailer tracking device 20 can estimate that the trailer has completed its movement to the dock and display "Trailer has completed its movement to docking" or the like on at least one of the operator browser device 31 or the yard jockey browser device 32. Also, if the trailer's traffic line is interrupted between 2nd parking and check-out, the trailer tracking device 20 can estimate that the trailer has completed its movement to 2nd parking and display "Trailer has completed its movement to 2nd parking" or the like on at least one of the operator browser device 31 or the yard jockey browser device 32. In this way, since the trailer moves along a predetermined route, even if the trailer's traffic line is interrupted along the way, the trailer tracking device 20 can estimate the location where the trailer has passed and display it on the display device 30 including the operator browser device 31 or the yard jockey browser device 32.As a result, even if the traffic line is interrupted while the trailer is moving along a set route, the locations that the trailer has passed through can be estimated and displayed on the display device 30, so the trailer tracking device 20 can narrow down the location of the trailer having the trailer ID "0001" within the vast yard YRD.

[0067] 5. Example of visualization of trailer position and traffic flow Next, with reference to Figs. 12 to 17, visualization of the position and movement line of the trailer in the yard YRD tracked by the trailer tracking system 100 according to the present embodiment will be described. Fig. 12 is a bird's-eye view of a first movement line example of the trailer TRA1 checked in to the yard YRD. Fig. 13 is a diagram showing an example of a mapping display screen displayed on the display device 30, which visualizes that the trailer in Fig. 12 has been parked in a correct position as instructed. Fig. 14 is a diagram showing an example of a mapping display screen displayed on the display device 30, which visualizes that the trailer TRA1 in Fig. 12 has been parked in an incorrect position different from the instruction. Fig. 15 is a diagram showing a bird's-eye view of a second movement line example of the trailer TRA1 checked in to the yard YRD. Fig. 16 is a diagram showing an example of a mapping display screen displayed on the display device 30, which visualizes that the trailer TRA1 in Fig. 15 has been parked in a correct position as instructed. Fig. 17 is a diagram showing an example of a mapping display screen displayed on the display device 30, which visualizes that the trailer TRA1 in Fig. 15 has been parked in an incorrect position different from the instruction.

[0068] 12 to 17, the same elements as those in FIGS. 1, 2 and 5 are given the same reference numerals, and the description thereof is simplified or omitted, and differences will be described.

[0069] (1) Example of visualization of the first traffic flow: see Figures 12, 13, and 14 As shown in Fig. 12, when a tractor TRC (i.e., a transport truck TK1) pulling a trailer TRA passes through a check-in gate CIG of a yard YRD, it moves along a travel route Tm1 toward a designated parking space in a parking space PKS. After a certain time has elapsed after the movement, the tractor TRC (i.e., a transport truck TK1) pulling a trailer TRA moves from the parking space toward a dock DCK along a travel route Tm2. Such travel routes Tm1 and Tm2 are assumed.

[0070] When the visualization application processing unit 26 of the trailer tracking device 20 receives a visualization request from the display device 30, it generates a mapping display screen W1 (see Figure 13) or a mapping display screen W2 (see Figure 14) that includes the current position of the trailer TRA1 specified in the visualization request up to the time the visualization request was received (in other words, the current time) and the traffic line FLW1 from check-in at the yard YRD to the current position.

[0071] First, consider a case where a driver or yard jockey driving a tractor TRC1 towing a trailer TRA1 passes through a check-in gate CIG, moves to a designated parking space in a parking space PKS, and then moves appropriately from the parking space to a stopping position in a designated dock DCK (see Fig. 13). In other words, consider a visualization example of the flow line when the trailer TRA1 moves as instructed by the operator.

[0072] In this case, the visualization application processing unit 26 extracts the position of the trailer TRA1 obtained by continuously detecting the movement from the check-in gate CIG to the designated parking space of the parking space PKS and the movement from the parking space to the designated parking position of the dock DCK by referring to the pairing database DB3. Furthermore, the visualization application processing unit 26 generates a flow line FLW1 that connects each extracted position with a straight line or the like. The visualization application processing unit 26 generates a mapping display screen W1 in which an indicator IG1 indicating the position (current position) of the extracted trailer TRA1 and the flow line FLW1 are superimposed on a bird's-eye view map MP1 of the yard YRD, and outputs (transmits) the screen to at least the display device 30 that is the sender of the visualization request. Note that data of the map MP1 is stored in advance in the memory 21. The indicator IG1 notifies, for example, the trailer ID of the trailer TRA1 that is the subject of the visualization request in association with information indicating the current position (for example, information indicating "D07", which is the seventh parking space of the dock DCK).

[0073] On the other hand, consider a case where the driver or yard jockey driving the tractor TRC1 towing the trailer TRA1 passes through the check-in gate CIG and moves appropriately to the designated parking space in the parking space PKS, but moves from that parking space to a stopping position different from the designated stopping position in the dock DCK (see Fig. 14). In other words, consider a visualization example of the flow line when the trailer TRA1 cannot move as instructed by the operator.

[0074] In this case, the visualization application processing unit 26 extracts the position of the trailer TRA1 obtained by continuously detecting the movement from the check-in gate CIG to the designated parking space of the parking space PKS and the movement from the parking space to the stop position of the dock DCK by referring to the pairing database DB3. Furthermore, the visualization application processing unit 26 generates an inappropriate flow line ERFLW1 by connecting each extracted position with a straight line or the like. The visualization application processing unit 26 generates a mapping display screen W2 in which an indicator ERIG1 indicating an inappropriate position (current position) of the extracted trailer TRA1 and the flow line ERFLW1 are superimposed on a bird's-eye map MP1 of the yard YRD, and outputs (transmits) the mapping display screen W2 to at least the display device 30 that is the sender of the visualization request. The visualization application processing unit 26 may also generate and output (transmit) the mapping display screen W2 in which an indicator IG1C indicating the current position that is an appropriate destination defined in the expected value database DB4 is added and superimposed on the map MP1. The indicator ERIG1, for example, associates and reports the trailer ID of the trailer TRA1 that is the subject of the visualization request with information indicating the inappropriate current position where the trailer TRA1 has become located due to an erroneous movement (for example, information indicating "D04", the fourth parking space of the dock DCK). The indicator IG1C, for example, associates and reports the trailer ID of the trailer TRA1 that is the subject of the visualization request with information indicating the current position where the trailer TRA1 should be located at the time of receiving the visualization request (for example, information indicating "D07", the seventh parking space of the dock DCK).

[0075] (2) Visualization example of the second traffic flow: see Figures 15, 16, and 17 As shown in Fig. 15, a tractor TRC (i.e., a transport truck TK1) pulling a trailer TRA passes through a check-in gate CIG of a yard YRD and moves along a movement route Tm1 toward a designated parking space of a parking space PKS. Such a movement route Tm1 is assumed.

[0076] When the visualization application processing unit 26 of the trailer tracking device 20 receives a visualization request from the display device 30, it generates a mapping display screen W1 (see Figure 16) or a mapping display screen W2 (see Figure 17) that includes the current position of the trailer TRA1 specified in the visualization request up to the time the visualization request was received (in other words, the current time) and the traffic line FLW1 from check-in at the yard YRD to the current position.

[0077] First, consider a case where a driver or yard jockey driving a tractor TRC1 towing a trailer TRA1 passes through a check-in gate CIG and moves appropriately to a designated parking space in the parking space PKS (see Fig. 16). In other words, consider a visualization example of the flow line when the trailer TRA1 moves as instructed by the operator.

[0078] In this case, the visualization application processing unit 26 extracts the position of the trailer TRA1 obtained by continuously detecting the trailer TRA1 during the movement from the check-in gate CIG to the designated parking space of the parking space PKS by referring to the pairing database DB3. Furthermore, the visualization application processing unit 26 generates a flow line FLW2 by connecting each of the extracted positions with a straight line or the like. The visualization application processing unit 26 generates a mapping display screen W3 in which an indicator IG2 indicating the position (current position) of the trailer TRA1 extracted and the flow line FLW2 are superimposed on an overhead map MP1 of the yard YRD, and outputs (transmits) the mapping display screen W3 to the display device 30 which is the sender of at least the visualization request. The indicator IG2 notifies, for example, the trailer ID of the trailer TRA1 which is the subject of the visualization request, in association with information indicating the current position (for example, information indicating "S01" which is the first parking space of the parking space PKS).

[0079] On the other hand, consider a case where the driver or yard jockey driving the tractor TRC1 towing the trailer TRA1 passes through the check-in gate CIG and moves to a parking space PKE different from the designated parking space PKS (see Fig. 17). In other words, consider a visualization example of the flow of traffic when the trailer TRA1 cannot move as instructed by the operator.

[0080] In this case, the visualization application processing unit 26 extracts the position of the trailer TRA1 obtained by continuously detecting the position during the movement from the check-in gate CIG to the parking space PKE by referring to the pairing database DB3. Furthermore, the visualization application processing unit 26 generates an inappropriate flow line ERFLW2 by connecting each extracted position with a straight line or the like. The visualization application processing unit 26 generates a mapping display screen W4 in which an indicator ERIG2 indicating an inappropriate position (current position) of the extracted trailer TRA1 and the flow line ERFLW2 are superimposed on a bird's-eye view map MP1 of the yard YRD, and outputs (transmits) the mapping display screen W4 to at least the display device 30 that is the source of the visualization request. The visualization application processing unit 26 may also generate and output (transmit) the mapping display screen W4 in which an indicator IG2C indicating the current position that is an appropriate destination defined in the expected value database DB4 is added and superimposed on the map MP1. The indicator ERIG2, for example, associates and reports the trailer ID of the trailer TRA1 that is the subject of the visualization request with information indicating an inappropriate current position where the trailer TRA1 has become located due to an erroneous movement (for example, information indicating "E08", the eighth parking space in the parking space PKE). The indicator IG2C, for example, associates and reports the trailer ID of the trailer TRA1 that is the subject of the visualization request with information indicating a current position where the trailer TRA1 should be located at the time of receiving the visualization request (for example, information indicating "S01", the first parking space in the parking space PKS).

[0081] 6. Trailer Tracking System Operation Procedure Next, the operation procedure of trailer tracking system 100 according to the present embodiment will be described with reference to Fig. 18 and Fig. 19. Fig. 18 is a sequence diagram showing the operation procedure from trailer check-in to trailer information pairing processing of trailer tracking system 100 according to the present embodiment in chronological order. Fig. 19 is a sequence diagram showing the operation procedure of trailer position and traffic line visualization of trailer tracking system 100 according to the present embodiment in chronological order.

[0082] As a premise for the explanation of Fig. 18, captured image data is periodically input to the trailer tracking device 20 from at least each of the trailer tracking cameras C1 to C8 in accordance with the frames, and further, captured image data may be periodically input to the trailer tracking device 20 from each of the trailer ID reading cameras IC1 and IC2 in accordance with the frame rate. In addition, in the explanation of Fig. 18 or Fig. 19, the visualization application processing unit is abbreviated as "visualization app", the database is abbreviated as "DB", and at least one of the operator browser device and the yard jockey browser device is abbreviated as "browser device".

[0083] In FIG. 18, the visualization application processing unit 26 of the trailer tracking device 20 acquires an instruction for a process of detecting the check-in of a transportation truck (e.g., a trailer) that is about to enter the yard YRD (step St1) by an operation input by a user (e.g., an operator) using a browser device (step St0). The visualization application processing unit 26 instructs the trailer ID extraction unit 23 to execute a process of detecting a trailer ID. Based on the instruction from the visualization application processing unit 26, the trailer ID extraction unit 23 detects and extracts the trailer identification information IDct1 (TID: trailer ID) shown in the captured image by the trailer ID reading camera IC1 (see FIG. 4) (step St2). Note that in step St2, the trailer ID extraction unit 23 may extract the trailer identification information IDct1 (TID: trailer ID) based on an operation input by a user (e.g., an operator) using a browser device instead of the captured image by the trailer ID reading camera IC1 (see FIG. 4).

[0084] The trailer ID extraction unit 23 sends a TID registration request to the ID database DB1 to associate the trailer ID extracted in step St2 with a timestamp indicating the time of detection and register it in the ID database DB1 (step St3). In response to the TID registration request from the trailer ID extraction unit 23, the ID database DB1 registers the trailer ID extracted in step St2 with a timestamp indicating the time of detection of the trailer ID in association with each other (step St4), and returns a registration completion response (Ack: Acknolwedgement) to the visualization application processing unit 26 (step St5).

[0085] During operation of the trailer tracking device 20, the trailer flow line extraction unit 24 acquires captured image data from each of the trailer tracking cameras C1 to C8 (step St6), and detects the position (POS) of the trailer shown in the captured images using the captured images, and extracts the position information (step St6). At this time, the trailer flow line extraction unit 24 determines whether the position indicated by the extracted position information constitutes the same or different flow line with the position indicated by the most recently (previously) extracted position information, based on a predetermined determination criterion (see above). In other words, when the trailer flow line extraction unit 24 determines that the position indicated by the extracted position information is included within a predictable range (see above) from the position indicated by the most recently (previously) extracted position information, it assigns the same flow line ID as the flow line ID of the existing flow line. On the other hand, if the trailer traffic flow extraction unit 24 determines that the position indicated by the extracted position information is not within a predictable range (see above) from the position indicated by the most recently (previously) extracted position information, it assigns a new traffic flow ID.

[0086] The trailer flow line extraction unit 24 sends a POS registration request to the flow line database DB2 to associate and register the trailer position information extracted in step St6, the assigned flow line ID, and the time stamp indicating the time when the position was detected in the flow line database DB2 (step St7). In response to the POS registration request from the trailer flow line extraction unit 24, the flow line database DB2 associates and registers the trailer position information extracted in step St6, the assigned flow line ID, and the time stamp indicating the time when the position was detected in the form of a single entry (step St8), and returns a registration completion response (Ack) to the visualization application processing unit 26 (step St9).

[0087] Here, the trailer flow line extraction unit 24 instructs the pairing processing unit 25 to determine whether or not predetermined conditions (eg, [Condition 1], [Condition 2], and [Condition 3], see FIG. 18) are satisfied (steps St10, St16, St20).

[0088] In step St10, the trailer flow line extraction unit 24 associates the trailer position information extracted in step St6 with the assigned flow line ID and a time stamp indicating the time when the position was detected, and sends them to the pairing processing unit 25 (step St10).

[0089] The pairing processing unit 25 accesses the trailer ID extraction unit 23 or the ID database DB1 (step St11) to obtain the data of the trailer ID extraction result from the trailer ID extraction unit 23 (step St12). Here, the pairing processing unit 25 determines, as [Condition 1], whether a new flow line ID (FID) has been assigned in step St6 and whether the position (POS) of the trailer near the check-in gate CIG, the flow line, and the trailer ID (TID) have been obtained. In other words, [Condition 1] specifies that when a trailer is detected for the first time at the check-in gate CIG, various data such as the timestamp, position, and flow line of the trailer are associated and stored as initial data.

[0090] The pairing processing unit 25 determines that [Condition 1] is satisfied (i.e., a new flow line ID (FID) has been assigned in step St6, and the position (POS) of the trailer near the check-in gate CIG, the flow line, and the trailer ID (TID) have been acquired). In this case, the pairing processing unit 25 sends a new registration request to the pairing database DB3 to associate the timestamp (TM) indicating the time when the trailer was detected near the check-in gate CIG, the position (POS) of the trailer, the flow line ID (FID) indicating the flow line of the trailer, and the trailer ID (TID) in the pairing database DB3 (step St13). In response to a new registration request from the pairing processing unit 25, the pairing database DB3 registers a timestamp (TM) indicating the time when the trailer was detected near the check-in gate CIG, the position (POS) of the trailer, a traffic flow ID (FID) indicating the traffic flow of the trailer, and an ID (TID) of the trailer in association with each other (step St14), and returns a response (Ack) of registration completion to the pairing processing unit 25 (step St15).

[0091] In step St16, the trailer flow line extraction unit 24 associates the trailer position information extracted in step St6 with the assigned flow line ID and a time stamp indicating the time when the position was detected, and sends them to the pairing processing unit 25 (step St16).

[0092] The pairing processing unit 25 determines, as [Condition 2], whether or not a new flow line ID (FID) has been assigned in step St6 and whether or not the trailer's position (POS) and flow line and trailer ID (TID) at a location in the yard YRD other than the check-in gate CIG have been acquired. In other words, [Condition 2] specifies that when a trailer once detected at the check-in gate CIG is erroneously detected as another trailer while moving in the yard YRD, various data such as the flow line, position, and time stamp of the erroneously detected trailer are stored as temporary floating flow line data.

[0093] The pairing processing unit 25 determines that [Condition 2] is satisfied (i.e., a new flow line ID (FID) was assigned in step St6, and the trailer position (POS) at a location in the yard YRD other than the check-in gate CIG, and the flow line and trailer ID (TID) were acquired). In this case, the pairing processing unit 25 sends a new registration request to the pairing database DB3 to associate the timestamp (TM) indicating the time when the trailer was detected at a location in the yard YRD other than the check-in gate CIG, the trailer position (POS), the flow line ID (FID: i.e. the newly assigned flow line ID) indicating the flow line of the trailer, and the trailer ID (TID) in the pairing database DB3, and register them in the pairing database DB3 (step St17). In response to a new registration request from the pairing processing unit 25, the pairing database DB3 registers a timestamp (TM) indicating the time when the trailer was detected at a location in the yard YRD other than the check-in gate CIG, the position (POS) of the trailer, a traffic line ID (FID: i.e., the newly assigned traffic line ID) indicating the traffic line of the trailer, and the trailer ID (TID) in association with each other (step St18), and returns a response (Ack) of registration completion to the pairing processing unit 25 (step St19).

[0094] In step St20, the trailer flow line extraction unit 24 associates the trailer position information extracted in step St6 with the assigned flow line ID and a time stamp indicating the time when the position was detected, and sends them to the pairing processing unit 25 (step St20).

[0095] The pairing processing unit 25 determines whether or not an existing flow line ID (FID) has been assigned in step St6 (in other words, the trailer ID corresponding to that flow line ID has already been detected) as [Condition 3]. In other words, [Condition 3] stipulates that when a trailer that has passed through the check-in gate CIG and is already moving within the yard YRD is normally detected within the yard YRD, various data of the trailer, such as its timestamp, position, flow line, etc. within the yard YRD, are associated and stored.

[0096] The pairing processing unit 25 determines that [Condition 3] is satisfied (i.e., an existing flow line ID (FID) has been assigned in step St6 (in other words, a trailer ID corresponding to that flow line ID has already been detected). In this case, the pairing processing unit 25 associates the timestamp (TM) indicating the time when the trailer was detected in a yard YRD other than the check-in gate CIG, the position (POS) of the trailer, the flow line ID (FID: i.e., an already assigned flow line ID) indicating the flow line of the trailer, and the trailer ID (TID) in the pairing database DB3, and registers them. In response to the additional registration request from the pairing processing unit 25, the pairing database DB3 registers a time stamp (TM) indicating the time when the trailer was detected in a yard YRD other than the check-in gate CIG, the position (POS) of the trailer, a flow line ID (FID: i.e., an already-assigned flow line ID) indicating the flow line of the trailer, and the trailer ID (TID) in association with each other (step St22), and returns a response (Ack) indicating the completion of registration to the pairing processing unit 25 (step St23).

[0097] In addition, the visualization application processing unit 26 of the trailer tracking device 20 acquires an instruction for a process of detecting the check-out of a transportation truck (e.g., a trailer) that is about to leave the yard YRD (step St25) by an operation input by a user (e.g., an operator) using the browser device (step St24). The visualization application processing unit 26 instructs the trailer ID extraction unit 23 to execute a process of deleting the trailer ID of the trailer that is no longer required to be managed because it is leaving the yard YRD. Based on the instruction from the visualization application processing unit 26, the trailer ID extraction unit 23 sends an ID deletion request to the ID database DB1 to delete the corresponding trailer ID and a timestamp indicating the time when the trailer ID was detected from the ID database DB1 (step St26). In accordance with the ID deletion request from the trailer ID extraction unit 23, the ID database DB1 deletes the trailer ID extracted in step St2 and the timestamp indicating the time when the trailer ID was detected (step St27), and returns a response (Ack) of the completion of deletion to the visualization application processing unit 26 (step St28).

[0098] 19, the browser device (display device 30) generates a visualization request (parking map display) for requesting visualization of the current trailer position and traffic line in response to an operation input by a user (e.g., a yard jockey) (step St31), and sends the visualization request to the trailer tracking device 20 (step St32). This visualization request (parking map display) is a command, for example, when the yard jockey moves to the location of a specified trailer but is unable to find it, and requests visualization of a parking map within the premises of yard YRD (i.e., the location on the map of the parking space where the trailer is parked). The visualization request (parking map display) includes at least the trailer ID of the desired trailer.

[0099] When the visualization application processing unit 26 of the trailer tracking device 20 receives the visualization request of step St32, it generates an information acquisition request regarding the current position (e.g., parking position) of the trailer corresponding to the trailer ID included in the visualization instruction (parking map display) and sends it to the pairing processing unit 25 or the pairing database DB3 (step St33). The pairing processing unit 25 or the pairing database DB3 acquires various corresponding data (e.g., trailer ID, timestamp indicating the current time, trailer position information at the current time, and traffic line ID) from the pairing database DB3 in accordance with the information acquisition request of step St33 (step St34). Note that the current time may be the time when the visualization request is generated by the browser device or the time when the visualization request is received by the trailer tracking device 20, and the same applies below.

[0100] The pairing processing unit 25 or the pairing database DB3 returns the various data (see above) acquired in step St34 to the visualization application processing unit 26 (step St35). The visualization application processing unit 26 generates (draws) a mapping display screen (see, for example, the mapping display screen W1 in FIG. 13 or the mapping display screen W3 in FIG. 16) using the various data (see above) acquired in step St35 (step St36). The visualization application processing unit 26 transmits the mapping display screen generated in step St36 to the browser device to output it (step St37).

[0101] Meanwhile, the browser device (display device 30) generates a visualization request (display of illegal parking position) for requesting visualization of whether or not a trailer is illegally (in other words, erroneously) parked at the current time in response to an operational input by a user (e.g., an operator) (step St41A), and sends the visualization request to the trailer tracking device 20 (step St42A). This visualization request (display of illegal parking position) is a command requesting visualization of a parking map within the yard YRD (i.e., the position on the map of the parking space where the trailer is illegally parked) when, for example, an operator periodically monitors whether or not there is a trailer illegally parked within the yard YRD. The visualization request (display of illegal parking position) includes at least the trailer ID of the target trailer.

[0102] In addition, if the trailer traffic flow extraction unit 24 determines that [Condition 2] (see step St16 in Figure 18) is satisfied (step St41B), it may generate the above-mentioned visualization request (display of illegal parking location) and send it to the visualization application processing unit 26 (step St42B).

[0103] When the visualization application processing unit 26 acquires the visualization request (display of illegal parking position), it generates a parking position information acquisition request for acquiring various data of the trailer corresponding to the trailer ID included in the visualization request (display of parking map) and sends it to the pairing processing unit 25 or the pairing database DB3 (step St43). The pairing processing unit 25 or the pairing database DB3 acquires the corresponding various data (e.g., trailer ID, timestamp indicating the current time, trailer position information at the current time, and traffic line ID) from the pairing database DB3 in accordance with the parking position information acquisition request of step St43 (step St44). The pairing processing unit 25 or the pairing database DB3 returns the various data acquired in step St44 (see above) to the visualization application processing unit 26 (step St45).

[0104] The visualization application processing unit 26 also generates a parking position information acquisition request for acquiring various data of the trailer corresponding to the trailer ID included in the visualization request (parking map display) and sends it to the expected value database DB4 (step St46). The expected value database DB4 acquires the corresponding various data (e.g., trailer ID, timestamp indicating the current time, trailer parking position information that should be located at the current time, trailer dock position that should be located at the current time) from the expected value database DB4 in accordance with the parking position information acquisition request of step St46 (step St47). The expected value database DB4 returns the various data acquired in step St47 (see above) to the visualization application processing unit 26 (step St48).

[0105] The visualization application processing unit 26 performs a comparison process to determine whether or not various data from the pairing database DB3 acquired in step St45 matches various data from the expected value database DB4 acquired in step St48 (step St49). The visualization application processing unit 26 generates (draws) a mapping display screen (see, for example, the mapping display screen W2 in FIG. 14 or the mapping display screen W4 in FIG. 17) showing the result of the comparison process in step St49 (step St50). The visualization application processing unit 26 transmits the mapping display screen generated in step St50 to the browser device to output it (step St51).

[0106] As described above, the trailer tracking device 20 according to the present embodiment includes an acquisition unit (communication interface 22) that acquires images captured by the cameras from at least one camera (trailer ID reading camera IC1, trailer tracking cameras C1 to C8) that captures images of a predetermined moving path of a trailer (trailer TRA1) in a yard YRD having a vast site, a database (pairing database DB3) that associates and stores trailer identification information (trailer ID) and information on the position of the trailer in the yard YRD based on the captured images of the cameras, and a visualization processing unit (visualization application processing unit 26) that outputs information on the current position of the trailer specified by a visualization instruction from an external browser device (display device 30) to the external browser device based on the database. As a result, the trailer tracking device 20 can appropriately manage the position information of the trailer in the yard YRD, so that the position of the trailer that is the target (for example, the trailer that needs to be towed) can be easily visualized for the worker (yard jockey), and the yard jockey can efficiently support finding the target trailer.

[0107] Moreover, trailer tracking device 20 further includes a pairing processing unit 25 that associates trailer identification information (trailer ID) with information on the position of the trailer within the trailer yard YRD based on the captured image of the camera. This allows trailer tracking device 20 to appropriately link and manage the trailer information as trailer information even if the device from which the trailer ID is acquired (e.g., trailer ID reading camera IC1) is different from the device from which the position information of the trailer within the trailer yard YRD is acquired (e.g., trailer tracking cameras C1 to C8).

[0108] The cameras are also provided at the entrance (check-in gate CIG) of yard YRD and include a first camera (trailer ID reading camera IC1) that reads the identification information (trailer ID) of the trailer. Trailer tracking device 20 further includes a trailer ID extraction unit 23 that extracts the identification information (trailer ID) of the trailer based on the image captured by the first camera. This allows trailer tracking device 20 to properly detect and extract the trailer ID of a trailer attempting to enter yard YRD based on the image captured by trailer ID reading camera IC1.

[0109] The cameras also include at least one second camera (trailer tracking cameras C1-C8) that is provided within the yard YRD and captures a predetermined movement path of the trailer (trailer TRA1) within the vast yard YRD. The trailer tracking device 20 estimates the position of the trailer within the yard YRD based on the captured image of the second camera, and further includes a trailer flow line extraction unit 24 that extracts the movement line of the trailer determined by the estimated position of the trailer within the yard YRD at each time. This allows the trailer tracking device 20 to quantify and store (register) the position and movement line (i.e., movement trajectory) of the trailer within the yard YRD using the captured image of the trailer tracking camera that can capture the movement path of the trailer within the yard YRD.

[0110] Furthermore, when the trailer flow line extraction unit 24 determines that the first position (latest detected position) indicating the position of the trailer in the yard YRD based on the image captured by the second camera (trailer tracking cameras C1-C8) is within the predicted range, it adds the flow line to the first position to the already extracted flow line. When the trailer flow line extraction unit 24 determines that the first position indicating the position of the trailer in the yard YRD based on the image captured by the second camera is outside the predicted range, it adds the flow line to the first position as a new flow line. In this way, the trailer tracking device 20 can manage the trailer flow line by connecting (wiring) it to an existing flow line depending on whether or not it is possible to predict the position of the trailer that may move within the yard YRD, and can also manage it as a new flow line starting from the trailer position.

[0111] Furthermore, the visualization processing unit (visualization application processing unit 26) associates the current position of the trailer specified by the visualization instruction with a flow line indicating the movement trajectory in the yard YRD up to the current position, and outputs the associated information to the external browser device (display device 30). This allows the trailer tracking device 20 to visualize and output to the display device 30 the current position of the trailer that the user (e.g., operator or yard jockey) who uses the display device 30 wants to find, and the flow line (movement trajectory) of the trailer in the yard YRD, in association with each other.

[0112] The trailer tracking device 20 further includes a second database (expected value database DB4) that stores information on the position of the trailer on the movement path for each time in the yard YRD. The visualization processing unit (visualization application processing unit 26) outputs to the external browser device (display device 30) the current position of the trailer specified by the visualization instruction and the position where the trailer should be stopped at the current time in a comparative manner. This allows the trailer tracking device 20 to visualize whether the current position of the trailer in the yard YRD, which is periodically monitored by a user (e.g., an operator) using the display device 30, matches the position where it should be located correctly, and provides the user with information for making a decision.

[0113] (Modification of the first embodiment) In a modified example of the first embodiment (hereinafter referred to as the "modified example"), the trailer tracking device calculates the trailer detection reliability using a prediction probability indicating whether or not the trailer itself is properly detected, and a score indicating the matching rate between the predicted position of the trailer and the actual detected position. The trailer tracking device changes the display mode of the trailer position and traffic line on the mapping display screen according to the calculation result of this trailer detection reliability. Note that the configuration of the trailer tracking system according to the modified example, other than the trailer tracking device, is the same as the configuration of the trailer tracking system 100 according to the first embodiment, other than the trailer tracking device 20.

[0114] 6. Configuration of trailer tracking device Next, with reference to Fig. 20A, Fig. 20B, and Fig. 20C, the trailer flow line extraction unit 24A, the flow line database DB2A, and the pairing database DB3A of the trailer tracking device 20A according to the modified example will be described. Fig. 20A is a block diagram showing a detailed functional configuration example of the trailer flow line extraction unit according to the modified example. Fig. 20B is a diagram showing a configuration example of the flow line database DB2A according to the modified example. Fig. 20C is a diagram showing a configuration example of the pairing database DB3A according to the modified example. In the description of each of Figs. 20A to 20C, the same reference numerals are given to the same configurations as those of the elements according to the first embodiment, and the description is simplified or omitted, and the different contents will be described.

[0115] The configuration of the trailer tracking device 20A according to the modified example is the same as the configuration of the trailer tracking device 20 according to the embodiment 1, except for the trailer flow line extraction unit 24A, the flow line database DB2A, and the pairing database DB3A. Therefore, Fig. 20A shows a detailed configuration example of the trailer flow line extraction unit 24A, Fig. 20B shows a detailed configuration example of the flow line database DB2A, and Fig. 20C shows a detailed configuration example of the pairing database DB3.

[0116] The trailer flow line extraction unit 24A includes a trailer detection unit 24A1, a tracking unit 24A2, and a reliability calculation unit 24A3.

[0117] The trailer detection unit 24A1 periodically acquires data from the communication interface 22 (i.e., captured image data from each of the trailer tracking cameras C1-C8). The trailer detection unit 24A1 periodically extracts position information indicating the position of the trailer shown in the captured image by performing analysis on the captured image data from the trailer tracking camera (e.g., trailer tracking camera C1). The trailer detection unit 24A1 sends trailer detection information (e.g., the size and position of the trailer detection frame) to the tracking unit 24A2. In addition, upon detecting the trailer, the trailer detection unit 24A1 calculates a predicted probability (score0) indicating the detection accuracy of the trailer itself, and sends it to the reliability calculation unit 24A3.

[0118] The tracking unit 24A2 refers to various data (traffic line information) stored (registered) in the traffic line database DB2A, determines whether the position indicated by the position information included in the detection information from the trailer detection unit 24A1 constitutes the same traffic line or a different traffic line with the position indicated by the most recently (previously) extracted position information, and assigns a traffic line ID based on the result of this determination. The tracking unit 24A2 also calculates a score (score1) indicating the matching rate between the predicted position of the trailer based on the most recently (previously) detected position of the trailer and the actual detected position (latest position), and sends it to the reliability calculation unit 24A3. The tracking unit 24A2 associates a timestamp, which is the time when the trailer was detected, with the traffic line ID (FID) of the trailer and the extracted position information, and stores (registers) it in the traffic line database DB2A.

[0119] The reliability calculation unit 24A3 calculates the trailer detection reliability CS by calculating a predetermined function (f(score0, score1)) to which the predicted probability (score0) from the trailer detection unit 24A1 and the score (score1) from the tracking unit 24A2 are input. The reliability calculation unit 24A3 stores (registers) the trailer detection reliability CS in each of the movement line database DB2A and the pairing database DB3A. Note that in each of Figs. 20A to 20C, the trailer detection reliability is abbreviated to "reliability".

[0120] The flow line database DB2A is a database that stores (registers) a time stamp (TM) indicating the time when the trailer position information was extracted by the trailer flow line extraction unit 24, a trailer position (POS) indicated by the extracted position information (e.g., two-dimensional coordinates), a flow line ID (FID) indicating whether the trailer position constitutes a new flow line or an existing flow line, and a trailer detection reliability calculated by the reliability calculation unit 24A3 in association with each other (see FIG. 20BB). As shown in FIG. 20B, the flow line database DB2A stores the time stamp (TM), the flow line ID (FID), the trailer position (POS), and the trailer detection reliability in association with each other.

[0121] The pairing database DB3A is a database that associates and stores (registers) a time stamp (TM) indicating the time when the trailer ID was extracted and the time when the trailer position information was extracted, the extracted trailer ID (TID), the trailer position (POS) indicated by the extracted position information, a flow line ID (FID) indicating whether the trailer position constitutes a new flow line or an existing flow line, and the trailer detection reliability calculated by the reliability calculation unit 24A3 (see FIG. 20C). As shown in FIG. 20C, the pairing database DB3A associates and stores the time stamp (TM), the trailer ID (TID), the flow line ID (FID), the trailer position (POS), and the trailer detection reliability.

[0122] 7. Mapping screen display mode Next, various display modes of the mapping display screen according to the modified example will be described with reference to Fig. 21 to Fig. 24. Fig. 21 is a diagram showing a first visualization example of the position of a trailer checked in to the yard YRD, displayed on the display device 30 according to the modified example. Fig. 22 is a diagram showing a second visualization example of the position of a trailer checked in to the yard YRD, displayed on the display device 30 according to the modified example. Fig. 23 is a diagram showing a third visualization example of the position of a trailer checked in to the yard YRD, displayed on the display device 30 according to the modified example. Fig. 24 is a diagram showing a fourth visualization example of the position of a trailer checked in to the yard YRD, displayed on the display device 30 according to the modified example.

[0123] For example, it is assumed that the trailer detection reliability calculated by the reliability calculation unit 24A3 is smaller than a predetermined threshold stored in the memory 21. In this case, the visualization application processing unit 26 generates a mapping display screen W5 in which an indicator IG3 indicating a predetermined area AR1 (e.g., a parking space PKS) including a position where the trailer is detected (e.g., any position of the parking space PKS) and a flow line FLW3 from the check-in gate CIG to the detected position are superimposed on a bird's-eye map MP1 of the yard YRD, based on a visualization request from the display device 30. The area AR1 is drawn in, for example, a single color. The indicator IG3 notifies, for example, the trailer ID "ID0001" of the trailer that is the subject of the visualization request, in association with information indicating the area AR1 including the position estimated as the current position (e.g., information indicating "AreaS", which is the identification information of the parking space PKS). The visualization application processing unit 26 outputs (transmits) the generated mapping display screen WD5 at least to the display device 30 that is the source of the visualization request (see FIG. 21). As a result, the trailer tracking device 20A can visualize a wide area where the trailer is likely to be located, even in situations where it is difficult to detect the trailer's position, such as when the trailer detection reliability is lower than a predetermined threshold, and visually present this to the operator or yard jockey.

[0124] For example, it is assumed that the trailer detection reliability calculated by the reliability calculation unit 24A3 is smaller than a predetermined threshold stored in the memory 21. In this case, the visualization application processing unit 26 generates a mapping display screen W6 in which an indicator IG3 indicating a predetermined area AR1a (e.g., a parking space PKS) including a position where the trailer is detected (e.g., any position of the parking space PKS) and a traffic line FLW3 from the check-in gate CIG to the detected position are superimposed on a bird's-eye map MP1 of the yard YRD, based on a visualization request from the display device 30. The area AR1a is drawn in a light color, for example, near the end thereof because the probability of parking is low, and is drawn in a dark color from the end toward the center because the probability of parking increases. The indicator IG3, for example, reports the trailer ID "ID0001" of the trailer that is the subject of the visualization request and information indicating the area AR1a including the position estimated to be the current position (e.g., information indicating "AreaS", which is the identification information of the parking space PKS) in association with each other. The visualization application processing unit 26 outputs (transmits) the generated mapping display screen WD6 to at least the display device 30 that is the sender of the visualization request (see FIG. 22). As a result, even in a situation where it is difficult to detect the position of the trailer, such as when the trailer detection reliability is lower than a predetermined threshold, the trailer tracking device 20A can visually present to the operator or yard jockey the magnitude of the detection probability in an area where the trailer is likely to be located, in a suggestive and wide range manner.

[0125] For example, it is assumed that the trailer detection reliability calculated by the reliability calculation unit 24A3 is smaller than a predetermined threshold stored in the memory 21. In this case, the visualization application processing unit 26 generates a mapping display screen W7 based on a visualization request from the display device 30, in which an indicator IG4 indicating the position where the trailer was detected (for example, any position of the parking space PKS) and a flow line FLW4 from the check-in gate CIG to the position where the trailer is estimated to be present are superimposed on a bird's-eye map MP1 of the yard YRD. The indicator IG4 at least notifies, for example, the trailer ID "ID0001" of the trailer that is the subject of the visualization request. The visualization application processing unit 26 outputs (transmits) the generated mapping display screen WD7 to at least the display device 30 that is the source of the visualization request (see FIG. 23). As a result, the trailer tracking device 20A can visualize the area around the trailer where it is likely to be located roughly even in situations where it is difficult to detect the trailer's position, such as when the trailer detection reliability is lower than a predetermined threshold, and can visually present this to the operator or yard jockey.

[0126] For example, it is assumed that the trailer detection reliability calculated by the reliability calculation unit 24A3 is smaller than a predetermined threshold value stored in the memory 21. In this case, the visualization application processing unit 26 generates a mapping display screen W8 on the basis of a visualization request from the display device 30, in which an indicator IG5a indicating the position where the trailer was detected (for example, any position of the parking space PKS) and a flow line FLW5a from the check-in gate CIG to the position where the trailer is estimated to be present are superimposed on a bird's-eye view map MP1 of the yard YRD. The indicator IG5a at least notifies, for example, the trailer ID "ID0001" of the trailer that is the subject of the visualization request and that the trailer detection reliability is relatively low (CS: low level).

[0127] For example, it is assumed that the trailer detection reliability calculated by the reliability calculation unit 24A3 is greater than a predetermined threshold stored in the memory 21. In this case, the visualization application processing unit 26 generates a mapping display screen W8 on the basis of a visualization request from the display device 30, in which an indicator IG5b indicating the position where the trailer was detected (for example, any position of the parking space PKS) and a flow line FLW5b from the check-in gate CIG to the position where the trailer is estimated to be present are superimposed on a bird's-eye view map MP1 of the yard YRD. The indicator IG5b at least notifies, for example, the trailer ID "ID0001" of the trailer that is the subject of the visualization request and that the trailer detection reliability is relatively high (CS: high level).

[0128] The visualization application processing unit 26 outputs (transmits) the generated mapping display screen WD8 to at least the display device 30 that is the sender of the visualization request (see FIG. 24). This allows the trailer tracking device 20A to flexibly change and visualize the width of the traffic line depending on the level of accuracy of trailer position detection (trailer detection reliability), such as when the trailer detection reliability is smaller or larger than a predetermined threshold, and visually present it to the operator or yard jockey.

[0129] As described above, in the trailer tracking device 20A according to the modified example, the trailer flow line extraction unit 24A calculates the reliability of the trailer (trailer detection reliability) based on the captured image of the second camera (trailer tracking cameras C1 to C8). The visualization processing unit (visualization application processing unit 26) changes the display mode (see Figs. 21 to 24) of the area indicator within the yard indicating the current position of the trailer specified by the visualization instruction from the external browser device (display device 30) based on the reliability calculation result, and outputs it to the external browser device. In this way, the trailer tracking device 20A can visually present to the operator or yard jockey the likelihood that the trailer has been detected according to the magnitude of the trailer detection reliability.

[0130] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the various embodiments described above may be arbitrarily combined within the scope of the invention.

[0131] In the above-mentioned first embodiment or its modification, the trailer flow line extraction unit 24, 24A may be configured to exclude objects other than the trailer from the detection target when the trailer tracking device 20, 20A detects the trailer using each of the trailer tracking cameras C1 to C8. Also, each of the trailer tracking cameras C1 to C8 may be equipped with a learning model capable of detecting the trailer, and the trailer may be detected using this learning model and transmitted to the trailer tracking device 20, 20A. In this way, the trailer tracking device 20, 20A can be configured to omit the process of detecting the trailer. [Industrial Applicability]

[0132] The present disclosure is useful as a trailer tracking device, a trailer tracking system, and a trailer tracking method that efficiently assist workers in a yard in easily visualizing the location of a target trailer. [Explanation of symbols]

[0133] 10. Camera 20, 20A Trailer Tracking Device 21 Memory 22, 27 Communication Interface 23 Trailer ID Extraction Unit 24 Trailer flow line extraction unit 24A1 Trailer detection unit 24A2 Tracking part 24A3 Reliability calculation unit 25 Pairing processing unit 26 Visualization application processing section 30 Display device 31 Operator Browser Device 32 Yard Jockey Browser Device 100 Trailer Tracking System C1, C2, C3, C4, C5, C6, C7, C8 Trailer Tracking Camera DB1 ID Database DB2 Traffic Database DB3 Pairing Database DB4 Expected Value Database IC1, IC2 Trailer ID reader camera LC1, LC2 license plate reading camera CIG Check-in Gate COG Checkout Gate TRA1 Trailer TRC1 Tractor YRD Yard

Claims

1. A memory and a processor, The processor uses the memory to: Acquiring an image captured by at least one camera that captures an image of a predetermined moving path of the trailer within the yard; The cameras include a first camera provided at an entrance of the yard to read the identification information of the trailer, and at least one second camera provided within the yard to capture an image of the travel path; The first camera periodically captures an image having a front portion of the trailer as a subject, The processor uses the memory to: storing the identification information of the trailer and information on the position of the trailer within the yard based on the image captured by the camera in a database in association with each other; Obtaining information identifying a first trailer; Based on the information identifying the first trailer, read out the location information of the first trailer corresponding to the first trailer from the database and output the read out location information of the first trailer; A position of the trailer within the yard is estimated based on the image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; When it is determined that a first position indicating the position of the trailer in the yard based on the image captured by the second camera is within a predicted range of the position of the trailer that may move in the yard, a flow line to the first position is added to the previously extracted flow line that has been extracted most recently; when it is determined that a first position indicating a position of the trailer in the yard based on an image captured by the second camera is outside a prediction range, a traffic line to the first position is added as a new traffic line that has not been extracted recently. Trailer tracking device.

2. The processor extracts identification information of the trailer based on the captured image captured by the first camera.

2. A trailer tracking device according to claim 1.

3. The processor outputs a current position of the first trailer in association with a flow line indicating a movement trajectory in the yard up to the current position, based on information identifying the first trailer.

2. A trailer tracking device according to claim 1.

4. The processor, storing information on the position of the first trailer on the travel route within the yard for each hour in a second database; outputting, based on information identifying the first trailer, a current position of the first trailer and a position where the first trailer should be stopped at the current time in a comparative manner; 4. A trailer tracking device according to claim 3.

5. A method and device comprising: The processor uses the memory to: Acquiring an image captured by at least one camera that captures an image of a predetermined moving path of the trailer within the yard; The cameras include a first camera provided at an entrance of the yard to read the identification information of the trailer, and at least one second camera provided within the yard to capture an image of the travel path; The first camera periodically captures an image having a front portion of the trailer as a subject, The processor uses the memory to: storing the identification information of the trailer and information on the position of the trailer within the yard based on the image captured by the camera in a database in association with each other; Obtaining information identifying a first trailer; Based on the information identifying the first trailer, read out the location information of the first trailer corresponding to the first trailer from the database and output the read out location information of the first trailer; The processor estimates a position of the trailer within the yard based on the image captured by the second camera, and extracts a traffic line of the trailer determined by the estimated position of the trailer within the yard at each time; Calculating a reliability of the first trailer based on a predicted probability indicating whether the trailer itself is properly detected or not and a score indicating a matching rate between the predicted position of the trailer and an actual detected position of the trailer, based on an image captured by the second camera; changing and outputting a display mode of an area indicator within the yard indicating the current position of the first trailer based on a calculation result of the reliability; Trailer tracking device.

6. A trailer tracking method executed by a trailer tracking device communicably connected to at least one camera that captures an image of a predetermined moving path of a trailer within a yard, comprising: Acquiring an image captured by at least one camera that captures an image of a predetermined moving path of the trailer within the yard; The cameras include a first camera provided at an entrance of the yard and configured to read identification information of the trailer, and at least one second camera provided within the yard and configured to capture an image of the travel path; The first camera periodically captures an image having a front portion of the tractor as a subject, storing the identification information of the trailer and information on the position of the trailer within the yard based on the image captured by the camera in a database in association with each other; Obtaining information identifying a first trailer; Based on the information identifying the first trailer, read out the location information of the first trailer corresponding to the first trailer from the database and output the read out location information of the first trailer; A position of the trailer within the yard is estimated based on the image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; When it is determined that a first position indicating the position of the trailer in the yard based on the image captured by the second camera is within a predicted range of the position of the trailer that may move in the yard, a flow line to the first position is added to the previously extracted flow line that has been extracted most recently; when it is determined that a first position indicating a position of the trailer in the yard based on an image captured by the second camera is outside a prediction range, a traffic line to the first position is added as a new traffic line that has not been extracted recently. Trailer tracking method.

7. A memory and a processor, The processor uses the memory to: Acquiring an image captured by at least one camera that captures an image of a predetermined moving path of the trailer within the yard; The cameras include a first camera provided at an entrance of the yard and configured to read identification information of the trailer, and at least one second camera provided within the yard and configured to capture an image of the travel path; The processor uses the memory to: The first camera periodically captures an image having a front portion of the tractor as a subject, storing the identification information of the trailer and information on the position of the trailer within the yard based on the image captured by the camera in a database in association with each other; Obtaining information identifying a first trailer; Based on the information identifying the first trailer, read out the location information of the first trailer corresponding to the first trailer from the database and output the read out location information of the first trailer; A position of the trailer within the yard is estimated based on the image captured by the second camera, and a flow line of the trailer determined by the estimated position of the trailer within the yard at each time is extracted; When it is determined that a first position indicating the position of the trailer in the yard based on the image captured by the second camera is within a predicted range of the position of the trailer that may move in the yard, a flow line to the first position is added to the previously extracted flow line that has been extracted most recently; when it is determined that a first position indicating a position of the trailer in the yard based on an image captured by the second camera is outside a prediction range, a traffic line to the first position is added as a new traffic line that has not been extracted recently. Trailer tracking system.

Images