Vehicle tracking system and vehicle tracking method

Abstract

To improve the efficiency of operations related to toll collection on toll roads. [Solution] The vehicle tracking system includes: an information acquisition unit that performs image recognition (first image recognition) on images of vehicles passing through the entrance toll gate of a toll road and acquires at least predetermined information represented by the license plate (hereinafter referred to as license plate information); a passage detection unit that, if at least a part of the license plate information could not be acquired as a result of the first image recognition (the vehicle in question is referred to as the tracked vehicle), performs image recognition on images of vehicles taken between the entrance toll gate and the exit toll gate of the toll road, compares them with the results of the first image recognition for the tracked vehicle, and detects the passage of the tracked vehicle, including its direction of travel; and a notification unit that, based on the time of passage and direction of travel of the tracked vehicle whose passage has been detected, notifies the exit toll gate where the tracked vehicle may arrive of the estimated time of arrival of the tracked vehicle.

Description

Technical Field

[0001] The present disclosure relates to a vehicle tracking system and a vehicle tracking method.

Background Art

[0002] In Patent Document 1, when, as a result of extracting predetermined license plate information described on a license plate from a camera image of a target vehicle, a part of the information could not be read, information representing the vehicle body color and vehicle type of the target vehicle is extracted from the camera image, and a complementary device that complements the license plate information using the extracted information is disclosed. The complementary device described in Patent Document 1 determines that the vehicles confirmed at Location X and Location Y are the same vehicle when, for example, the license plate information recognized at a different Location Y, the vehicle body color, and the vehicle type match the information at Location X, when the extraction of the license plate information at Location X is incomplete. Then, the complementary device complements the license plate information of the target vehicle at Location X using the information at Location Y.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, in toll collection systems for toll roads, for example, at the entrance toll gate, the vehicle type classification for toll road fees is determined for each vehicle based on information such as the license plate, and at the exit toll gate, the toll collection process is carried out based on the vehicle type classification determined at the entrance toll gate. In this case, for example, if the license plate information cannot be read correctly at the entrance toll gate, the toll may be charged for the wrong vehicle type. In such cases, for example, after exiting the toll gate, a so-called post-processing step of settling the toll is required, which can lead to a decrease in operational efficiency compared to cases where this step is not necessary.

[0005] This disclosure was made to solve the above-mentioned problems and aims to provide a vehicle tracking system and a vehicle tracking method that can improve the efficiency of operations related to toll collection on toll roads. [Means for solving the problem]

[0006] To solve the above problems, the vehicle tracking system according to this disclosure includes: an information acquisition unit that performs image recognition on an image of a vehicle passing through an entrance toll gate of a toll road (hereinafter, the image recognition of an image of a vehicle passing through the entrance toll gate is referred to as first image recognition) and acquires at least predetermined information represented by the license plate (hereinafter referred to as license plate information); a passage detection unit that, if at least a part of the license plate information could not be acquired as a result of the first image recognition (hereinafter referred to as the vehicle being tracked), performs image recognition on an image of a vehicle taken between the entrance toll gate and the exit toll gate of the toll road, compares it with the result of the first image recognition for the vehicle being tracked, and detects the passage of the vehicle being tracked, including its direction of travel; and a notification unit that, based on the time of passage and the direction of travel of the vehicle being tracked, notifies the exit toll gate where the vehicle being tracked may arrive of the estimated time of arrival of the vehicle being tracked.

[0007] The vehicle tracking method relating to this disclosure includes the steps of: performing image recognition on an image of a vehicle passing through an entrance toll gate of a toll road (hereinafter, the image recognition of an image of a vehicle passing through the entrance toll gate is referred to as first image recognition) and obtaining at least predetermined information represented by the license plate (hereinafter referred to as license plate information); if, as a result of the first image recognition, at least a portion of the license plate information could not be obtained (hereinafter referred to as the vehicle being tracked), performing image recognition on an image of a vehicle taken between the entrance toll gate and the exit toll gate of the toll road, comparing it with the result of the first image recognition for the vehicle being tracked, and detecting the passage of the vehicle being tracked, including its direction of travel; and notifying the exit toll gate where the vehicle being tracked may arrive of the estimated arrival time of the vehicle being tracked, based on the time of passage and the direction of travel of the vehicle being tracked. [Effects of the Invention]

[0008] The vehicle tracking system and vehicle tracking method described herein can improve the efficiency of operations related to toll collection on toll roads. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram showing the general configuration of a fee collection system according to the embodiment of this disclosure. [Figure 2] This is a functional block diagram of an entrance toll booth according to an embodiment of this disclosure. [Figure 3] This figure shows an example of the configuration of target vehicle information according to the embodiment of this disclosure. [Figure 4] This figure shows an example of a license plate according to the embodiments of this disclosure. [Figure 5] This figure shows an example of an image of a license plate and its surrounding area according to an embodiment of the present disclosure. [Figure 6] This figure shows an example of an image of a license plate and its surrounding area with some information missing according to an embodiment of this disclosure. [Figure 7] This is a functional block diagram of the management server according to the embodiment of this disclosure. [Figure 8] It is a diagram showing a configuration example of tracking target information according to an embodiment of the present disclosure. [Figure 9] It is a diagram showing a configuration example of a tracked vehicle database according to an embodiment of the present disclosure. [Figure 10] It is a functional block diagram of an FF gantry according to an embodiment of the present disclosure. [Figure 11] It is a diagram showing a configuration example of passing information according to an embodiment of the present disclosure. [Figure 12] It is a functional block diagram of an exit toll gate according to an embodiment of the present disclosure. [Figure 13] It is a diagram showing a configuration example of arrival information according to an embodiment of the present disclosure. [Figure 14] It is a schematic diagram for explaining a toll collection system according to an embodiment of the present disclosure. [Figure 15] It is a schematic diagram for explaining a toll collection system according to an embodiment of the present disclosure. [Figure 16] It is a schematic diagram for explaining a toll collection system according to an embodiment of the present disclosure. [Figure 17] It is a flowchart showing an operation example of a toll collection system according to an embodiment of the present disclosure. [Figure 18] It is a flowchart showing an operation example of a toll collection system according to an embodiment of the present disclosure. [Figure 19] It is a schematic block diagram showing the configuration of a computer according to an embodiment of the present disclosure.

Mode for Carrying Out the Invention

[0010] Hereinafter, a vehicle tracking system and a vehicle tracking method according to an embodiment of the present disclosure will be described with reference to FIGS. 1 to 19. In each figure, the same or corresponding components are denoted by the same reference numerals, and the description thereof will be omitted as appropriate. <000009I> As shown in FIG. 1, a toll collection system 10 according to an embodiment of the present disclosure includes a management server 1, an entrance toll gate 2, an FF (Free-Flow) gantry 3, and an exit toll gate 4. Note that the toll collection system 10 is an example of a configuration of the "vehicle tracking system" of the present disclosure. Also, in FIG. 1, the toll collection system 10 includes one entrance toll gate 2, one FF gantry 3, and one exit toll gate 4 each, but there may be a plurality of entrance toll gates 2, FF gantries 3, and exit toll gates 4. Also, in FIG. 1, the toll collection system 10 includes one management server 1, but for example, it may include a plurality of management servers 1 having a hierarchical structure. The entrance toll gate 2 and the exit toll gate 4 are toll gates at the entrance and exit installed on the same toll road (not shown) such as a highway. The entrance toll gate 2 and the exit toll gate 4 perform toll collection processing by performing predetermined wireless communication with, for example, an in-vehicle device mounted on a vehicle and roadside wireless devices 25 and 43 installed at the entrance toll gate 2 and the exit toll gate 4. The FF gantry 3 is a gate-shaped structure installed between the entrance toll gate 2 and the exit toll gate 4 of the toll road where the entrance toll gate 2 and the exit toll gate 4 are installed, for example, straddling the toll road. A vehicle traveling on the toll road can pass through the FF gantry 3 without reducing its speed.

[0012] The entrance toll gate 2 is a toll gate corresponding to two lanes, lane A and lane B, and includes a device group 2A for lane A and a device group 2B for lane B. The device groups 2A and 2B include a license plate recognition device 21, a vehicle type discrimination device 22, a monitoring camera 23, an image recognition device 24, a roadside wireless device 25, a control device 26, and a communication device 27. Note that the entrance toll gate 2 includes devices such as a vehicle detector (not shown) on the roadside or in the lane, an axle information detection device, a display device, a barrier, and a ticket issuing device for issuing a toll ticket to a vehicle not equipped with an in-vehicle device.

[0013] The NP recognition device 21 includes a camera for capturing license plate images, a recognition device that recognizes information contained in the license plate from the image captured by the camera, etc. When an approaching vehicle is detected by a vehicle detector (not shown), the device captures an image of the approaching vehicle and reads the license plate information of the approaching vehicle (hereinafter also referred to as NP information) from the captured image. The images captured by the NP recognition device 21 are stored, for example, in the management server 1. The vehicle detector (not shown) is installed, for example, on the roadside of the lane at the entrance toll booth 2, and scans a laser beam in a vertical plane parallel to the width direction of the lane to detect the presence or absence of an approaching vehicle. The NP information includes, for example, information representing the predetermined contents of the license plate, information representing differences in size, and information representing differences in color between the background of the plate and the code representing the contents. Examples of NP information will be described later.

[0014] The vehicle type discrimination device 22 includes an axle information detection device and the like for detecting axle information of an approaching vehicle, and determines the vehicle type of the approaching vehicle (vehicle type classification, which is the classification of toll road fees by vehicle type) based on NP information and the number of axles. In this embodiment, the result of vehicle type discrimination by the vehicle type discrimination device 22 is also referred to as the vehicle type discrimination result.

[0015] The surveillance camera 23 is equipped with one or more cameras, lighting devices, etc., and captures images of the vehicle body, including an overall image of the vehicle body, a front image, a rear image, and an image including the license plate and the surrounding area, as the vehicle passes through the entrance toll gate 2.

[0016] The image recognition device 24 performs image recognition on NP images captured by the NP recognition device 21, as well as on NP images and vehicle body images captured by the surveillance camera 23. It extracts features from the NP images and vehicle body images, and identifies (or classifies) the vehicle name (a unique name devised by the manufacturer). There are no limitations on the image recognition method used by the image recognition device 24. For example, multiple feature points can be extracted as features from NP images and front images, similar to the feature point identification method used for fingerprint recognition. The image recognition devices 32 and 42 of the FF gantry 3 and exit toll booth 4, described later, use the feature points extracted by the image recognition device 24 as a template for comparison and determine the degree of image matching (or calculate the matching rate). Vehicle name identification can be performed, for example, using a trained machine learning model that has been trained using vehicle body images labeled with vehicle names as training data, and outputs vehicle name classification results (e.g., the matching rate for each vehicle name) when a vehicle body image is input. Furthermore, the features extracted from the vehicle image may be limited to the location and shape of scratches and dents on the vehicle body, the shape of the tire wheels, etc. There are no limitations on the type or number of features. In addition, image recognition by the image recognition device 24 may be performed only when the NP recognition device 21 was unable to acquire at least a portion of the predetermined NP information.

[0017] The roadside wireless device 25 communicates wirelessly with an on-board unit installed in an approaching vehicle to send and receive various information necessary for toll collection.

[0018] The control device 26 controls, for example, each of the devices (including cameras) 21 to 25 and device 27.

[0019] The communication device 27 performs predetermined communications, such as wired or wireless, with the management server 1, and transmits captured images, or, if the NP recognition device 21 is unable to acquire at least a portion of the predetermined NP information, it transmits target vehicle information (information on the target vehicle to be tracked), which will be described later.

[0020] The FF Gantry 3 is a system that accommodates two lanes, Lane A and Lane B, and comprises a device group 3A for Lane A and a device group 3B for Lane B. Device groups 3A and 3B each include an FF camera 31, an image recognition device 32, a control device 33, and a communication device 34. The FF camera 31 includes one or more cameras, lighting devices, etc., and captures images of vehicles passing through the FF Gantry 3, including an overall image of the vehicle body, a front image, a rear image, and an image including the license plate and its surrounding area.

[0021] The image recognition device 32 performs image recognition on the image captured by the FF camera 31 in the same manner as the image recognition device 24, and compares it with the results of image recognition by the image recognition device 24. If, for example, a matching rate exceeding a predetermined threshold is obtained, the image recognition device 32 detects that the vehicle is one that should be tracked at the entrance toll gate 2. Note that the image recognition by the image recognition device 32 and the comparison of the results of image recognition by the image recognition device 32 with the results of image recognition by the image recognition device 24 may be performed integrally (for example, the image recognition process may include the result comparison process).

[0022] The control device 33 controls, for example, each device (including cameras) 31-32 and device 34. The communication device 34 communicates with the management server 1 via wired, wireless, or other predetermined means, transmitting captured images, receiving tracking target information (described later), and transmitting passage information when a target vehicle is detected.

[0023] Exit tollbooth 4 is a tollbooth that accommodates two lanes, Lane A and Lane B, and is equipped with a device group 4A for Lane A and a device group 4B for Lane B. Device groups 4A and 4B are equipped with a surveillance camera 41, an image recognition device 42, a roadside wireless device 43, a control device 44, and a communication device 45. Exit tollbooth 4 is also equipped with devices on the roadside or in the lanes, such as vehicle detectors, display devices, barriers, and toll collection devices that collect tolls using toll tickets from vehicles that do not have on-board devices.

[0024] The surveillance camera 41 is equipped with one or more cameras, lighting devices, etc., and captures images of the vehicle body of a vehicle attempting to pass through the exit toll gate 4, including an overall image of the vehicle body, a front image, a rear image, and an image including the license plate and the surrounding area.

[0025] The image recognition device 42 performs image recognition on the image captured by the surveillance camera 41 in the same manner as the image recognition device 24, and compares it with the results of the image recognition by the image recognition device 24. If, for example, a matching rate exceeding a predetermined threshold is obtained, the image recognition device 42 detects that the vehicle is one that should be tracked at the entrance toll gate 2. Note that the comparison between the image recognition by the image recognition device 42 and the image recognition by the image recognition device 24 may be performed integrally.

[0026] The roadside wireless device 43 communicates wirelessly with an on-board unit installed in the vehicle to send and receive various information necessary for toll collection.

[0027] The control device 44 controls, for example, each device (including cameras) 41 to 43 and device 45. The communication device 45 communicates with the management server 1 via wired, wireless, or other predetermined means, transmitting captured images, receiving tracking target information (described later), and transmitting arrival information when a target vehicle is detected.

[0028] Next, referring to the functional blocks shown in Figure 2, the functions realized by the entrance tollbooth 2 using the devices (including cameras) 21-27 shown in Figure 1 will be described. The entrance tollbooth 2 comprises, as a functional block, an image acquisition unit 210, an image recognition unit 220, a target vehicle information transmission unit 230, and a vehicle-to-infrastructure communication unit 240. The image acquisition unit 210 includes an NP image acquisition unit 211 and a vehicle body image acquisition unit 212. The image recognition unit 220 includes an NP image recognition unit 221 and a vehicle body image recognition unit 222. Note that the image recognition unit 220 is an example of the configuration of the "information acquisition unit" related to this disclosure.

[0029] The NP image acquisition unit 211 uses, for example, a license plate imaging camera provided in the NP recognition device 21 to image vehicles passing through the toll gate 2 at the entrance to the toll road. The vehicle body image acquisition unit 212 uses a surveillance camera 23 to image vehicles passing through the toll gate 2 at the entrance to the toll road.

[0030] The NP image recognition unit 221 is a function implemented using the NP recognition device 21, etc. It performs image recognition on images captured by the NP image capturing unit 211, images captured by the vehicle body image capturing unit 212, or both, and acquires predetermined information (NP information) represented by the license plate of a passing vehicle. Figure 4 shows an example of a license plate that the NP image recognition unit 221 recognizes. The license plate 50 shown in Figure 4 displays characters 51 indicating the transport bureau or the vehicle inspection and registration business office, a classification number 52 according to the type of vehicle, characters 53 indicating the use such as private, rental, or commercial, a serial designation number 54, etc. In addition, the difference in size of the license plate 50 indicates differences in gross vehicle weight, maximum load capacity, and passenger capacity. Furthermore, the difference in the background color of the plate and the color of the characters etc. indicates the difference between private use etc. and commercial use. In this embodiment, the NP information includes at least one of the following: information representing the predetermined contents of the license plate 50 (character 51, classification number 52, character 53, and serial designation number 54); information representing differences in size; and information representing differences in color between the background of the plate and the symbols representing the contents (character 51, classification number 52, character 53, and serial designation number 54).

[0031] The vehicle image recognition unit 222 is a function implemented using an image recognition device 24, etc. It performs image recognition on an image captured by the NP image capturing unit 211, an image captured by the vehicle image capturing unit 212, or both, and extracts predetermined feature quantities from the NP image 56, which includes the license plate 50 and a predetermined surrounding area 55 of the license plate 50, as shown in Figure 5. The vehicle image recognition unit 222 also performs image recognition on an image captured by the NP image capturing unit 211, an image captured by the vehicle image capturing unit 212, or both, and extracts feature quantities from the vehicle image or classifies the vehicle name. For example, if there is an area 57 that cannot be captured due to folds, dirt, obstruction from the license plate 50, etc., as shown by the shading in Figure 6, the vehicle image recognition unit 222 extracts predetermined feature quantities from the NP image 56, including the area 57 (i.e., the extracted feature quantities represent the features of the area 57). Furthermore, the image recognition performed by the vehicle body image recognition unit 222 can be configured to be performed only when the NP image recognition unit 221 was unable to acquire at least a portion of the NP information.

[0032] In this embodiment, the image recognition unit 220 performs image recognition on an image of a vehicle passing through the entrance toll gate 2 of the toll road (hereinafter, image recognition of an image of a vehicle passing through the entrance toll gate 2 is also referred to as first image recognition), and acquires at least NP information, which is predetermined information represented by the license plate 50. That is, the image recognition unit 220 may acquire only NP information (at least a part of the NP information described above), or it may further acquire predetermined feature quantities of the NP image 56 including the license plate 50 and a predetermined surrounding area 55 of the license plate 50, or acquire vehicle name classification results, etc. In other words, in first image recognition, NP information is acquired, or further, predetermined feature quantities of the NP image 56 are acquired, or vehicle name classification results, etc. are acquired.

[0033] If, as a result of the first image recognition by the image recognition unit 220, at least a portion of the NP information could not be obtained (hereinafter, the vehicle from which at least a portion of the NP information could not be obtained is also referred to as the tracked vehicle), the target vehicle information transmission unit 230 transmits, for example, the target vehicle information D1 shown in Figure 3 to the management server 1 using the communication device 27. The target vehicle information D1 shown in Figure 3 is information relating to the tracked vehicle and includes information representing the entrance toll gate ID (identification code) D11, vehicle identification number D12, entrance passage time D13, NP information recognition result D14, NP image features D15, vehicle body image features D16, and vehicle name classification result D17.

[0034] The entrance toll gate ID (D11) is the identification information of entrance toll gate 2 that detected the tracked vehicle (a vehicle for which at least part of the NP information could not be obtained). The vehicle identification number D12 is the identification information of the tracked vehicle, and can be, for example, a serial number determined for each entrance toll gate 2, an identification number of the in-vehicle device, a unique identification number assigned by the management server 1, etc. The entrance passage time D13 is the date and time when the tracked vehicle passed through entrance toll gate 2. The NP information recognition result D14 is the information that could be recognized from the NP information explained with reference to Figure 4. The NP image features D15 are predetermined features of the NP image 56 shown in Figure 5. The vehicle body image features D16 are one or more types of features extracted from the vehicle body image. The vehicle name classification result D17 is the result of classification by vehicle name, and includes information representing, for example, the vehicle name with the highest classification confidence, for example, multiple vehicle names with high confidence, for example, one or more vehicle names with confidence above a predetermined threshold. In this embodiment, the confidence (or accuracy) of the classification is also referred to as the agreement rate. Furthermore, the target vehicle information D1 may include NP images 56 and vehicle body images.

[0035] Furthermore, the vehicle-to-infrastructure communication unit 240 uses the roadside wireless device 25 to send and receive predetermined information with an on-board unit mounted on the vehicle.

[0036] Next, referring to the functional blocks shown in Figure 7, the functions implemented by the management server 1 will be described. The management server 1 comprises, as functional blocks, a target vehicle information receiving unit 110, an arrival information receiving unit 120, a passing information receiving unit 130, a destination calculation unit 140, a tracking target information transmission unit 150, a database update unit 160, and a tracking target vehicle database 170. Note that a configuration consisting of the tracking target information transmission unit 150, or a combination of the passing information receiving unit 130, the destination calculation unit 140, and the tracking target information transmission unit 150, is an example of the configuration of the "notification unit" related to this disclosure.

[0037] The target vehicle information receiving unit 110 receives the target vehicle information D1 transmitted by the target vehicle information transmitting unit 230. The arrival information receiving unit 120 receives arrival information D4 (Figure 13) from the arrival information transmitting unit 460 (Figure 12), which will be described later. The passage information receiving unit 130 receives passage information D3 (Figure 11) from the passage information transmitting unit 350 (Figure 10), which will be described later. When the passage of the tracked vehicle is detected in the FF gantry 3, the destination calculation unit 140 calculates one or more other FF gantry 3s and one or more exit toll gates 4 to which the tracked vehicle may arrive, based on the time of passage and direction of travel of the tracked vehicle, and calculates the estimated arrival time for each.

[0038] The tracking target information transmission unit 150 distributes, for example, the tracking target information D2 shown in Figure 8 to each FF gantry 3 and each exit toll gate 4. Figure 8 shows an example of the configuration of the tracking target information D2. The tracking target information D2 shown in Figure 8 includes information representing the vehicle identification number D21, NP information recognition result D22, NP image feature quantity D23, vehicle body image feature quantity D24, vehicle name classification result D25, data D26 representing the FF gantry ID and expected arrival time for each FF gantry 3, or "none", and data D27 representing the exit toll gate ID and expected arrival time for each exit toll gate 4, or "none". Vehicle identification number D21, NP information recognition result D22, NP image feature D23, vehicle body image feature D24, and vehicle name classification result D25 are information that corresponds to vehicle identification number D12, NP information recognition result D14, NP image feature D15, vehicle body image feature D16, and vehicle name classification result D17 included in the target vehicle information D1 explained with reference to Figure 3. Data D26 is information that indicates the expected time (if there is a possibility of arrival) or none (if there is no possibility of arrival) when the tracked vehicle with vehicle identification number D21 will arrive at FF gantry 3, which has an FF gantry ID. Data D27 is information that indicates the expected time (if there is a possibility of arrival) or none (if there is no possibility of arrival) when the tracked vehicle with vehicle identification number D21 will arrive at exit toll gate 4, which has an exit toll gate ID.

[0039] The database update unit 160 updates the contents of the tracked vehicle database 170 when it receives target vehicle information D1, passing information D3, or arrival information D4.

[0040] Figure 9 shows an example of the configuration of each record R100 when the tracked vehicle database 170 is configured as a table containing multiple records R100. The record R100 of the tracked vehicle database 170 shown in Figure 9 includes a field 1701 that stores the vehicle identification number, a field 1702 that stores the entrance toll gate ID, a field 1703 that stores the time of passage through the entrance, a field 1704 that stores the NP information recognition result, a field 1705 that stores NP image features, a field 1706 that stores the vehicle body image features, and a field 1707 that stores the vehicle name classification result. The data stored in fields 1701 to 1707 correspond to the vehicle identification number D12, entrance toll gate ID (D11), entrance time of passage D13, NP information recognition result D14, NP image features D15, vehicle body image features D16, and vehicle name classification result D17. Furthermore, record R100 shown in Figure 9 includes a pair of fields 1708 and 1709 (corresponding to data D26) and a pair of fields 1710 and 1711 (corresponding to data D27). Also, record R100 shown in Figure 9 includes a pair of fields 1712 for storing the FF gantry ID and 1713 for storing whether the tracked vehicle passed through the FF gantry 3 and, if so, the time of passage; a pair of fields 1714 for storing the exit toll gate ID and 1715 for storing whether the tracked vehicle arrived at the exit toll gate 5 and, if so, the time of arrival. Record R100 may also include fields for storing data (or data indicating the storage location) of the tracked vehicle's NP image or body image.

[0041] Next, referring to the functional blocks shown in Figure 10, the functions realized by the FF gantry 3 using the devices (including cameras) 31 to 34 shown in Figure 1 will be described. The FF gantry 3 includes, as functional blocks, a tracking target information receiving unit 310, an image capturing unit 320, an image recognition unit 330, a recognition result matching unit 340, and a passage information transmission unit 350. Note that a configuration combining the image capturing unit 320, the image recognition unit 330, and the recognition result matching unit 340, or a configuration combining the image recognition unit 330 and the recognition result matching unit 340, are examples of the configuration of the "passage detection unit" according to this disclosure.

[0042] The tracking target information receiving unit 310 receives the tracking target information D2 from the management server 1 using the communication device 34.

[0043] The image acquisition unit 320 uses the FF camera 31 to capture images of vehicles passing through the FF gantry 3.

[0044] The image recognition unit 330 is a function implemented using an image recognition device 32, etc., and performs image recognition of an image captured by the image acquisition unit 320 in the same manner as the image recognition unit 220 shown in Figure 2. The recognition result matching unit 340 then compares and matches the recognition result obtained from image recognition with the NP information recognition result D22, NP image feature quantity D23, vehicle body image feature quantity D24, and vehicle name classification result D25 (or at least a part of them) contained in the tracking target information D2 (or matches the feature quantities in the process of image recognition). For example, if a matching rate exceeding a predetermined threshold is obtained, the unit detects (determines) that the vehicle is identified by the vehicle identification number D21 contained in the tracking target information D2 (the tracking vehicle).

[0045] When a tracked vehicle is detected, the passing information transmission unit 350 uses the communication device 34 to transmit, for example, the passing information D3 shown in Figure 11 to the management server 1. The passing information D3 shown in Figure 11 includes information representing the FF gantry ID (D31), vehicle identification number D32, passing time D33, NP information recognition result D34, NP image matching rate D35, vehicle body image matching rate D36, vehicle name matching rate D37, and direction of travel D38.

[0046] The FF Gantry ID (D31) is the ID of the FF Gantry 3 in question. The Vehicle Identification Number D32 is the vehicle identification number D21 of the detected vehicle. The Passing Time D33 is the passing time of the vehicle. The NP Information Recognition Result D34 is the NP information recognized by the image recognition unit 330 from the captured image. The NP Image Match Rate D35 is the match rate between the feature quantities recognized by the image recognition unit 330 from the captured image (NP image 56) and the NP image feature quantities D23. This match rate (similarity) can be calculated, for example, by treating the feature quantities as vectors and comparing the distance and direction between the vectors. However, the method of calculation is not limited to this. The Vehicle Body Image Match Rate D36 is the match rate between the feature quantities recognized by the image recognition unit 330 from the captured image (vehicle body image) and the vehicle body image feature quantities D24. The Vehicle Name Match Rate D37 is, for example, the confidence level for the classification of the vehicle name with the highest confidence level included in the Vehicle Name Classification Result D25. The direction of travel D38 is the direction of travel of the vehicle, determined based on information such as the lane in which the FF camera 31 is installed. Furthermore, regarding multiple matching rates, it can be determined whether or not a vehicle is being pursued if all of them exceed a predetermined threshold, or if the average matching rate exceeds a predetermined threshold.

[0047] Next, referring to the functional blocks shown in Figure 12, the functions realized by the exit tollbooth 4 using the devices (including cameras) 41-45 shown in Figure 1 will be explained. The exit tollbooth 4 includes, as a functional block, a tracking target information receiving unit 410, an image capturing unit 420, an image recognition unit 430, a recognition result matching unit 440, a corresponding processing execution unit 450, an arrival information transmission unit 460, and a vehicle-to-infrastructure communication unit 470. Note that a configuration combining the image capturing unit 420, the image recognition unit 430, and the recognition result matching unit 440, or a configuration combining the image recognition unit 430 and the recognition result matching unit 440, are examples of the configuration of the "specific unit" in this disclosure.

[0048] The tracking target information receiving unit 410 receives tracking target information D2 from the management server 1 using the communication device 45. When the tracking target information receiving unit 410 receives tracking target information D2 from the management server 1, if a predicted arrival time for the tracked vehicle has been set for the exit toll gate 4, it may, for example, display the predicted arrival time on a predetermined monitor connected to the control device 44, or notify the predicted arrival time to a personal computer, tablet terminal, smartphone, or other terminal used by the monitoring officer. This allows preparations to be made before the arrival of the tracked vehicle, even if some manual processing is required for the tracked vehicle at the exit toll gate 4.

[0049] The image acquisition unit 420 uses the surveillance camera 41 to capture images of vehicles attempting to exit the toll gate 4.

[0050] The image recognition unit 430 is a function implemented using an image recognition device 42, etc., and performs image recognition of an image captured by the image acquisition unit 420 in the same manner as the image recognition unit 220 shown in Figure 2. The recognition result matching unit 440 then compares and matches the recognition result obtained from image recognition with the NP information recognition result D22, NP image features D23, vehicle body image features D24, and vehicle name classification result D25 (or at least some of them) contained in the tracking target information D2 (or matches the features during the image recognition process). For example, if a matching rate exceeding a predetermined threshold is obtained, the unit identifies (detects) that the vehicle is identified by the vehicle identification number D21 contained in the tracking target information D2 (the tracking vehicle).

[0051] If the response processing unit 450 identifies a vehicle as being pursued, it performs response processing, such as keeping the barrier bar blocked and displaying on the display device that a monitor will respond. The monitor checks, for example, the license plate information and vehicle type classification, corrects the license plate information and vehicle type classification of the vehicle as necessary, and performs billing processing, etc.

[0052] When a tracked vehicle is detected, the passing information transmission unit 450 uses the communication device 45 to transmit arrival information D4, for example, as shown in Figure 13, to the management server 1. The arrival information D4 shown in Figure 13 includes information representing the exit toll gate ID (D41), vehicle identification number D42, arrival time D43, NP information recognition result D44, NP image matching rate D45, vehicle body image matching rate D46, and vehicle name matching rate D47.

[0053] The exit toll gate ID (D41) is the ID of the exit toll gate 4. The vehicle identification number D42 is the vehicle identification number D21 of the detected vehicle. The arrival time D43 is the arrival time of the vehicle. The NP information recognition result D44 is the NP information recognized by the image recognition unit 430 from the captured image. The NP image matching rate D45 is the matching rate between the feature quantity recognized by the image recognition unit 430 from the captured image (NP image 56) and the NP image feature quantity D23. The vehicle body image matching rate D46 is the matching rate between the feature quantity recognized by the image recognition unit 430 from the captured image (vehicle body image) and the vehicle body image feature quantity D24. The vehicle name matching rate D47 is, for example, the confidence level for the classification of the vehicle name with the highest confidence level included in the vehicle name classification result D25.

[0054] Furthermore, the vehicle-to-infrastructure communication unit 470 uses the roadside wireless device 43 to send and receive predetermined information with an on-board device mounted on the vehicle.

[0055] Next, with reference to Figures 14 to 16, an overview of examples of information acquired by each device in the toll collection system 10 will be provided. For example, the threshold for the matching rate is assumed to be 70%. Figure 14 shows examples of information acquired at the entrance toll gate 2, FF gantry 3, and exit toll gate 4 located on the toll road R. As shown in Figure 14, at the entrance toll gate 2, the NP recognition device 21 acquires the NP image and NP information. The vehicle type discrimination device 22 acquires the vehicle type discrimination result. The surveillance camera 23 acquires the vehicle body image. The image recognition device 24 acquires the image recognition result. At the FF gantry 3, the FF camera 31 acquires captured images (NP images and vehicle body images). The image recognition device 32 acquires the image recognition and matching results. At the exit toll gate 4, the surveillance camera 41 acquires captured images (NP images and vehicle body images). The image recognition device 42 acquires the image recognition and matching results.

[0056] Figure 15 shows a specific example of information obtained when some of the NP information is not acquired by the NP recognition device 21. In the example shown in Figure 15, for vehicle 60B, which has the vehicle name "B" and is equipped with the onboard unit 61B, some of the NP information is missing, and the vehicle type identification result is unknown (cannot be identified). In this case, at the entrance toll gate 2, as a result of image recognition, for example, the matching rate for the vehicle name "B" is calculated to be 95%, and a template for matching the NP image is created. At the FF gantry 3, as a result of image recognition, for example, the matching rate for the vehicle name "B" is 90%, and the matching rate for the NP (image) is 80%, and the vehicle is determined to be vehicle 60B, which is the tracking vehicle. At the exit toll gate 4, as a result of image recognition, for example, the matching rate for the vehicle name "B" is 95%, and the matching rate for the NP (image) is 95%, and the vehicle is determined to be vehicle 60B, which is the tracking vehicle.

[0057] Figure 16 shows a specific example of the information acquired when all NP information is acquired by the NP recognition device 21. In the example shown in Figure 16, for vehicle 60A, which has the vehicle name "A" and is equipped with the onboard unit 61A, all NP information is acquired, and the vehicle type is determined to be "regular car". In this case, the FF gantry 3 determines, as an image recognition result, that the matching rate for vehicle name "B" is 10% and the matching rate for NP (image) is 17%, and that the vehicle in question is not vehicle 60B, which is the vehicle being tracked. Also, at the exit toll gate 4, as an image recognition result, it determines, for example, that the matching rate for vehicle name "B" is 5% and the matching rate for NP (image) is 17%, and that the vehicle in question is not vehicle 60B, which is the vehicle being tracked.

[0058] Next, the processing flow in the toll collection system 10 will be explained with reference to Figures 17 and 18. In Figures 17 and 18, the tracked vehicle database 170 is abbreviated as "DB". It is assumed that before the processing shown in Figure 17 begins, no tracked vehicles are registered in the tracked vehicle database 170. The first tracked vehicle is assumed to travel from the entrance toll gate 2, through the FF gantry 3, to the exit toll gate 4, as shown in Figures 17 and 18. At the entrance toll gate 2, the FF gantry 3, and the exit toll gate 4, for example, the time-based waiting operations and the operation of each functional block are controlled by control devices 26, 33, and 44.

[0059] In Figure 17, when a vehicle approaches entrance tollbooth 2 (step S1), an NP image and a vehicle body image are taken (step S2). Next, image recognition is performed on the NP image (step S3). If all NP information is acquired during the image recognition in step S3 (step S4: pass), no further image recognition is performed on the vehicle body image for that vehicle.

[0060] On the other hand, if at least a portion of the NP information cannot be obtained (Step S4: Failure), image recognition is performed on the vehicle body image (Step S5), and based on the recognition result, the target vehicle information D1 is transmitted from the entrance toll gate 2 to the management server 1 (Step S6).

[0061] On the management server 1, a new record R100 based on the target vehicle information D1 is added to the tracked vehicle database 170 (step S7). Also, the possible exit toll gate 4 and FF gantry 3 are calculated (step S8). Tracked information D2 is distributed to FF gantry 3 and exit toll gate 4 (steps S9 and S10). In this case, the tracked information D2 includes information representing the estimated arrival times for FF gantry 3 and FF gantry 3 at exit toll gate 4.

[0062] FF Gantry 3 and Exit Toll Gate 4 wait until the expected time (or a predetermined time before the expected time) (repeating step S11: NO or step S12: NO). If the expected time arrives at FF Gantry 3 (step S11: YES), each time a vehicle arrives (each time step S13: YES), image capture (step S14), image recognition (step S15), and verification of the image recognition results (step S16) are repeatedly performed until the vehicle is determined to be a vehicle being pursued (until step S17: YES).

[0063] If the vehicle is determined to be a vehicle to be tracked (Step S17: YES), passage information D3 is sent from FF gantry 3 to management server 1 (Step S18). Passage information D3 includes information indicating the time of passage and direction of travel. When management server 1 receives passage information D3 (Step S19), it calculates the possible exit toll gate 4 and FF gantry 3 that the vehicle may pass (Step S20). The tracked vehicle database 170 is then updated (Step S21). If the information is updated, the tracked information D2 is again distributed to FF gantry 3 and exit toll gate 4 (Steps S23 and S22). In this case, the tracked information D2 includes information that FF gantry 3 will not be arriving, and the exit toll gate 4 includes a recalculated estimated arrival time.

[0064] When tracking target information D2 is received (step S22), the waiting time for the predicted time is changed at the exit toll gate 4 (step S24 (step S12)). As shown in Figure 18, when the predicted time arrives (step S24 (step S12): YES), each time a vehicle arrives (each time step S25: YES), image capture (step S26), image recognition (step S27), and matching of the image recognition results (step S28) are repeatedly performed until the vehicle is determined to be the vehicle being tracked (step S29: YES).

[0065] If the vehicle is determined to be a vehicle to be tracked (Step S29: YES), the corresponding process is executed (Step S30), and arrival information D4 is sent from the exit toll gate 4 to the management server 1 (Step S31). When the management server 1 receives arrival information D4, the tracked vehicle database 170 is updated (Step S32). Subsequently, tracked vehicle information D2, which includes information that the vehicle is no longer likely to arrive, is distributed to the exit toll gate 4 and the FF gantry 3 (step not shown).

[0066] (Effects and Benefits) The toll collection system 10 (vehicle tracking system) with the above configuration includes an information acquisition unit (image recognition unit 220), a passage detection unit (configuration combining an image acquisition unit 320, an image recognition unit 330, and a recognition result matching unit 340, or a configuration combining an image recognition unit 330 and a recognition result matching unit 340), and a notification unit (a tracking target information transmission unit 150, or a configuration combining a passage information receiving unit 130, a transmission destination calculation unit 140, and a tracking target information transmission unit 150). The information acquisition unit performs image recognition (first image recognition) on an image of a vehicle passing through the entrance toll gate 2 of the toll road and acquires at least predetermined information represented by the license plate (license plate information). If, as a result of the first image recognition, at least a portion of the license plate information could not be acquired (the vehicle being tracked), the passage detection unit performs image recognition on an image of a vehicle taken between the entrance toll gate 2 and the exit toll gate 4 of the toll road, compares it with the result of the first image recognition for the tracked vehicle, and detects the passage of the tracked vehicle, including its direction of travel. The notification unit then notifies the toll gates where the tracked vehicle is likely to arrive of the estimated arrival time of the tracked vehicle, based on the time of passage and direction of travel of the tracked vehicle whose passage has been detected. According to this embodiment and the embodiments described below, the estimated arrival time of the tracked vehicle can be notified to the toll gates, thereby improving the efficiency of toll collection operations on toll roads compared to when no notification is made. Furthermore, according to this embodiment, there is no need to use vehicle-to-infrastructure communication to detect the tracked vehicle.

[0067] Furthermore, the toll collection system 10 (vehicle tracking system) of this embodiment includes an identification unit (configuration combining an image acquisition unit 420, an image recognition unit 430, and a recognition result comparison unit 440, or a configuration combining an image recognition unit 430 and a recognition result comparison unit 440) that performs image recognition on an image of a vehicle attempting to exit the toll gate 4, compares it with the result of a first image recognition for the vehicle being tracked, and identifies the vehicle being tracked. With this configuration, there is no need to use vehicle-to-infrastructure communication to identify the vehicle being tracked at the toll gate 4.

[0068] Furthermore, in this embodiment, the toll collection system 10 (vehicle tracking system) includes predetermined feature quantities in the NP image 56, which includes the license plate and a predetermined surrounding area of ​​the license plate, as the result of the first image recognition. With this configuration, it is possible to detect or identify the tracked vehicle based on the NP image 56 of the license plate and its surrounding area.

[0069] Furthermore, in this embodiment, the toll collection system 10 (vehicle tracking system) includes the vehicle name classification result in the first image recognition. With this configuration, since the first image recognition result includes the vehicle name classification result, it is easy to notify the exit toll gate 4 of the vehicle name classification result in addition to the estimated arrival time.

[0070] (Other embodiments) Although embodiments of this disclosure have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and may include design changes and the like that do not depart from the gist of this disclosure.

[0071] <Computer Configuration> Figure 19 is a schematic block diagram showing the configuration of a computer according to the present disclosure. The computer 90 includes a processor 91, main memory 92, storage 93, and an interface 94. The aforementioned devices, including the management server 1, NP recognition device 21, vehicle type discrimination device 22, image recognition device 24, control device 26, image recognition device 32, control device 33, image recognition device 42, and control device 44, are implemented in the computer 90. The operation of each of the aforementioned processing units is stored in storage 93 in the form of a program. The processor 91 reads the program from storage 93, loads it into main memory 92, and executes the above processing according to the program. The processor 91 also allocates memory areas in main memory 92 corresponding to each of the aforementioned storage units according to the program.

[0072] The program may be for implementing some of the functions that the computer 90 is to perform. For example, the program may perform functions in combination with other programs already stored in storage, or in combination with other programs implemented in other devices. In other embodiments, the computer may include a custom LSI (Large Scale Integrated Circuit) such as a PLD (Programmable Logic Device) in addition to, or instead of, the above configuration. Examples of PLDs include PAL (Programmable Array Logic), GAL (Generic Array Logic), CPLD (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array), etc. In this case, some or all of the functions implemented by the processor may be implemented by the integrated circuit.

[0073] Examples of storage 93 include HDDs (Hard Disk Drives), SSDs (Solid State Drives), magnetic disks, magneto-optical disks, CD-ROMs (Compact Disc Read Only Memory), DVD-ROMs (Digital Versatile Disc Read Only Memory), and semiconductor memory. Storage 93 may be an internal medium directly connected to the bus of the computer 90, or an external medium connected to the computer 90 via an interface 94 or a communication line. Furthermore, if this program is distributed to the computer 90 via a communication line, the computer 90 that receives the program may expand it into main memory 92 and execute the above processing. In at least one embodiment, storage 93 is a tangible storage medium that is not temporary.

[0074] <Note> The toll collection system 10 (vehicle tracking system) described in the embodiment of this disclosure is understood, for example, as follows:

[0075] (1) The toll collection system 10 (vehicle tracking system) according to the first embodiment includes: an information acquisition unit that performs image recognition on an image of a vehicle passing through an entrance toll gate of a toll road (hereinafter, the image recognition of an image of a vehicle passing through the entrance toll gate is referred to as first image recognition) and acquires at least predetermined information represented by the license plate (hereinafter referred to as license plate information); a passage detection unit that, if at least a part of the license plate information could not be acquired as a result of the first image recognition (hereinafter referred to as the vehicle being tracked), performs image recognition on an image of a vehicle taken between the entrance toll gate and the exit toll gate of the toll road, compares it with the result of the first image recognition for the vehicle being tracked, and detects the passage of the vehicle being tracked, including its direction of travel; and a notification unit that, based on the time of passage and the direction of travel of the vehicle being tracked, notifies the exit toll gate where the vehicle being tracked may arrive of the estimated time of arrival of the vehicle being tracked. According to this embodiment and each of the following embodiments, the efficiency of operations related to toll collection on toll roads can be improved.

[0076] (2) The toll collection system 10 (vehicle tracking system) according to the second embodiment is the toll collection system 10 (vehicle tracking system) of (1), further comprising an identification unit that performs image recognition on an image of a vehicle about to exit the exit toll gate, compares it with the result of the first image recognition for the tracked vehicle, and identifies the tracked vehicle.

[0077] (3) The toll collection system 10 (vehicle tracking system) according to the third embodiment is the toll collection system 10 (vehicle tracking system) of (1) or (2), wherein the result of the first image recognition includes predetermined feature quantities of an image including a license plate and a predetermined surrounding area of ​​the license plate.

[0078] (4) The toll collection system 10 (vehicle tracking system) according to the fourth embodiment is the toll collection system 10 (vehicle tracking system) of (1) to (3), wherein the result of the first image recognition includes the result of classifying the vehicle name.

[0079] (5) The toll collection system 10 (vehicle tracking system) according to the fifth embodiment is the toll collection system 10 (vehicle tracking system) of (1) to (4), wherein the license plate information includes at least one of the following: information representing the predetermined contents of the license plate, information representing the difference in size, and information representing the difference in color between the background of the plate and the code representing the contents. [Explanation of Symbols]

[0080] 10…Toll collection system (vehicle tracking system) 1…Management Server 2…Entrance toll booth 3...FF Gantry 4…Exit toll gate 130... Passing information receiving unit (notification unit) 140...Destination calculation unit (notification unit) 150... Tracking target information transmission unit (notification unit) 220...Image recognition unit (information acquisition unit) 320...Image acquisition unit (passage detection unit) 330...Image recognition unit (passage detection unit) 340... Recognition result verification unit (passage detection unit) 420...Image acquisition unit (specific unit) 430...Image recognition unit (specification unit) 440... Recognition result verification unit (identification unit) R... Toll road

Claims

1. An information acquisition unit that performs image recognition on an image of a vehicle passing through a toll gate at the entrance of a toll road (hereinafter, the image recognition of an image of a vehicle passing through the toll gate is referred to as first image recognition) and acquires at least predetermined information represented by the license plate (hereinafter referred to as license plate information), If, as a result of the first image recognition, at least a portion of the license plate information could not be obtained (hereinafter referred to as the tracked vehicle), a passage detection unit performs image recognition on images of the vehicle taken between the entrance toll gate and the exit toll gate of the toll road, compares these images with the results of the first image recognition for the tracked vehicle, and detects the passage of the tracked vehicle, including its direction of travel. A notification unit that, based on the time of passage and direction of travel of the tracked vehicle whose passage has been detected, notifies the exit toll gate where the tracked vehicle may arrive of the estimated time of arrival of the tracked vehicle, A vehicle tracking system equipped with the following features.

2. An identification unit performs image recognition on an image of a vehicle attempting to exit the aforementioned toll gate, compares it with the result of the first image recognition for the tracked vehicle, and identifies the tracked vehicle. The vehicle tracking system according to claim 1, further comprising:

3. The result of the first image recognition includes predetermined feature quantities of an image that includes a license plate and a predetermined surrounding area of ​​the license plate. The vehicle tracking system according to claim 1 or 2.

4. The results of the first image recognition mentioned above include the classification results of the car name. The vehicle tracking system according to claim 1 or 2.

5. The aforementioned license plate information includes at least one of the following: information representing the predetermined contents of the license plate, information representing differences in size, and information representing the difference in color between the background of the plate and the code representing the aforementioned contents. The vehicle tracking system according to claim 1 or 2.

6. The process involves performing image recognition on an image of a vehicle passing through a toll gate at the entrance to a toll road (hereinafter, the image recognition of an image of a vehicle passing through the toll gate is referred to as the first image recognition), and obtaining at least predetermined information represented by the license plate (hereinafter, referred to as license plate information), If, as a result of the first image recognition, at least a portion of the license plate information could not be obtained (hereinafter referred to as the tracked vehicle), the image recognition of the vehicle taken between the entrance toll gate and the exit toll gate of the toll road is performed, and the results are compared with the results of the first image recognition for the tracked vehicle to detect the passage of the tracked vehicle, including its direction of travel. The steps include notifying the toll gate at the exit where the tracked vehicle may arrive of the estimated arrival time of the tracked vehicle, based on the time of passage and direction of travel of the tracked vehicle whose passage has been detected, Vehicle tracking methods including...

Images