Vehicle confirmation support device and vehicle confirmation support method

Abstract

To provide a technology for supporting vehicle model discrimination by a collector.SOLUTION: A vehicle confirmation support device according to an embodiment includes: a storage unit that stores a captured image captured by a wide-angle-lens-equipped camera; a curvature revision unit that revises a curvature of the captured image to generate a plan view; a segmentation control unit that segments from the plan view three-face views of a front view, a side view, and a rear view of a vehicle; a vehicle feature information acquisition unit that acquires vehicle feature information including values obtained by estimating the number of shafts of the vehicle as well as a vehicle width, a vehicle height, and a vehicle length of the vehicle detected based on the three-face views; a vehicle model candidate determination unit that determines a vehicle model candidate of the vehicle based on the vehicle feature information; a display control unit that exerts control so as to display the vehicle feature information and the vehicle model candidate on an output device; and an input information acquisition unit that acquires corrected vehicle feature information obtained by correcting the values of the vehicle feature information. The vehicle model candidate determination unit determines the vehicle model candidate of the vehicle again based on the corrected vehicle feature information.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] An embodiment of the present invention relates to a vehicle confirmation assistance device and a vehicle confirmation assistance method. [Background technology]

[0002] Toll fees for toll roads are generally set according to the distance traveled and the type of vehicle. Toll fees can be roughly divided into two types: toll collection systems, where a toll ticket is collected at the entrance toll gate and the toll is paid in cash or other form at the exit toll gate based on the information on the toll ticket, and ETC systems, where toll fees are collected at the entrance toll gate and exit toll gate of the toll road by wireless communication with on-board devices installed in vehicles passing through the toll gate lanes.

[0003] Currently, the use of ETC systems has come to dominate due to an increase in vehicles that can use ETC systems (hereafter simply referred to as ETC vehicles). However, there are still quite a few vehicles that are not ETC compatible (hereafter simply referred to as non-ETC vehicles).

[0004] In addition, in the toll collection system for charging non-ETC vehicles, the vehicle type is identified at the entrance toll gate and a ticket is issued that records the entrance IC, the time of passage, and the identified vehicle type.Then, at the exit toll gate, the toll is determined based on the ticket received at the entrance toll gate, and the driver of the non-ETC vehicle pays in cash or other means.

[0005] For this reason, in the toll collection system for charging non-ETC vehicles, a booth was set up in each lane at the entrance and exit toll gates, and a toll collector in the booth would visually determine the type of vehicle passing through, issue a ticket at the entrance toll gate, and collect the toll at the exit toll gate.

[0006] In the future, the number of vehicles that can use the ETC system will continue to increase, but non-ETC vehicles that do not use the ETC system will continue to remain, so it is necessary to consider how to deal with non-ETC vehicles. However, since the number of vehicles that do not use the ETC system will decrease, it is not realistic from a cost perspective to install expensive vehicle type discrimination devices at the toll gates on the exit side.

[0007] Recently, in order to reduce the number of people required and to enable remote operations, there has been an increase in cases where automatic toll collection machines are installed in the lanes rather than having toll collectors stationed in booths set up at the lanes, and toll collectors are stationed in toll booth offices away from the lanes or in remote locations away from the toll booths, with the collectors determining the type of vehicle based solely on camera images. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2019-197314 A [Patent Document 2] JP 2019-082968 A [Patent Document 3] JP 2019-160003 A [Patent Document 4] JP 2002-170138 A [Patent Document 5] JP 2010-072931 A Summary of the Invention [Problem to be solved by the invention]

[0009] The task of determining the vehicle type by the attendant has problems in that there are multiple items to be checked for some vehicles, and it requires the attendant to acquire knowledge. In addition, there is a problem that it takes time to call up the required vehicle information during collection work.

[0010] This invention was made in light of the above circumstances, and its purpose is to provide a technology that supports vehicle type identification by vehicle collectors. In other words, it is to provide a technology that enables vehicle type identification accuracy to be improved by inferring candidates from vehicle images and accepting information input by the vehicle collector through visual inspection. Furthermore, it is an object of the present invention to provide a technology that enables required vehicle information to be called up from specified information, thereby making collection operations more efficient. [Means for solving the problem]

[0011] A vehicle confirmation assistance device according to an embodiment includes a memory unit that stores an image captured by a camera equipped with a wide-angle lens, a curvature correction unit that corrects the curvature of the captured image to generate a plan view, an extraction control unit that extracts three views of the front, side, and rear of the vehicle from the plan view, a vehicle feature information acquisition unit that acquires vehicle feature information including estimated values ​​of the number of axles, width, height, and length of the vehicle detected based on the three-view views, a vehicle model candidate determination unit that determines candidate vehicle models for the vehicle based on the vehicle feature information, a display control unit that controls the vehicle feature information and the candidate vehicle models to be displayed on an output device, and an input information acquisition unit that acquires modified vehicle feature information obtained by modifying the values ​​of the vehicle feature information, and the candidate vehicle model determination unit re-determines candidate vehicle models for the vehicle based on the modified vehicle feature information. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of a schematic configuration of a toll road fee processing system according to an embodiment. [Diagram 2] FIG. 2 is a top view and a side view showing a configuration example of lane equipment installed in a lane of an exit toll gate according to the embodiment. [Diagram 3] FIG. 3 is a block diagram showing an example of a schematic configuration of a lane device of the toll road fee processing system according to the embodiment. [Figure 4]FIG. 4 is a diagram illustrating an example of a schematic configuration of a vehicle confirmation support device according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a schematic configuration of a central vehicle confirmation support device according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a display image displayed on the display of the output device. [Figure 7] FIG. 7 is a flowchart showing an example of a procedure for generating a display screen to be displayed on the display of the output device. [Figure 8] FIG. 8 is a flowchart for explaining in more detail the tracing process of step ST105 shown in FIG. [Figure 9] FIG. 9 is a diagram showing an example of cutting out a vehicle license plate number image of a vehicle from a plan view. [Figure 10] FIG. 10 is a diagram showing an example of cutting out a front view of a vehicle from a plan view. [Figure 11] FIG. 11 is a diagram showing an example of cutting out a side view of a vehicle from a plan view. [Figure 12] FIG. 12 is a diagram showing an example of cutting out a rear view of a vehicle from a plan view. [Figure 13] FIG. 13 is a diagram showing an example of the processing operation of the vehicle confirmation support device when a collector determines the vehicle type. [Figure 14] FIG. 14 is a diagram showing an example of a display screen that is initially displayed on the display of an output device in a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the vehicle confirmation support device and the vehicle confirmation support method of the present embodiment will be described in detail with reference to the drawings. In the following embodiment, the parts with the same numbers perform the same operation, and the duplicated description will be omitted. For example, when there are a plurality of identical or similar elements, a common symbol may be used to describe each element without distinguishing between them, and a sub-number may be used in addition to the common symbol to describe each element with distinction between them.

[0014] [Embodiment] (composition) FIG. 1 is a block diagram showing an example of a schematic configuration of a toll road fee processing system according to an embodiment. Toll road toll processing systems are roughly divided into ETC systems that collect tolls at the entrance and exit toll gates of a toll road by wireless communication with on-board devices installed in vehicles passing through the lanes of the toll gates, and toll collection systems that receive a toll ticket at the entrance toll gate and pay the toll in cash or the like at the exit toll gate based on the information on the toll ticket. Furthermore, in this embodiment, a vehicle identification support system is provided to remotely support toll collectors located at the toll gate office, etc. in identifying the types of vehicles passing through.

[0015] As shown in Fig. 1, the vehicle confirmation support system for supporting vehicle type identification is composed of a central vehicle confirmation support device 6 provided in the center, vehicle confirmation support devices 3 provided at each toll gate (toll gate office), and lane equipment 4. Here, the vehicle confirmation support system is effective for identifying the vehicle types of passing vehicles at the exit toll gate of the toll collection system, but may also be used for identifying the vehicle types at the entrance toll gate.

[0016] The vehicle confirmation support device 3 is a device for supporting toll collectors in determining the vehicle type. The vehicle confirmation support device 3 controls the timing of photography by the wide-angle lens mounted camera 421 installed in the roadside device, to create a three-sided view of the vehicle from the vehicle number of the vehicle entering the lane and an image of the vehicle (including the axles), and further displays the vehicle's characteristic information to be used as a reference for determining the vehicle type. The vehicle confirmation support device 3 is installed in the toll gate office at the exit toll gate, and is connected to the central vehicle confirmation support device 6, and exchanges various information with the central vehicle confirmation support device 6.

[0017] Furthermore, the vehicle confirmation assistance device 3 is also connected to the toll collection device 2 and the lane server 5c, and exchanges various types of information with the toll collection device 2 and the lane server 5c.

[0018] 1 shows an example in which the central vehicle confirmation support device 6 is connected to one vehicle confirmation support device 3, but in reality, it is connected to multiple vehicle confirmation support devices 3 installed at each of the exit toll gates. The central vehicle confirmation support device 6 exchanges various information with the vehicle confirmation support devices 3.

[0019] The toll road toll processing system is composed of a central toll collection system 1 installed in the center, toll collection devices 2 installed in the toll gate offices of each toll gate, and lane equipment 4 installed in each lane on which vehicles travel at the toll gate.

[0020] The central toll collection system 1 is a server for managing toll collection for non-ETC vehicles. Therefore, the central toll collection system 1 receives and manages various data such as toll information collected from toll collection devices 2 installed at each toll gate.

[0021] The toll collection device 2 is a device for collecting tolls at the exit toll gate using a toll ticket from non-ETC vehicles. The toll collection device 2 is installed in the toll gate machine room of the entrance toll gate and the exit toll gate, and is connected to a device associated with the toll collection device 2 among the lane devices 4 installed in each lane through which non-ETC vehicles can pass. The toll collection device 2 also transmits information such as the paid toll to the central toll collection system 1.

[0022] The ETC system is composed of an ETC central unit 5a installed in the center, a toll gate server 5b installed at each toll gate (toll gate machine room), a lane server 5c that controls the lane equipment 4 of each lane, and lane equipment 4 installed in each lane on which vehicles at the toll gate travel.

[0023] The ETC central device 5a is connected to a tollgate server 5b installed at each tollgate, and is a device that performs toll calculation and charging processing for ETC vehicles. In addition, the ETC central device 5a exchanges various data with each tollgate server 5b.

[0024] The toll gate server 5b is a server for managing the lane servers 5c installed at the entrance toll gates and the exit toll gates, and therefore exchanges various data with the lane servers 5c.

[0025] The lane server 5c is provided in the toll booth machine rooms at the entrance and exit toll booths corresponding to lanes where ETC vehicles can pass, and is connected to the lane devices 4 installed in each lane where ETC vehicles can pass. The lane server 5c controls the lane devices 4 based on sensor data and the like transmitted from devices among the lane devices 4 associated with the lane server 5c.

[0026] The lane server 5c stores the exit information of the exit toll gate where it is installed, and also stores the toll table by section and by vehicle type distributed from the ETC central device 5a. The lane server 5c calculates the toll by referring to the toll table based on the vehicle-mounted device information, ETC card information, entrance information, exit information, and vehicle type information obtained by vehicle type discrimination. The lane server 5c also transmits the toll to the ETC central device 5a via the toll gate server 5b.

[0027] The lane server 5c transmits the calculated toll to the lane equipment 4.

[0028] The on-board unit installed in an ETC vehicle stores vehicle model information of the vehicle in which it is installed. Here, the vehicle model information is information that indicates the vehicle model, such as a compact car, a normal car, or a large car.

[0029] Next, the configuration of a toll gate will be described, taking the configuration of an exit toll gate as an example.

[0030] FIG. 2 is a top view and a side view showing an example of installation of lane equipment 4 installed in a lane of an exit toll gate according to the embodiment.

[0031] 2, the exit toll gate has lanes for vehicles separated by islands 40, and vehicles travel on these lanes from upstream to downstream. In this embodiment, the lanes are used as dual-use lanes that can be used by both ETC vehicles and non-ETC vehicles that pay tolls by cash or the like.

[0032] Further, on the island 40, in the order from upstream to downstream of the lane, there are provided a wide-angle lens mounted camera 421, a first vehicle detector S1, a second vehicle detector S2, an automatic toll collection machine 424, an intercom 422, a roadside indicator 423, a starting control device 425 and a starting control bar 4251, a third vehicle detector S4, and a lane monitoring camera 426. Note that the positions of the wide-angle lens mounted camera 421 and the first vehicle detector S1 may be reversed.

[0033] In addition, two antenna installation gates are provided in the two islands 40 from upstream to downstream across the lanes. A first antenna 411 is provided at the first antenna installation gate. A recovery antenna 413 is provided at the second antenna installation gate.

[0034] FIG. 3 is a block diagram showing an example of a schematic configuration of the lane device 4 of the toll road fee processing system according to the embodiment.

[0035] In FIG. 3, the lane equipment 4 includes devices related to toll collection, ETC, and vehicle type identification support.

[0036] As the lane equipment 4 related to the vehicle confirmation support device 3, a wide-angle lens mounted camera 421 (photographing unit) 421 is provided and connected to the vehicle confirmation support device 3 at the toll gate office.

[0037] The ETC-related lane equipment 4 includes, via an interface aggregation unit 401, a first antenna 411, a recovery antenna 413, a first vehicle detector S1, a second vehicle detector S2, a third vehicle detector S4, an intercom 422, a roadside display 423, a starting control device 425, and a lane monitoring camera 426.

[0038] An automatic toll collection machine 424 is provided as a roadside device related to toll collection, and is connected to a toll collection device 2 installed in a toll gate office. Furthermore, the toll collection device 2 is also connected to a starting controller 425.

[0039] First, the roadside equipment related to toll collection will be described. The automatic toll collection machine 424 is a device for cash settlement. When a driver of a non-ETC vehicle inserts a toll ticket received at the entrance toll gate into the automatic toll collection machine 424, the toll is calculated and displayed on the automatic toll collection machine 424. The driver of the non-ETC vehicle can execute settlement processing by depositing the displayed toll in cash (or by credit card, etc.).

[0040] Next, the roadside equipment related to the vehicle confirmation assistance device 3 will be described. The wide-angle lens mounted camera 421 operates as a vehicle confirmation support system and is therefore connected to the vehicle confirmation support device 3. The connection to the vehicle confirmation support device 3 may be either a direct connection or a connection via the interface aggregation unit 401.

[0041] 2, the wide-angle lens mounted camera 421 is installed in the vicinity of the first vehicle detector S1, closer to the installation position of the first vehicle detector S1 than to the installation position of the second vehicle detector S2. The wide-angle lens mounted camera 421 is installed within a range of a predetermined length (for example, 2 m) along the direction in which the lanes extend, based on the installation position of the first vehicle detector S1.

[0042] In addition, the height of the wide-angle lens mounted camera 421 from the road surface is desirably a height that allows the axle (tire) to be photographed directly in front (for example, about 30 cm to 100 cm from the road surface) while taking into consideration mud splashes from the vehicle, but the height from the road surface can be freely set depending on the characteristics of the wide-angle lens.

[0043] The wide-angle lens-mounted camera 421 is equipped with a wide-angle lens (lens with a short focal length) with a wider angle of view than a standard lens, and captures a predetermined range of the lane. The predetermined range of the lane to be captured includes a first point where the arrival of the vehicle is detected by the first vehicle detector S1 and a second point where the arrival of the vehicle is detected by the second vehicle detector S2. For example, the wide-angle lens-mounted camera 421 captures a predetermined range of about 15 m (about 30 m in total) in the upstream and downstream directions of the lane based on the installation position of the wide-angle lens-mounted camera 421. It is desirable for the wide-angle lens to be capable of capturing the entire side of various types of vehicles (such as light vehicles, ordinary vehicles, medium-sized vehicles, large vehicles, and extra-large vehicles) traveling on the lane, and is desirably a lens (fisheye lens) that can capture a range of about 180°, depending on the installation environment.

[0044] If the wide-angle lens mounted camera 421 is a camera that captures video, it starts capturing video based on a first capture instruction (instruction to start capturing video) from the vehicle confirmation assistance device 3, stops capturing video based on a second capture instruction (instruction to end capturing video), continues capturing video from the instruction to start capturing video to the instruction to end capturing video, and outputs the captured video.

[0045] Finally, we will explain the ETC-related roadside equipment. The interface aggregation unit 401 aggregates information from each unit and outputs it to the lane server 5c, and also outputs information from the lane server 5c to each unit.

[0046] The interface consolidation unit 401 is connected to the toll collection device 2 and the lane server 5c. The interface consolidation unit 401 outputs a vehicle detection signal ON or a vehicle detection signal OFF to the lane server 5c.

[0047] Here, the first vehicle detector S1 is a detector that detects a vehicle that reaches a position (first point) for detecting the entry of a vehicle into a lane. When the first vehicle detector S1 detects the entry of a vehicle, the first antenna 411 communicates with the vehicle's on-board device and receives various information transmitted from the on-board device, such as on-board device information, ETC card information, and entrance information. If the first antenna 411 cannot communicate with the on-board device, the vehicle will reach a second point downstream from the first point.

[0048] The interphone 422 is a device that allows the driver to talk to the toll collector when a problem occurs in the ETC system. Problems include malfunctions of the ETC system, or a non-ETC vehicle mistakenly entering an ETC-only lane.

[0049] The second vehicle detector S2 is a detector that detects a vehicle arriving at the second point. If communication between the first antenna 411 and the vehicle-mounted device cannot be established, the vehicle is stopped once and the collector is called. After the driver and the collector talk via the intercom 422 or the like, the collector operates the recovery antenna 413. Then, the recovery antenna 413 and the vehicle-mounted device start communication, and various information transmitted from the vehicle-mounted device is received.

[0050] The roadside display 423 displays the toll and vehicle type information output from the interface aggregation unit 401 to the driver of the vehicle.

[0051] The departure control device 425 controls the opening and closing of the departure control bar 4251 based on the departure control information from the interface aggregation unit 401. The departure control bar 4251 physically blocks the passage of vehicles (exiting a lane) when in a closed state, and allows the passage of vehicles when in an open state.

[0052] The third vehicle detector S4 is a detector that detects that the vehicle has reached the exit of the lane (a third point downstream from the second point). When the third vehicle detector S4 no longer detects the vehicle, it determines that the vehicle has left the lane and closes the start control bar 4251 for the next vehicle.

[0053] The lane monitoring camera 426 captures images all the time, and outputs captured images of vehicles entering or exiting a lane to the lane server 5c, for example.

[0054] Next, the configuration of the vehicle confirmation support device 3 installed in the toll gate office will be described.

[0055] FIG. 4 is a diagram showing an example of a schematic configuration of the vehicle confirmation support device 3 according to the embodiment. In addition, the vehicle confirmation assistance device 3 is a device that controls the shooting timing of the wide-angle lens mounted camera 421 installed in the roadside device, and determines and displays the vehicle number of a vehicle entering the lane and the image of the vehicle (including the axles) from these images (whether the number of axles and the vehicle length (whether they exceed the vehicle type-specific standard for determining the toll category).

[0056] The vehicle confirmation assistance device 3 is one or more computers, and includes a control unit 31, a program storage unit 32, a data storage unit 33, a communication interface 34, an input / output interface 35, an input device 36, and an output device 37.

[0057] The control unit 31 controls the vehicle confirmation assistance device 3. The control unit 31 includes a hardware processor such as a central processing unit (CPU). For example, the control unit 31 may be an integrated circuit capable of executing various programs.

[0058] The control unit 31 includes an image acquisition unit 3101, a curvature correction unit 3102, a vehicle recognition unit 3103, a vehicle position determination unit 3104, a vehicle length determination unit 3105, a vehicle number recognition unit 3106, an image cropping control unit 3107, axle number detection unit 3108, a lift-up detection unit 3109, a display control unit 3110, a communication control unit 3111, a vehicle information search unit 3112, a three-dimensional drawing search unit 3113, a vehicle feature information acquisition unit 3114, a vehicle model candidate determination unit 3115, and an input information acquisition unit 3116.

[0059] The image acquisition unit 3101 is an acquisition unit that acquires a captured image output by the wide-angle lens mounted camera 421 of the lane device 4. The image acquisition unit 3101 outputs the acquired captured image to the display control unit 3110 and the curvature correction unit 3102. In addition, the image acquisition unit 3101 may store the acquired captured image in the captured image storage unit 331.

[0060] The curvature correction unit 3102 is a correction unit that generates a plan view in which the curvature of the captured image is corrected. The curvature correction unit 3102 converts the captured image into a plan view in which the curvature is corrected, and outputs the plan view to the vehicle recognition unit 3103. In addition, the curvature correction unit 3102 may store the plan view in the captured image storage unit 331.

[0061] The vehicle recognition unit 3103 is a determination unit that determines whether or not to recognize a vehicle. Details of the determination of whether or not a vehicle has entered will be described later. The vehicle recognition unit 3103 outputs various signals to each unit according to the location where the vehicle has arrived.

[0062] The vehicle position determination unit 3104 is a determination unit that determines where the vehicle is located based on a plan view. For example, the vehicle position determination unit 3104 determines where the vehicle is located on the plan view by generating information required to display the vehicle position on a bird's-eye view based on the determination result.

[0063] The vehicle length determination unit 3105 is a determination unit that determines whether the vehicle is longer than a predetermined length. For example, when the front end of the vehicle reaches a predetermined position (predetermined line) on the plan view, and the rear end of the vehicle does not come to a line set on the edge side of the plan view, the vehicle length determination unit 3105 determines that the vehicle is longer than the predetermined length.

[0064] Conversely, if the rear end of the vehicle is downstream of the line set on the end side of the plan view when the front end of the vehicle reaches the specified line, the vehicle length determination unit 3105 determines that the vehicle is shorter than the specified length.

[0065] The vehicle number recognition unit 3106 cuts out the license plate arranged on the front of the vehicle in the acquired plan view, i.e., the vehicle number image. The vehicle number recognition unit 3106 may also recognize the vehicle number from the cut-out license plate.

[0066] The image cutout control unit 3107 is a control unit that cuts out one of the three views of the vehicle (front view, side view, and rear view) when the vehicle approaches an optimal position for cutting out one of the three views of the vehicle. Details of the cutting out method will be described later.

[0067] The axle number detection unit 3108 is a detection unit that detects the number of axles of the vehicle from the side view cut out by the image cut-out control unit 3107. For example, the axle number detection unit 3108 may detect tires using AI technology and detect the number of axles based on the detected tires.

[0068] The lift-up detection unit 3109 is a detection unit that detects whether the tires are lifted up. Here, lift-up refers to a situation in which a large vehicle such as a large truck or trailer, when empty or with a light load, runs with some of the axles raised and the tires not on the ground.

[0069] The vehicle information search unit 3112 is a search unit that searches for matching vehicle information from the vehicle information storage unit 332 (described later) of the data storage unit 33, and searches for vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include the vehicle model previously determined, vehicle characteristic information, vehicle number information, etc. The vehicle characteristic information and vehicle number information will be described later.

[0070] The three-planar view search unit 3113 is a search unit that acquires three-planar images stored in a three-planar image storage unit 333 (described later) based on the vehicle features, vehicle number, and the like, when necessary.

[0071] The display control unit 3110 is a control unit that controls the display image displayed on the display of the output device 37 each time the vehicle passes through a toll gate. Here, the display image includes a three-dimensional view, vehicle information, and the like.

[0072] The communication control unit 3111 is a control unit for transmitting and receiving various information to and from the central vehicle confirmation support device 6 through the communication interface 34. Specifically, the communication control unit 3111 performs control for transmitting the three-dimensional view, vehicle information, etc. acquired by each control unit to the central vehicle confirmation support device 6. Furthermore, when the communication control unit 3111 is unable to properly recognize the three-dimensional view, vehicle information, etc., it performs control for making a search request to the central vehicle confirmation support device 6 and receiving the three-dimensional view, vehicle information, etc. stored in the central vehicle confirmation support device 6.

[0073] The vehicle characteristic information acquisition unit 3114 is an acquisition unit that acquires vehicle characteristic information. The vehicle characteristic information includes the vehicle height, vehicle width, vehicle length, axles, etc. For example, the vehicle characteristic information acquisition unit 3114 operating as a vehicle height estimation unit and a vehicle width estimation unit estimates the vehicle height and vehicle width based on a captured image. Details of the method of estimating the vehicle height and vehicle width will be described later. Furthermore, the vehicle characteristic information acquisition unit 3114 operating as a vehicle length estimation unit estimates the vehicle length based on a captured image. Details of the method of estimating the vehicle length will be described later. Furthermore, the number of axles detected by the axle number detection unit 3108 may be used for the axles.

[0074] The vehicle model candidate determination unit 3115 determines vehicle model candidates based on the vehicle feature information and vehicle number information. The vehicle model candidate determination unit 3115 determines a plurality of vehicle model candidates based on the vehicle feature information and vehicle number information. Here, the method of determining the vehicle model may be determining a vehicle model that can be formed from specific numerical values ​​of the vehicle feature information and the vehicle number information. Note that the vehicle model candidate determination unit 3115 may determine the vehicle model candidates from either the vehicle feature information or the vehicle number information. Furthermore, the vehicle model candidate determination unit 3115 determines the vehicle model candidates again based on the corrected vehicle feature information and vehicle number information acquired by the input information acquisition unit 3116 described later.

[0075] The input information acquisition unit 3116 acquires the vehicle characteristic information and the vehicle number information corrected by the collector. For example, the collector corrects the vehicle characteristic information and the vehicle number information based on the drawing shown in the display image. The input information acquisition unit 3116 acquires the corrected information.

[0076] The data storage unit 33 is a storage unit for storing data acquired and generated in the course of various processes performed by the control unit 31 by executing a program. As a storage medium, for example, it is a storage that uses a combination of a non-volatile memory that can be written and read at any time, such as an HDD or a memory card, and a volatile memory, such as a RAM.

[0077] The data storage unit 33 includes a captured image storage unit 331 , a vehicle information storage unit 332 , and a three-dimensional image storage unit 333 .

[0078] The captured image memory unit 331 is a memory unit that stores all of the captured images acquired by the image acquisition unit 3101 and the plan views corrected by the curvature correction unit 3102 from the time when the first shooting instruction (instruction to start shooting) is received until the time when the second shooting instruction (instruction to end shooting) is received.

[0079] The vehicle information storage unit 332 is used to store vehicle information including the vehicle type, vehicle feature information, and vehicle number information.

[0080] The three-sided image storage unit 333 is a storage unit that stores the three-sided views of the vehicle cut out by the image cut-out control unit 3107. The communication interface 34 is a communication module that connects to the central vehicle confirmation assistance device 6, the toll collection device 2, and the lane server 5c by wire or wirelessly.

[0081] The input / output interface 35 is an interface that enables transmission and reception of information between the input device 36 and the output device 37. The input / output interface 35 may be integrated with the communication interface 34.

[0082] The input device 36 includes, for example, a keyboard, a pointing device, etc., through which a collector or an administrator who manages the vehicle confirmation support device 3 inputs various instructions such as image search to the vehicle confirmation support device 3. The input device 36 may also include a reader for reading data to be stored in the program storage unit 32 or the data storage unit 33 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.

[0083] The output device 37 includes a display for displaying output data to be presented from the vehicle inspection assistance device 3 to the collector or manager, and a printer for printing the same.

[0084] FIG. 5 is a diagram showing an example of a schematic configuration of the central vehicle confirmation assistance device 6 according to the embodiment. The central vehicle confirmation support device 6 is a device connected to the vehicle confirmation support device 3 provided in the toll gate office of each toll gate. The central vehicle confirmation support device 6 is one or more computers, and includes a control unit 61, a program storage unit 62, a data storage unit 63, a communication interface 64, an input / output interface 65, an input device 66, and an output device 67.

[0085] The control unit 61 is a control unit that controls the central vehicle confirmation support device 6. The control unit 61 includes a hardware processor such as a central processing unit (CPU).

[0086] The control unit 61 includes a vehicle information search unit 611 , a three-dimensional view search unit 612 , and a communication control unit 613 .

[0087] The vehicle information search unit 611 is a search unit that searches for matching vehicle information from a vehicle information storage unit 631 (described later) in the data storage unit 63, and searches for vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include a vehicle model previously determined for the vehicle, etc.

[0088] The three-planar view search unit 612 is a search unit that, when necessary in the vehicle confirmation support device 3, acquires three-planar images stored in a three-planar image storage unit 632 (described later) based on vehicle information.

[0089] The communication control unit 613 is a communication control unit for communicating with the vehicle confirmation support device 3 installed in the toll booth office. In response to a transfer request from the vehicle confirmation support device 3, the communication control unit 613 controls the transmission of vehicle information acquired by the vehicle information search unit 611 and the transmission of three-dimensional images acquired by the three-dimensional view search unit 612. In addition, the communication control unit 613 controls the reception of vehicle information and three-dimensional views of the vehicle from each vehicle confirmation support device 3.

[0090] The program storage unit 62 stores programs necessary for executing various processes. Therefore, the control unit 61 can realize various controls and operations by reading and executing the programs stored in the program storage unit 62.

[0091] The data storage unit 63 is a storage unit that stores data acquired and generated in the process of the control unit 61 performing various processes. The data storage unit 63 is a storage that uses a combination of a non-volatile memory that can be written and read at any time, such as a HDD or a memory card, and a volatile memory such as a RAM (Random Access Memory), as a storage medium. The data storage unit 63 also includes a vehicle information storage unit 631 and a three-sided image storage unit 632.

[0092] The vehicle information storage unit 631 is used to store vehicle information received from the vehicle confirmation support device 3 and other central vehicle confirmation support devices 6 when a vehicle passes through an exit toll gate.

[0093] The three-sided image storage unit 632 is used to store three-sided views of a vehicle received from the vehicle confirmation support device 3 and the central vehicle confirmation support device 6 when the vehicle passes through the exit toll gate.

[0094] The communication interface 64 is a communication module that is connected by wire or wirelessly to the vehicle confirmation support device 3 provided in the toll gate office of each toll gate.

[0095] The input / output interface 65 is an interface that enables transmission and reception of information between the input device 66 and the output device 67. The input / output interface 65 may be integrated with the communication interface 64.

[0096] The input device 66 is, for example, a device through which an administrator who manages the central vehicle confirmation support device 6 inputs various instructions to the central vehicle confirmation support device 6, and includes a keyboard, a pointing device, etc. The input device 66 may also include a reader for reading data to be stored in the program storage unit 62 or the data storage unit 63 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.

[0097] The output device 67 includes a display for displaying output data to be presented to the manager from the central vehicle confirmation support device 6, a printer for printing the output data, and the like.

[0098] (operation) First, the operation of this embodiment when applied will be described. In this embodiment, the processing operation of generating a display screen to be displayed to a collector based on a video captured by the wide-angle lens mounted camera 421 will be described.

[0099] FIG. 6 is a diagram showing an example of a display image displayed on the display of the output device 37. As shown in FIG. As shown in Fig. 6, the output device 37 displays vehicle type candidates (1), vehicle characteristic information (2), vehicle number information (3) (shown as No. information in Fig. 6), registration (4), three-sided drawing (5), recalculation (6), storage (7), etc. The toll collector will decide the vehicle type of the non-ETC vehicle based on this information.

[0100] Vehicle model candidate (1) displays vehicle model candidates determined by the vehicle model candidate determination unit 3115 based on vehicle feature information and vehicle number information. For example, Fig. 6 shows an example in which the vehicle model candidate determination unit 3115 determines that the vehicle model candidate is either large or extra large, and the display control unit 3110 controls to display the result in vehicle model candidate (1).

[0101] The vehicle characteristic information (2) displays the number of axles detected by the axle number detection unit 3108 and the lift-up detection unit 3109, and the vehicle width, height, length, etc. estimated by the vehicle characteristic information acquisition unit 3114. Details of the method of estimating the vehicle width, height, and length will be described later.

[0102] Vehicle number information (3) displays various information recognized from the vehicle number image extracted by the vehicle number recognition unit 3106. In Fig. 6, plate size, place name, vehicle classification, usage classification, serial number, color, etc. are displayed. Information that could not be recognized is recorded as unknown. Fig. 6 shows an example in which the plate size, place name, vehicle classification, and color are unknown.

[0103] Registration (4) is a button that the collector presses when additional information is received by the collector in the vehicle characteristics information (2) and vehicle number information (3). In other words, the collector corrects the information in the vehicle characteristics information (2) and vehicle number information (3) from the three-sided drawing (described later) and presses the registration button.

[0104] The three-plan view (5) displays a three-plan view of the vehicle cut out by the image cut-out control unit 3107. The method of cutting out the three-plan views will be described in detail later.

[0105] Recalculate (6) is a button for recalculating the vehicle type based on the vehicle feature information or vehicle number information corrected by registering (4). When this button is pressed, the vehicle type candidate determination unit 3115 will re-determine the vehicle type candidates.

[0106] Save (7) is a button for saving the vehicle model, vehicle characteristics information, vehicle number information, and three-dimensional drawing after the collector has selected the vehicle model from the vehicle model candidates. When the control unit 31 detects that the collector has pressed this button, it stores the vehicle information including the vehicle model, vehicle characteristics information, vehicle number information, and three-dimensional drawing in the vehicle information storage unit 332.

[0107] FIG. 7 is a flowchart showing an example of a procedure for generating a display screen to be displayed on the display of the output device 37. When the vehicle confirmation support device 3 outputs a first shooting instruction (shooting start instruction) to the wide-angle lens mounted camera 421, the wide-angle lens mounted camera 421 starts shooting the lane and outputs the shot image to the vehicle confirmation support device 3.

[0108] This flowchart shows an example in which the wide-angle lens mounted camera 421 is disposed on the left side of the vehicle's traveling direction, i.e., the vehicle moves from right (upstream side) to left (downstream side) of the captured image. However, it is of course also applicable to a case in which the wide-angle lens mounted camera 421 is disposed on the right side of the vehicle's traveling direction, i.e., the vehicle moves from left (upstream side) to right (downstream side) of the captured image.

[0109] In step ST101, the image acquisition unit 3101 acquires a captured image output by the wide-angle lens mounted camera 421, and outputs the acquired captured image to the display control unit 3110 and the curvature correction unit 3102. Here, the image acquisition unit 3101 may store the acquired captured image in the captured image storage unit 331.

[0110] In step ST102, the display control section 3110 outputs the acquired photographed image to the output device 37, and controls the output device 37 so that the lane image displayed on the display thereof is updated.

[0111] In step ST103, the curvature correcting unit 3102 corrects the curvature of the captured image and converts it into a plan view. Here, the curvature correcting unit 3102 may store the plan view in the captured image storage unit 331.

[0112] In step ST104, the vehicle recognition section 3103 determines whether or not to recognize the vehicle.

[0113] The vehicle recognition unit 3103 determines that a moving object is a vehicle when the leading edge of the moving object reaches line A (hereinafter referred to as the "vehicle entry judgment line"), which is a line for recognizing whether or not a vehicle is entering and is set at the entrance end of the plan view, which is the direction from which the vehicle will enter. Here, the vehicle entry judgment line is a predetermined line set upstream (toward the entrance) from the center of the plan view. In step ST104, if no moving object is captured on the vehicle entry judgment line, the vehicle recognition unit 3103 determines that no vehicle has entered, and the process returns to step ST101.

[0114] On the other hand, if a moving object is captured on the vehicle entry judgment line in the plan view, the vehicle recognition unit 3103 judges that a vehicle has been recognized, and the process proceeds to step ST105.

[0115] In step ST105, the vehicle recognition unit 3103 outputs a signal indicating that a tracing process of an object (vehicle) has started to the vehicle position determination unit 3104. Details of the tracing process in step ST105 will be described later with reference to FIG.

[0116] In step ST106, the control unit 31 determines whether or not an instruction to end shooting has been received. If an instruction to end shooting has not been received, the process returns to step ST101. On the other hand, if an instruction to end shooting has been received, the instruction is output to the wide-angle lens mounted camera 421, and the process ends.

[0117] FIG. 8 is a flowchart for explaining in more detail the tracing process of step ST105 shown in FIG. In step ST201, the vehicle recognition unit 3103 judges whether the leading edge of the vehicle is on a line (line B, hereinafter referred to as the "vehicle number recognition reference line") used when cutting out a vehicle number image of the vehicle in the plan view. Here, the vehicle number recognition reference line is a line set downstream (to the exit side) of the vehicle entry detection line. If the vehicle has not advanced to the vehicle number recognition reference line, the process returns to the judgment process of step ST201. At this time, before executing the process of step ST201, the control unit 31 acquires a captured image, updates the lane image, and corrects the curvature, that is, executes the processes of steps ST101 to ST103 in FIG. 7.

[0118] If it is determined that the leading edge of the vehicle has reached the vehicle number recognition reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has reached the vehicle number recognition reference line to the vehicle number recognition unit 3106. Then, the process proceeds to step ST202.

[0119] In step ST202, the vehicle number recognition unit 3106 cuts out a vehicle number image of the vehicle. The vehicle number recognition unit 3106 cuts out a vehicle number image including the license plate placed on the front of the vehicle in the plan view. The vehicle number recognition unit 3106 recognizes vehicle number information including plate size, place name, vehicle classification, use classification, serial number, and color from the license plate included in the vehicle number image. Then, the vehicle number recognition unit 3106 outputs the recognized vehicle number information to the display control unit 3110. Note that if the vehicle number recognition unit 3106 cannot recognize any of the plate size, place name, vehicle classification, use classification, serial number, or color, it may mark it as unknown.

[0120] FIG. 9 is a diagram showing an example of cutting out a vehicle license plate number image of a vehicle from a plan view. As shown in Figure 9, when the front end of the vehicle approaches the vehicle number recognition reference line (line B in Figure 9), the vehicle number recognition unit 3106 cuts out a vehicle number image including the license plate located on the front of the vehicle in the plan view.

[0121] 8, in step ST203, the display control unit 3110 updates the vehicle number information. The display control unit 3110 outputs the vehicle number information received from the vehicle number recognition unit 3106 to the output device 37. That is, the display control unit 3110 controls the display of the output device 37 to display the vehicle number information including the plate size, place name, vehicle classification, usage classification, serial number, and color recognized in step ST202.

[0122] Furthermore, the communication control unit 3111 transmits the acquired vehicle number information to the central vehicle confirmation support device 6 through the communication interface 34. The vehicle information search unit 611 of the central vehicle confirmation support device 6 may search for matching vehicle number information from the vehicle information storage unit 631 of the data storage unit 63, and acquire vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include the vehicle model previously determined, vehicle feature information, vehicle number information, etc.

[0123] Then, the vehicle information search unit 611 outputs the vehicle information to the communication control unit 613. Furthermore, the tri-planar view search unit 612 may search for a tri-planar view corresponding to the vehicle number information from the tri-planar image storage unit 632, and acquire the corresponding tri-planar view. Then, the tri-planar view search unit 612 may output the tri-planar view to the communication control unit 613.

[0124] The communication control unit 613 may transmit the vehicle information to the vehicle confirmation support device 3 through the communication interface 64. The display control unit 3110 may perform control so that the received vehicle information and three-sided diagram are displayed on the output device 37.

[0125] As described above, if the vehicle confirmation support device 3 has the function of the vehicle information search unit 611 of the central vehicle confirmation support device 6, the vehicle confirmation support device 3 may of course perform the above processing. For example, the vehicle information search unit 3112 may search for matching vehicle number information from the vehicle information storage unit 332 and obtain vehicle information corresponding to the relevant vehicle.

[0126] The three-plan view search unit 3113 may search for a three-plan view corresponding to the vehicle number information from the three-plan image storage unit 333, and acquire the corresponding three-plan view. Then, the vehicle information search unit 3112 and the three-plan view search unit 3113 each output the acquired image to the display control unit 3110. The display control unit 3110 controls the display of the output device 37 to display the image.

[0127] In this way, by displaying information about past vehicles (vehicle information and three-sided drawings) on the display of output device 37, it becomes easier for the collector to determine the type of vehicle.

[0128] Here, the control unit 31 executes the processes of obtaining a captured image, updating a lane image, and correcting curvature, that is, steps ST101 to ST103 in FIG.

[0129] Next, in step ST204, the vehicle recognition unit 3103 judges whether the leading edge of the vehicle is on a line (line C in FIG. 14, described later, hereinafter referred to as the "front cut-out reference line") used when cutting out the front of the vehicle in the plan view. Here, the front cut-out reference line is a line set downstream of the vehicle entrance detection line and the vehicle number recognition reference line.

[0130] If the vehicle has not reached the front cut-out reference line, the process returns to the determination process in step ST204.

[0131] When it is determined that the leading edge of the vehicle has reached the front cut-out reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has reached the front cut-out reference line to the image cut-out control unit 3107. Then, the process proceeds to step ST205.

[0132] In step ST205, the image clipping control unit 3107 clips out the front view. The image clipping control unit 3107 determines that the vehicle has advanced to an optimal position for clipping out the front view of the vehicle, and clips out the front view of the vehicle. The image clipping control unit 3107 outputs the clipped front view to the display control unit 3110. Then, the image clipping control unit 3107 stores the clipped front view in the three-sided image storage unit 333.

[0133] FIG. 10 is a diagram showing an example of cutting out a front view of a vehicle from a plan view. When the leading edge of the vehicle approaches the front cut-out reference line (line C in FIG. 10) as shown in FIG. 10, the image cut-out control unit 3107 cuts out a front view of the vehicle. Here, the front view of the vehicle may be either area C1 or area C2.

[0134] Area C1 is an area in which the front view of the vehicle can be captured regardless of the size of the vehicle. Therefore, by cropping out the area like area C1, the size of the vehicle can be compared with other vehicles. Area C2 is an area in which the front of the vehicle is enlarged as much as possible. For example, area C2 may be cropped into a square in accordance with the longer side or longer side of the front of the vehicle. Therefore, by cropping out the area like area C2, it becomes easier to capture the characteristics of the front of the vehicle.

[0135] The receipt clerk may be able to switch between cutting out area C1 and cutting out area C2. In other words, the receipt clerk may set which area to cut out before this processing flow starts.

[0136] In step ST206, the vehicle characteristic information acquisition unit 3114 operating as a vehicle height estimation unit and a vehicle width estimation unit estimates the vehicle height and the vehicle width based on the front view. For example, as shown in FIG. 10, a predetermined object (a stick 51 in the example of FIG. 10) serving as an index is set up on or near the front cut-out reference line. Then, the vehicle characteristic information acquisition unit 3114 operating as the vehicle height estimation unit calculates the vehicle height from the ratio of the height of the stick in the front view to the vehicle. Furthermore, the vehicle characteristic information acquisition unit 3114 may estimate the vehicle height based on the ratio of the distance between the tip of the plan view and the vehicle. Alternatively, the vehicle characteristic information acquisition unit 3114 may estimate the vehicle height using a machine learning model learned from past data.

[0137] Furthermore, for example, the value of the vehicle width of the lane is measured and set in advance (for example, the vehicle width is set to 3000 mm). Then, the vehicle characteristic information acquisition unit 3114 operating as a vehicle width estimation unit estimates the vehicle width based on the ratio between the vehicle width measured in advance (i.e., a predetermined object) in the front view and the vehicle width. Furthermore, the vehicle characteristic information acquisition unit 3114 may estimate the vehicle width based on the ratio between the tip of the plan view and the distance to the vehicle. Alternatively, the vehicle characteristic information acquisition unit 3114 may estimate the vehicle width using a machine learning model learned from past data.

[0138] The vehicle characteristic information acquisition unit 3114 outputs the estimated vehicle width and vehicle height to the display control unit 3110.

[0139] 8, in step ST207, the display control unit 3110 updates the three-sided view and the vehicle characteristic information. The display control unit 3110 controls the display of the output device 37 to display the front view cut out in step ST205. Furthermore, the display control unit 3110 controls the display of the output device 37 to display the vehicle width and vehicle height, which are the estimated vehicle characteristic information.

[0140] Here, the control unit 31 executes the processes of obtaining a captured image, updating a lane image, and updating a curvature correction, that is, steps ST101 to ST103 in FIG.

[0141] Next, in step ST208, the vehicle recognition unit 3103 judges whether the center of the vehicle has progressed to the center position of the plan view (hereinafter referred to as "first position"). Here, the position where the center of the vehicle is at the center position of the plan view refers to a position where the distance between the front end of the vehicle and the front end of the plan view is the same as the distance between the rear end of the vehicle and the rear end of the plan view.

[0142] If the vehicle has not reached the first position, the process returns to the determination process in step ST208.

[0143] When it is determined that the vehicle is at the first position, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has proceeded to the first position to the image cutout control unit 3107. Then, the process proceeds to step ST209.

[0144] In step ST209, the image cutout control unit 3107 cuts out a side view of the vehicle at the best position. The image cutout control unit 3107 cuts out the side view of the vehicle, assuming that the first position is the best position for cutting out the side view of the vehicle. The image cutout control unit 3107 outputs the cut-out side view to the display control unit 3110.

[0145] FIG. 11 is a diagram showing an example of cutting out a side view of a vehicle from a plan view. As shown in Figure 11, a side view of the vehicle is cut out at a position where the distance from the front end of the plan view to the front end of the vehicle (indicated by d in the example of Figure 11) is the same as the distance from the rear end of the plan view to the rear end of the vehicle (indicated by e in the example of Figure 11).

[0146] In step ST210, the axle number detection unit 3108 detects the number of axles based on the side view, and the vehicle characteristic information acquisition unit 3114 operating as a vehicle length estimation unit estimates the vehicle length based on the side view. For example, the axle number detection unit 3108 detects the number of axles of the vehicle from the side view using a common method. Furthermore, the lift-up detection unit 3109 may detect whether the tires are lifted up from the side view using a common method. The axle number detection unit 3108 outputs the detected number of axles to the display control unit 3110.

[0147] Furthermore, the vehicle characteristic information acquisition unit 3114 operating as a vehicle length estimation unit estimates the vehicle length. For example, as shown in FIG. 11, a predetermined object (a stick 52 in the example of FIG. 11) serving as an index is erected near the position d. Then, the vehicle characteristic information acquisition unit 3114 operating as a vehicle length estimation unit calculates the vehicle length from the ratio of the distance from the stick to the front end of the vehicle and the distance to the rear end of the vehicle. Furthermore, for example, the total length of the plan view is measured in advance. Then, the vehicle characteristic information acquisition unit 3114 operating as a vehicle length estimation unit estimates the vehicle length from the ratio of the distance from the front end of the plan view to the front end of the vehicle, that is, the distance d, to the distance from the rear end of the plan view to the rear end of the vehicle, that is, the distance e. Alternatively, the vehicle characteristic information acquisition unit 3114 may estimate the vehicle length using a machine learning model learned from past data. The vehicle characteristic information acquisition unit 3114 outputs the estimated vehicle length to the display control unit 3110.

[0148] In step ST211, the display control unit 3110 updates the three-sided view and the vehicle characteristic information. For example, referring to Fig. 6, a side view is displayed in addition to a front view in the three-sided view on the display of the output device 37. Furthermore, the number of axles and the vehicle length, which are the vehicle characteristic information shown in Fig. 6, are displayed on the display of the output device 37.

[0149] Before executing the process of step ST212, the control unit 31 acquires a captured image, updates a lane image, and corrects curvature, that is, executes the processes of steps ST101 to ST103 in FIG.

[0150] Then, in step ST212, the vehicle recognition unit 3103 determines whether the rear end of the vehicle is ahead of the line used when cutting out the rear view of the vehicle from the plan view (the line f in Figure 12 described later, hereinafter referred to as the "rear cut-out reference line"). If the vehicle has not reached the rear cut-out reference line, the process returns to the determination process in step ST212.

[0151] When it is determined that the rear end of the vehicle has advanced up to the rear surface cut-out reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has advanced up to the rear surface cut-out reference line to the image cut-out control unit 3107. Then, the process proceeds to step ST213.

[0152] In step ST213, the image cutout control unit 3107 cuts out the rear view. The image cutout control unit 3107 determines that the vehicle has advanced to an optimal position for cutting out the rear view of the vehicle, and cuts out the rear view of the vehicle. The image cutout control unit 3107 outputs the cut-out rear view to the display control unit 3110.

[0153] FIG. 12 is a diagram showing an example of cutting out a rear view of a vehicle from a plan view. As shown in FIG. 12, when the rear end of the vehicle approaches the rear view cropping reference line (line f in FIG. 12), the image cropping control unit 3107 crops out a rear view of the vehicle.

[0154] Here, the vehicle characteristic information acquisition unit 3114 operating as a vehicle height estimation unit and a vehicle width estimation unit may re-estimate the vehicle height and vehicle width. For example, as shown in Fig. 12, a predetermined object (stick 53 in the example of Fig. 12) serving as an index is placed on or near the rear surface cutting reference line. The vehicle height and vehicle length may be estimated in the same manner as described in step ST206. For example, if the vehicle width and vehicle height are greater than those estimated in step ST206, the vehicle height and vehicle width may be updated in the same manner as in step ST207.

[0155] In step ST214, the display control unit 3110 updates the three-sided view. The display control unit 3110 outputs the rear view of the vehicle to the output device 37. That is, the display control unit 3110 controls the display of the output device 37 to display the rear view.

[0156] As shown in FIG. 6, in step ST214, all three views, that is, the front view, the side view, and the rear view, are displayed.

[0157] Furthermore, the display control unit 3110 controls the display of the output device 37 to display the received vehicle number image and a three-view diagram including a front view, a side view, and a rear view. This allows the collector to check the vehicle model in a three-view diagram that is easier to recognize.

[0158] Here, the communication control unit 3111 may transmit the tri-planar view to the central vehicle confirmation support device 6 via the communication interface 34. The tri-planar view search unit 612 of the central vehicle confirmation support device 6 may store the received tri-planar view in the tri-planar image storage unit 632. The tri-planar view search unit 612 may acquire the tri-planar image stored in the tri-planar image storage unit 632 when needed by another vehicle confirmation support device 3. Then, the communication control unit 613 may transmit the acquired tri-planar image to the other vehicle confirmation support device 3.

[0159] Although not shown in this flowchart, the vehicle license plate number recognition unit 3106 may cut out a rear vehicle license plate number image behind the vehicle by the same method as described above, either before or after the process of cutting out the rear view in step ST213. Furthermore, as described in step ST202, the vehicle license plate number recognition unit 3106 recognizes vehicle license plate number information.

[0160] Basically, the vehicle number information recognized in step ST202 is displayed on the display of the output device 37. However, if the vehicle number information cannot be recognized at the front, the vehicle number information recognized from the license plate on the rear side may be displayed on the display of the output device 37. Also, in the case of a two-wheeled vehicle or the like, there is no license plate on the front side. In this case, the vehicle number information recognized from the license plate on the rear side is displayed on the display of the output device 37.

[0161] Before executing the process of step ST215, the control unit 31 acquires a captured image, updates a lane image, and corrects curvature, that is, executes the processes of steps ST101 to ST103 in FIG.

[0162] Then, in step ST215, the vehicle recognition unit 3103 judges whether the rear end of the vehicle has advanced to a line (hereinafter referred to as a "vehicle exit detection line") that is used to judge whether the vehicle in the plan view has exited the toll gate. Here, the vehicle exit detection line is a predetermined line that is set downstream of the rear cutout reference line and at the exit side end of the plan view.

[0163] When the rear end of the vehicle approaches the vehicle exit detection line, the vehicle recognition unit 3103 determines that the vehicle has exited the toll gate.

[0164] Here, if the vehicle has not proceeded to the vehicle exit detection line, the process returns to the determination process of step ST215.

[0165] On the other hand, if it is determined in step ST215 that the vehicle has reached the vehicle exit detection line, the process proceeds to step ST216.

[0166] In step ST216, the vehicle model candidate determination unit 3115 determines vehicle model candidates based on the vehicle feature information and vehicle number information obtained in steps ST203, ST206, ST210, etc. The vehicle model candidate determination unit 3115 determines multiple vehicle model candidates based on the vehicle feature information and vehicle number information. Note that the vehicle model candidate determination unit 3115 may determine vehicle model candidates from either the vehicle feature information or the vehicle number information.

[0167] The vehicle model candidate determination unit 3115 outputs the determined vehicle model candidate to the display control unit 3110. The display control unit 3110 controls to display the vehicle model candidate. For example, Fig. 6 shows an example in which the vehicle model candidate is large or extra large based on the vehicle information and the vehicle number information.

[0168] As described above, each time a vehicle passes, a display screen to be shown to the collector is generated based on video captured by the wide-angle lens camera 421.

[0169] The operation of step ST216 may be performed immediately after the operations of steps ST203, ST206, and ST210. In other words, candidate vehicle models may be determined based on the information obtained in the processes up to that point, and displayed on the display of the output device 37.

[0170] In step ST217, the vehicle recognition section 3103 ends the tracing of the vehicle, and then proceeds to step ST106 in FIG.

[0171] Next, the processing operation of the vehicle confirmation support device 3 when the collector determines the vehicle type in this embodiment will be described.

[0172] FIG. 13 is a diagram showing an example of the processing operation of the vehicle confirmation support device 3 when the collector determines the vehicle type. First, it is assumed that the vehicle has passed through a toll booth and an image such as that shown in FIG. 6 is being displayed on the display of the output device 37 in accordance with the processing described above.

[0173] In step ST301, the input information acquisition unit 3116 acquires correction information. The collector corrects the vehicle characteristic information and vehicle number information based on the three-dimensional view shown in the display image shown in FIG. 6. For example, in FIG. 6, it was determined to be four-axis, but when the collector looks at the three-dimensional view, it is lifted up, so it is corrected to three-axis. Then, when the correction is completed, the collector presses registration (4). When the input information acquisition unit 3116 detects that registration (4) has been pressed, it acquires correction information for the vehicle characteristic information or vehicle number information corresponding to the detected registration (4).

[0174] In step ST302, the vehicle model candidate determination unit 3115 updates the candidate vehicle models based on the correction information. The vehicle model candidate determination unit 3115 updates the vehicle model candidates based on the corrected vehicle feature information and vehicle number information. This update may be performed each time the registration (4) is pressed, or may be performed when it is detected that the collector has pressed the recalculate (6) button.

[0175] In step ST303, the input information acquisition unit 3116 detects the pressing of the save button. The collector decides on a vehicle model from the vehicle model candidates and presses it. Then, the collector checks that there are no errors in the various information and then presses the save button. The input information acquisition unit 3116 detects the pressing of the save button.

[0176] In step ST304, the input information acquisition unit 3116 stores the vehicle information including the determined vehicle type, vehicle feature information, vehicle number information, and three-sided drawings in the vehicle information storage unit 332. Furthermore, the input information acquisition unit 3116 outputs the vehicle information to the communication control unit 3111. The communication control unit 3111 transmits the vehicle information to the central vehicle confirmation support device 6 through the communication interface 34. The central vehicle confirmation support device 6 stores the vehicle information in the vehicle information storage unit 631.

[0177] As described above, when a vehicle passes, the control unit 31 of the vehicle confirmation assistance device 3 creates a three-dimensional diagram, and further acquires vehicle characteristic information and vehicle number information within the detectable range. Furthermore, the control unit 31 determines a vehicle type candidate based on the vehicle characteristic information and vehicle number information. The control unit 31 then displays this information on the display of the output device 37. The collector determines the vehicle type by complementing or correcting recognizable information from the three-dimensional diagram (i.e., by visual inspection) among the vehicle characteristic information and vehicle number information.

[0178] (Effects of the embodiment) According to the embodiment, when a collector determines the vehicle type, the vehicle confirmation support device 3 creates a three-dimensional drawing of the vehicle and acquires vehicle characteristic information and vehicle number information. The collector can determine the vehicle type while correcting this information. The collector can determine the vehicle type from a variety of information, which improves the collector's accuracy in determining the vehicle type.

[0179] [Modification of the embodiment] Next, when the vehicle type is determined and the fee is charged to the user, the user may inquire that he or she does not drive such a vehicle type. In order to deal with such a case, a modified example of the embodiment for searching for a target vehicle will be described.

[0180] FIG. 14 is a diagram showing an example of a display screen that is initially displayed on the display of the output device 37 in the modified example of the embodiment. FIG. 14 shows an example in which the passing date and time (1), designated vehicle type (2), vehicle characteristic information (3), vehicle number information (4), three-sided drawing (5), search button (6), and forward / backward key (7) are displayed on the display of the output device 37.

[0181] First, information from the user is input. For example, the user inputs information that is recognizable from among the passing date and time (1), the designated vehicle type (2), the vehicle characteristic information (3), and the vehicle number information (4). The display control unit 3110 acquires vehicle information based on this information from the vehicle information storage unit. The communication control unit 3111 transmits the information to the central vehicle confirmation support device 6 and acquires information corresponding to the information.

[0182] Based on the entered information, by pressing the search button (6), a three-sided drawing (5) of the candidate vehicle, the passing date and time (1), the designated vehicle model (2), vehicle characteristic information (3), and vehicle number information (4) are displayed. If there are multiple candidates, pressing the forward / backward key (7) switches to information on other vehicles. In this way, the system responds to the user's inquiry by explaining what images and information were used to make the decision.

[0183] In addition, the central vehicle confirmation support device 6 may respond to inquiries from users.

[0184] (Effects of the Modification of the Embodiment) According to the advantageous effect of the modified embodiment, in order to respond to an inquiry from a user, it is possible to call up the necessary vehicle information based on the information from the user, thereby making it possible to appropriately respond to the inquiry from the user.

[0185] [Other embodiments] It should be noted that the present invention is not limited to the above-described embodiment.

[0186] For example, the hardware and software configurations constituting the vehicle confirmation support device 3 may be configured within the central vehicle confirmation support device 6. In other words, the processing executed by the vehicle confirmation support device 3 may be executed within the central vehicle confirmation support device 6.

[0187] Also, for example, the hardware configuration and software configuration constituting the central vehicle confirmation support device 6 may be configured within the central toll collection system 1 or the toll collection device 2. In other words, the processing executed by the central vehicle confirmation support device 6 may be executed by the central toll collection system 1 or the toll collection device 2.

[0188] Also, for example, it is not necessary to execute all of the operations in the flowcharts described with reference to FIGS.

[0189] For example, when the collector is able to identify the type of vehicle, he or she may press a button on the input device 36 indicating that processing is to be stopped.

[0190] For example, if the vehicle type can be identified from only the front and side views and the rear view is not required, the collector may press the button to stop the process. In this case, the vehicle tracing may be terminated without executing the process from step ST210 onwards shown in Figure 8, and the process may proceed to step ST106.

[0191] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0192] 1. Central toll collection system 2. Toll collection device 3. Vehicle identification support device 31...Control unit 3101…Image acquisition unit 3102…curvature correction unit 3103…Vehicle Recognition Unit 3104…Vehicle position determination unit 3105...Vehicle commander judgment section 3106…Vehicle Number Recognition Department 3107...Image cutout control unit 3108…Axle number detector 3109...Lift-up detector 3110: Display control unit 3111…Communication control section 3112…Vehicle Information Search Department 3113…3-Drawing Search Section 3114…Vehicle characteristic information acquisition unit 3115…Vehicle Candidate Decision Department 3116…Input information acquisition unit 32...Program memory section 33...Data storage unit 331... Photographed image storage unit 332...Vehicle information storage unit 333...3-sided image storage section 34…Communication interface 35... Input / Output Interface 36...Input device 37...Output device 4…Lane equipment 40…Island 401...Interface aggregation unit 411…First antenna 413…Recovery antenna 421…Wide-angle lens camera 422…Intercom 423…Roadside indicator 424…Automatic toll collection machine 425…Launch control unit 4251…Launch control bar 426…Lane monitoring camera S1…Vehicle detector S2: Vehicle detector S4…Vehicle detector 5a…ETC central device 5b…Tollbooth server 5c…Lane Server 6...Central vehicle confirmation support device 61...Control unit 611…Vehicle Information Search Department 612...3-Drawing Search Section 613…Communication control unit 62...Program memory section 63...Data storage unit 631... Vehicle information storage unit 632...3-sided image storage section 64…Communication interface 65... Input / Output Interface 66...Input device 67...Output device

Claims

1. A storage unit that stores images captured by a wide-angle lens-equipped camera; a curvature correction unit that corrects the curvature of a captured image to generate a plan view; a cut-out control unit that cuts out three views, a front view, a side view, and a rear view, of the vehicle from the plan view; a vehicle characteristic information acquisition unit that acquires vehicle characteristic information including estimated values ​​of the number of axles, width, height, and length of the vehicle detected based on the three-sided views; a vehicle model candidate determination unit that determines vehicle model candidates for the vehicle based on the vehicle feature information; a display control unit that controls an output device to display the vehicle feature information and the vehicle model candidates; an input information acquisition unit that acquires corrected vehicle characteristic information obtained by correcting a value of the vehicle characteristic information; wherein the vehicle model candidate determination unit re-determines vehicle model candidates for the vehicle based on the corrected vehicle characteristic information.

2. A vehicle number recognition unit is further provided for extracting a vehicle number image from the plan view and acquiring vehicle number information from the extracted vehicle number image, The vehicle type candidate determination unit determines the vehicle type candidates further based on the vehicle number information. The vehicle confirmation assistance device according to claim 1.

3. The input information acquisition unit further acquires corrected vehicle number information obtained by correcting a value of the vehicle number information, The vehicle type candidate determination unit re-determines the vehicle type candidates based on the corrected vehicle number information. The vehicle confirmation assistance device according to claim 2.

4. The vehicle characteristic information acquisition unit estimates a vehicle width, a vehicle height, and a vehicle length of the vehicle based on a ratio between the vehicle in the plan view and a predetermined object. The vehicle confirmation assistance device according to claim 1.

5. The vehicle characteristic information acquisition unit estimates a width, a height, and a length of the vehicle based on a ratio of a distance between the leading end of the plan view and the vehicle. The vehicle confirmation assistance device according to claim 1.

6. The vehicle characteristic information acquisition unit estimates the vehicle width, vehicle height, and vehicle length of the vehicle using a machine learning model trained from past data. The vehicle confirmation assistance device according to claim 1.

7. the input information acquisition unit acquires vehicle type determination information from the re-determined vehicle type; The vehicle confirmation support device further includes a communication control unit that transmits vehicle information including the vehicle type determination information and the modified vehicle characteristic information to a central vehicle confirmation support device. The vehicle confirmation assistance device according to claim 1.

8. A vehicle confirmation assistance method executed by a vehicle confirmation assistance device, comprising: A storage unit stores an image captured by a camera equipped with a wide-angle lens; a curvature correction unit corrects the curvature of the captured image to generate a plan view; a cut-out control unit cuts out three views, a front view, a side view, and a rear view, of the vehicle from the plan view; A vehicle characteristic information acquisition unit acquires vehicle characteristic information including estimated values ​​of the number of axles, width, height, and length of the vehicle detected based on the three-sided views; a vehicle type candidate determination unit determining vehicle type candidates for the vehicle based on the vehicle feature information; a display control unit controlling the display of the vehicle feature information and the vehicle model candidates on an output device; An input information acquisition unit acquires corrected vehicle characteristic information obtained by correcting a value of the vehicle characteristic information; the vehicle type candidate determination unit re-determines vehicle type candidates for the vehicle based on the corrected vehicle characteristic information; A vehicle confirmation assistance method comprising:

Images