Vehicle checking support device and vehicle checking support method

The vehicle confirmation support device addresses the issue of poor vehicle type identification at toll plazas by using image processing to correct and display vehicle images, ensuring accurate toll collection despite adverse conditions.

JP2024169219APending Publication Date: 2024-12-05KK TOSHIBA +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023086505
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-25
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

The challenge of accurately identifying vehicle types using camera images at toll plazas is exacerbated by adverse conditions such as rain or vehicle lights, leading to poor image quality and incorrect vehicle type determination.

Method used

A vehicle confirmation support device that includes a storage unit, curvature correction unit, cutout control unit, and display control unit to process and display vehicle images, enabling accurate vehicle type identification even under adverse conditions.

Benefits of technology

Enhances the ability to correctly identify vehicle types by providing clear, corrected, and multiple-view images, facilitating efficient toll collection even in challenging environmental conditions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024169219000001_ABST
    Figure 2024169219000001_ABST
Patent Text Reader

Abstract

To provide a technology for supporting vehicle type determination performed by a collector.SOLUTION: A vehicle checking support device according to an embodiment includes: a storage unit that stores a captured image captured by a camera equipped with a wide-angle lens; a curvature correction unit that corrects a curvature of the captured image and generates a plan view; a segmentation control unit that segments vehicle images for recognizing a vehicle type of a vehicle from the plan view; a previous / subsequent image selection unit that segments previous and subsequent vehicle images by a predetermined number of frames of the segmented vehicle images; a display control unit that controls an output device to display the segmented vehicle images and an image previous / subsequent frame display button; and a previous / subsequent image display control unit that when detecting that the image previous / subsequent frame display button has been pressed, controls the output device to display the vehicle images segmented by the previous / subsequent image selection unit.SELECTED DRAWING: Figure 7
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] An embodiment of the present invention relates to a vehicle confirmation assistance device and a vehicle confirmation assistance method. [Background technology]

[0002] Toll fees for toll roads are generally set according to the distance traveled and the type of vehicle. Toll fees can be roughly divided into two types: toll collection systems, where a toll ticket is collected at the entrance toll gate and the toll is paid in cash or other form at the exit toll gate based on the information on the toll ticket, and ETC systems, where toll fees are collected at the entrance toll gate and exit toll gate of the toll road by wireless communication with on-board devices installed in vehicles passing through the toll gate lanes.

[0003] Currently, the use of ETC systems has come to dominate due to an increase in vehicles that can use ETC systems (hereafter simply referred to as ETC vehicles). However, there are still quite a few vehicles that are not ETC compatible (hereafter simply referred to as non-ETC vehicles).

[0004] In addition, in the toll collection system for charging non-ETC vehicles, the vehicle type is identified at the entrance toll gate and a ticket is issued that records the entrance IC, the time of passage, and the identified vehicle type.Then, at the exit toll gate, the toll is determined based on the ticket received at the entrance toll gate, and the driver of the non-ETC vehicle pays in cash or other means.

[0005] For this reason, in the toll collection system for charging non-ETC vehicles, a booth was set up in each lane at the entrance and exit toll gates, and a toll collector in the booth would visually determine the type of vehicle passing through, issue a ticket at the entrance toll gate, and collect the toll at the exit toll gate.

[0006] In the future, the number of vehicles that can use the ETC system will continue to increase, but non-ETC vehicles that do not use the ETC system will continue to remain, so it is necessary to consider how to deal with non-ETC vehicles. However, since the number of vehicles that do not use the ETC system will decrease, it is not realistic from a cost perspective to install expensive vehicle type discrimination devices at the toll gates on the exit side.

[0007] Recently, in order to reduce the number of people required and to enable remote operations, there has been an increase in cases where automatic toll collection machines are installed in the lanes rather than having toll collectors stationed in booths set up at the lanes, and toll collectors are stationed in toll booth offices away from the lanes or in remote locations away from the toll booths, with the collectors determining the type of vehicle based solely on camera images. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] JP 2011-060138 A [Patent Document 2] JP 2008-250677 A [Patent Document 3] JP 2020-154959 A [Patent Document 4] JP 2019-087159 A Summary of the Invention [Problem to be solved by the invention]

[0009] If vehicle type is identified from only camera images at a toll booth office far from the lane or from a remote location far from the toll booth, there is a problem that necessary information may not be obtained if the shooting conditions are poor, for example, when water droplets adhere to the camera during rainy weather, etc. Furthermore, at night, the camera is illuminated by the headlights of vehicles, and the camera cannot follow the change in the amount of light, causing halation or capturing extremely dark images.

[0010] This invention was made in light of the above-mentioned circumstances, and its purpose is to provide technology that can display information that can complement the characteristics of a vehicle even under poor conditions, thereby assisting collectors in identifying the vehicle type. [Means for solving the problem]

[0011] The vehicle confirmation assistance device of the embodiment includes a memory unit that stores an image captured by a camera equipped with a wide-angle lens, a curvature correction unit that corrects the curvature of the captured image to generate a plan view, an extraction control unit that extracts a vehicle image for identifying the vehicle model from the plan view, a front and rear image selection unit that extracts a predetermined number of frames of the vehicle image before and after the extracted vehicle image, a display control unit that controls an output device to display the extracted vehicle image and an image front and rear frame display button, and a front and rear image display control unit that controls the output device to display the vehicle image extracted by the front and rear image selection unit when it detects that the image front and rear frame display button has been pressed. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a block diagram showing an example of a schematic configuration of a toll road fee processing system according to an embodiment. [Diagram 2] FIG. 2 is a top view and a side view showing a configuration example of lane equipment installed in a lane of an exit toll gate according to the embodiment. [Diagram 3] FIG. 3 is a block diagram showing an example of a schematic configuration of a lane device of the toll road fee processing system according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of a schematic configuration of a vehicle confirmation support device according to an embodiment. [Diagram 5] FIG. 5 is a diagram illustrating an example of a schematic configuration of a central vehicle confirmation support device according to the embodiment. [Figure 6] FIG. 6 is a diagram showing an example of a series of steps at an exit toll gate regarding toll collection when this embodiment is applied. [Figure 7]FIG. 7 is a diagram showing an example of a display image displayed on the display of the output device. [Figure 8] FIG. 8 is a flowchart showing an example of a procedure for generating a display screen to be displayed on the display of the output device. [Figure 9] FIG. 9 is a diagram showing an example of a plan view after the curvature has been corrected. [Figure 10] FIG. 10 is a diagram illustrating an example in which the vehicle recognition unit recognizes a vehicle. [Figure 11] FIG. 11 is a flowchart for explaining in more detail the tracing process of step ST105 shown in FIG. [Figure 12] FIG. 12 is a diagram showing an example of a bird's-eye view displayed on the display of the output device. [Figure 13] FIG. 13 is a diagram showing an example of cutting out a vehicle license plate number image of a vehicle from a plan view. [Figure 14] FIG. 14 is a diagram showing an example of cutting out a front view of a vehicle from a plan view. [Figure 15] FIG. 15 is a diagram showing an example of a three-sided view displayed on the display of an output device. [Figure 16] FIG. 16 is a diagram showing an example of cutting out a side view of a vehicle from a plan view. [Figure 17] FIG. 17 is a diagram showing an example of cutting out a rear view of a vehicle from a plan view. [Figure 18] FIG. 18 is a diagram showing an example of a vehicle number plate image displayed by the front and rear image display control unit. [Figure 19] FIG. 19 is a diagram showing an example of a front view and a rear view displayed by the front and rear image display control unit. [Figure 20] FIG. 20 is a plan view showing an example of a vehicle exiting a toll booth. [Figure 21] FIG. 21 is a diagram showing an example of a display screen that is initially displayed on the display of an output device in a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the vehicle confirmation support device and the vehicle confirmation support method of the present embodiment will be described in detail with reference to the drawings. In the following embodiment, the parts with the same numbers perform the same operation, and the duplicated description will be omitted. For example, when there are a plurality of identical or similar elements, a common symbol may be used to describe each element without distinguishing between them, and a sub-number may be used in addition to the common symbol to describe each element with distinction between them.

[0014] [Embodiment] (composition) FIG. 1 is a block diagram showing an example of a schematic configuration of a toll road fee processing system according to an embodiment. Toll road toll processing systems are roughly divided into ETC systems that collect tolls at the entrance and exit toll gates of a toll road by wireless communication with on-board devices installed in vehicles passing through the lanes of the toll gates, and toll collection systems that receive a toll ticket at the entrance toll gate and pay the toll in cash or the like at the exit toll gate based on the information on the toll ticket. Furthermore, in this embodiment, a vehicle identification support system is provided to remotely support toll collectors located at the toll gate office, etc. in identifying the types of vehicles passing through.

[0015] As shown in Fig. 1, the vehicle confirmation support system for supporting vehicle type identification is composed of a central vehicle confirmation support device 6 provided in the center, vehicle confirmation support devices 3 provided at each toll gate (toll gate office), and lane equipment 4. Here, the vehicle confirmation support system is effective for identifying the vehicle types of passing vehicles at the exit toll gate of the toll collection system, but may also be used for identifying the vehicle types at the entrance toll gate.

[0016] The vehicle confirmation support device 3 is a device for supporting toll collectors in determining the vehicle type. The vehicle confirmation support device 3 controls the timing of photography by the wide-angle lens mounted camera 421 installed in the roadside device, to create a three-sided view of the vehicle from the vehicle number of the vehicle entering the lane and an image of the vehicle (including the axles), and further displays the vehicle's characteristic information to be used as a reference for determining the vehicle type. The vehicle confirmation support device 3 is installed in the toll gate office at the exit toll gate, and is connected to the central vehicle confirmation support device 6, and exchanges various information with the central vehicle confirmation support device 6.

[0017] Furthermore, the vehicle confirmation assistance device 3 is also connected to the toll collection device 2 and the lane server 5c, and exchanges various types of information with the toll collection device 2 and the lane server 5c.

[0018] 1 shows an example in which the central vehicle confirmation support device 6 is connected to one vehicle confirmation support device 3, but in reality, it is connected to multiple vehicle confirmation support devices 3 installed at each of the exit toll gates. The central vehicle confirmation support device 6 exchanges various information with the vehicle confirmation support devices 3.

[0019] The toll road toll processing system is composed of a central toll collection system 1 installed in the center, toll collection devices 2 installed in the toll gate offices of each toll gate, and lane equipment 4 installed in each lane on which vehicles travel at the toll gate.

[0020] The central toll collection system 1 is a server for managing toll collection for non-ETC vehicles. Therefore, the central toll collection system 1 receives and manages various data such as toll information collected from toll collection devices 2 installed at each toll gate.

[0021] The toll collection device 2 is a device for collecting tolls at the exit toll gate using a toll ticket from non-ETC vehicles. The toll collection device 2 is installed in the toll gate machine room of the entrance toll gate and the exit toll gate, and is connected to a device associated with the toll collection device 2 among the lane devices 4 installed in each lane through which non-ETC vehicles can pass. The toll collection device 2 also transmits information such as the paid toll to the central toll collection system 1.

[0022] The ETC system is composed of an ETC central unit 5a installed in the center, a toll gate server 5b installed at each toll gate (toll gate machine room), a lane server 5c that manages and controls the lane equipment 4 of each lane, and lane equipment 4 installed in each lane on which vehicles at the toll gate travel.

[0023] The ETC central device 5a is connected to a tollgate server 5b installed at each tollgate, and is a device that performs toll calculation and charging processing for ETC vehicles. In addition, the ETC central device 5a exchanges various data with each tollgate server 5b.

[0024] The toll gate server 5b is a server for managing the lane servers 5c installed at the entrance toll gates and the exit toll gates, and therefore exchanges various data with the lane servers 5c.

[0025] The lane server 5c is provided in the toll booth machine rooms at the entrance and exit toll booths corresponding to lanes where ETC vehicles can pass, and is connected to the lane devices 4 installed in each lane where ETC vehicles can pass. The lane server 5c controls the lane devices 4 based on sensor data and the like transmitted from devices among the lane devices 4 associated with the lane server 5c.

[0026] The lane server 5c stores the exit information of the exit toll gate where it is installed, and also stores the toll table by section and by vehicle type distributed from the ETC central device 5a. The lane server 5c calculates the toll by referring to the toll table based on the vehicle-mounted device information, ETC card information, entrance information, exit information, and vehicle type information obtained by vehicle type discrimination. The lane server 5c also transmits the toll to the ETC central device 5a via the toll gate server 5b.

[0027] The lane server 5c transmits the calculated toll to the lane device 4.

[0028] The on-board unit installed in an ETC vehicle stores vehicle model information of the vehicle in which it is installed. Here, the vehicle model information is information that indicates the vehicle model, such as a compact car, a normal car, or a large car.

[0029] Next, the configuration of a toll gate will be described, taking the configuration of an exit toll gate as an example.

[0030] FIG. 2 is a top view and a side view showing an example of installation of lane equipment 4 installed in a lane of an exit toll gate according to the embodiment.

[0031] 2, the exit toll gate has lanes for vehicles separated by islands 40, and vehicles travel on these lanes from upstream to downstream. In this embodiment, the lanes are used as dual-use lanes that can be used by both ETC vehicles and non-ETC vehicles that pay tolls in cash or the like.

[0032] Further, on the island 40, in the order from upstream to downstream of the lane, there are provided a wide-angle lens mounted camera 421, a first vehicle detector S1, a second vehicle detector S2, an automatic toll collection machine 424, an intercom 422, a roadside indicator 423, a starting control device 425 and a starting control bar 4251, a third vehicle detector S4, and a lane monitoring camera 426. Note that the positions of the wide-angle lens mounted camera 421 and the first vehicle detector S1 may be reversed.

[0033] In addition, two antenna installation gates are provided in the two islands 40 from upstream to downstream across the lanes. A first antenna 411 is provided at the first antenna installation gate. A recovery antenna 413 is provided at the second antenna installation gate.

[0034] FIG. 3 is a block diagram showing an example of a schematic configuration of the lane device 4 of the toll road fee processing system according to the embodiment.

[0035] In FIG. 3, the lane equipment 4 includes devices related to toll collection, ETC, and vehicle type identification support.

[0036] As the lane equipment 4 related to the vehicle confirmation support device 3, a wide-angle lens mounted camera 421 (photographing unit) 421 is provided and connected to the vehicle confirmation support device 3 at the toll gate office.

[0037] The ETC-related lane equipment 4 includes, via an interface aggregation unit 401, a first antenna 411, a recovery antenna 413, a first vehicle detector S1, a second vehicle detector S2, a third vehicle detector S4, an intercom 422, a roadside display 423, a starting control device 425, and a lane monitoring camera 426.

[0038] An automatic toll collection machine 424 is provided as a roadside device related to toll collection, and is connected to a toll collection device 2 installed in a toll gate office. Furthermore, the toll collection device 2 is also connected to a starting controller 425.

[0039] Further, a wide-angle lens mounted camera (photographing unit) 421 is provided as a lane device 4 related to the vehicle confirmation support device 3, and is connected to the vehicle confirmation support device 3 installed in the toll gate office.

[0040] First, the roadside equipment related to toll collection will be described. The automatic toll collection machine 424 is a device for cash settlement. When the driver of a non-ETC vehicle inserts the toll ticket received at the entrance toll gate into the automatic toll collection machine 424, the toll is calculated and displayed on the automatic toll collection machine 424. The driver of a non-ETC vehicle can execute settlement processing by depositing the displayed toll in cash (or by credit card, etc.).

[0041] Next, the roadside equipment related to the vehicle confirmation assistance device 3 will be described. The wide-angle lens mounted camera 421 operates as a vehicle confirmation support system and is therefore connected to the vehicle confirmation support device 3. The connection to the vehicle confirmation support device 3 may be either a direct connection or a connection via the interface aggregation unit 401.

[0042] 2, the wide-angle lens mounted camera 421 is installed near the first vehicle detector S1 and closer to the installation position of the first vehicle detector S1 than to the installation position of the second vehicle detector S2. The wide-angle lens mounted camera 421 is installed within a range of a predetermined length (for example, 2 m) along the direction in which the lanes extend, based on the installation position of the first vehicle detector.

[0043] In addition, the height of the wide-angle lens mounted camera 421 from the road surface is desirably a height that allows the axle (tire) to be photographed directly in front (for example, about 30 cm to 100 cm from the road surface) while taking into consideration mud splashes from the vehicle, but the height from the road surface can be freely set depending on the characteristics of the wide-angle lens.

[0044] The wide-angle lens-mounted camera 421 is equipped with a wide-angle lens (lens with a short focal length) with a wider angle of view than a standard lens, and captures a predetermined range of the lane. The predetermined range of the lane to be captured includes a first point where the arrival of the vehicle is detected by the first vehicle detector S1 and a second point where the arrival of the vehicle is detected by the second vehicle detector S2. For example, the wide-angle lens-mounted camera 421 captures a predetermined range of about 15 m (about 30 m in total) in the upstream and downstream directions of the lane based on the installation position of the wide-angle lens-mounted camera 421. It is desirable for the wide-angle lens to be capable of capturing the entire side of various types of vehicles (such as light vehicles, ordinary vehicles, medium-sized vehicles, large vehicles, and extra-large vehicles) traveling on the lane, and is desirably a lens (fisheye lens) that can capture a range of about 180°, depending on the installation environment.

[0045] If the wide-angle lens mounted camera 421 is a camera that captures video, it starts capturing video based on a first capturing instruction (instruction to start capturing video) from the vehicle confirmation assistance device 3, and ends capturing video based on a second capturing instruction (instruction to end capturing video), continuing to capture video from the instruction to start capturing video to the instruction to end capturing video, and outputting the captured video.

[0046] Finally, we will explain the ETC-related roadside equipment. The interface aggregation unit 401 aggregates information from each unit and outputs it to the lane server 5c, and also outputs information from the lane server 5c to each unit.

[0047] The interface consolidation unit 401 is connected to the toll collection device 2 and the lane server 5c. The interface consolidation unit 401 outputs a vehicle detection signal ON or a vehicle detection signal OFF to the lane server 5c.

[0048] Here, the first vehicle detector S1 is a detector that detects a vehicle that reaches a position (first point) for detecting the entry of a vehicle in a lane. When the first vehicle detector S1 detects the entry of a vehicle, The first antenna 411 communicates with the vehicle-mounted device of the vehicle and receives various information transmitted from the vehicle-mounted device, such as vehicle-mounted device information, ETC card information, entrance information, etc. If the first antenna 411 cannot communicate with the vehicle-mounted device, the vehicle will reach a second point downstream from the first point.

[0049] The interphone 422 is a device that allows the driver to talk to the toll collector when a problem occurs in the ETC system. Problems include malfunctions of the ETC system, or a non-ETC vehicle mistakenly entering an ETC-only lane.

[0050] The second vehicle detector S2 is a detector that detects a vehicle arriving at the second point. If communication between the first antenna 411 and the vehicle-mounted device cannot be established, the vehicle is stopped once and the collector is called. After the driver and the collector talk via the intercom 422 or the like, the collector operates the recovery antenna 413. Then, the recovery antenna 413 and the vehicle-mounted device start communication, and various information transmitted from the vehicle-mounted device is received.

[0051] The roadside display 423 displays the toll and vehicle type information output from the interface aggregation unit 401 to the driver of the vehicle.

[0052] The departure control device 425 controls the opening and closing of the departure control bar 4251 based on the departure control information from the interface aggregation unit 401. The departure control bar 4251 physically blocks the passage of vehicles (exiting a lane) when in a closed state, and allows the passage of vehicles when in an open state.

[0053] The third vehicle detector S4 is a detector that detects when the vehicle reaches the exit of the lane (a third point downstream from the second point). When the third vehicle detector S4 no longer detects the vehicle, it determines that the vehicle has left the lane and closes the launch control 425 for the next vehicle.

[0054] The lane monitoring camera 426 captures images all the time, and outputs captured images of vehicles entering or exiting a lane to the lane server 5c, for example.

[0055] Next, the configuration of the vehicle confirmation support device 3 installed in the toll gate office will be described.

[0056] FIG. 4 is a diagram showing an example of a schematic configuration of the vehicle confirmation support device 3 according to the embodiment. In addition, the vehicle confirmation assistance device 3 is a device that controls the shooting timing of the wide-angle lens mounted camera 421 installed in the roadside device, and determines and displays the vehicle number of a vehicle entering the lane and the image of the vehicle (including the axles) from these images (whether the number of axles and the vehicle length (whether they exceed the vehicle type-specific standard for determining the toll category).

[0057] The vehicle confirmation assistance device 3 is one or more computers, and includes a control unit 31, a program storage unit 32, a data storage unit 33, a communication interface 34, an input / output interface 35, an input device 36, and an output device 37.

[0058] The control unit 31 controls the vehicle confirmation assistance device 3. The control unit 31 includes a hardware processor such as a central processing unit (CPU). For example, the control unit 31 may be an integrated circuit capable of executing various programs.

[0059] The control unit 31 includes an image acquisition unit 3101, a curvature correction unit 3102, a vehicle recognition unit 3103, a vehicle position determination unit 3104, a vehicle length determination unit 3105, a vehicle number recognition unit 3106, an image clipping control unit 3107, an axle number detection unit 3108, a lift-up detection unit 3109, a display control unit 3110, a communication control unit 3111, a vehicle information search unit 3112, a three-dimensional view search unit 3113, a front and rear image selection unit 3114, and a front and rear image display control unit 3115. The data storage unit 33 includes a captured image storage unit 331, a vehicle information storage unit 332, and a three-dimensional image storage unit 333.

[0060] The image acquisition unit 3101 is an acquisition unit that acquires a captured image output by the wide-angle lens mounted camera 421 of the lane device 4. The image acquisition unit 3101 outputs the acquired captured image to the display control unit 3110 and the curvature correction unit 3102. In addition, the image acquisition unit 3101 may store the acquired captured image in the captured image storage unit 331.

[0061] The curvature correction unit 3102 is a correction unit that generates a plan view in which the curvature of the captured image is corrected. The curvature correction unit 3102 converts the captured image into a plan view in which the curvature is corrected, and outputs the plan view to the vehicle recognition unit 3103. In addition, the curvature correction unit 3102 may store the plan view in the captured image storage unit 331.

[0062] The vehicle recognition unit 3103 is a determination unit that determines whether or not to recognize a vehicle. Details of the determination of whether or not a vehicle has entered will be described later. The vehicle recognition unit 3103 outputs various signals to each unit according to the location where the vehicle has arrived.

[0063] The vehicle position determination unit 3104 is a determination unit that determines where the vehicle is located based on the plan view. The vehicle position determination unit 3104 determines where the vehicle is located on the plan view by generating information required to display the vehicle position on a bird's-eye view based on the determination result.

[0064] The vehicle length determination unit 3105 is a determination unit that determines whether the vehicle is longer than a predetermined length. For example, when the front end of the vehicle reaches a predetermined position (predetermined line) on the plan view, and the rear end of the vehicle does not come to a line set on the edge side of the plan view, the vehicle length determination unit 3105 determines that the vehicle is longer than the predetermined length.

[0065] Conversely, if the rear end of the vehicle is downstream of the line set on the end side of the plan view when the front end of the vehicle reaches the specified line, the vehicle length determination unit 3105 determines that the vehicle is shorter than the specified length.

[0066] The vehicle number recognition unit 3106 cuts out the license plate arranged on the front of the vehicle in the acquired plan view, i.e., the vehicle number image. The vehicle number recognition unit 3106 may also recognize the vehicle number from the cut-out license plate.

[0067] The image cutout control unit 3107 is a control unit that cuts out one of the three views of the vehicle (front view, side view, and rear view) when the vehicle approaches an optimal position for cutting out one of the three views of the vehicle. Details of the cutting out method will be described later.

[0068] The axle number detection unit 3108 is a detection unit that detects the number of axles of the vehicle from the side view cut out by the image cut-out control unit 3107. For example, the axle number detection unit 3108 may detect tires using AI technology and detect the number of axles based on the detected tires.

[0069] The lift-up detection unit 3109 is a detection unit that detects whether the tires are lifted up. Here, lift-up refers to a situation in which a large vehicle such as a large truck or trailer, when empty or with a light load, runs with some of the axles raised and the tires not on the ground.

[0070] The vehicle information search unit 3112 is a search unit that searches for matching vehicle information from a vehicle information storage unit 332 (described later) in the data storage unit 33, and searches for vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include the vehicle type, the number of axles, etc., previously determined for the vehicle.

[0071] The three-planar view search unit 3113 is a search unit that acquires three-planar images stored in a three-planar image storage unit 333 (described later) based on the vehicle features, vehicle number, and the like, when necessary.

[0072] The display control unit 3110 is a control unit that controls the display image displayed on the display of the output device 37 each time the vehicle passes through a toll gate. Here, the display image is a lane image, a vehicle number image, a bird's-eye view, a three-sided view, vehicle information, etc.

[0073] The communication control unit 3111 is a control unit for transmitting and receiving various information to and from the central vehicle confirmation support device 6 through the communication interface 34. Specifically, the communication control unit 3111 performs control for transmitting the lane image, vehicle number image, bird's-eye view, three-sided view, vehicle information, etc. acquired by each control unit to the central vehicle confirmation support device 6. Furthermore, when the communication control unit 3111 cannot properly recognize the vehicle number image, bird's-eye view, three-sided view, vehicle information, etc., it performs control for making a search request to the central vehicle confirmation support device 6 and receiving the lane image, vehicle number image, bird's-eye view, three-sided view, vehicle information, etc. stored in the central vehicle confirmation support device 6.

[0074] The front and rear image selection unit 3114 is an extraction unit that extracts a front vehicle number plate image from the captured images before and after the vehicle number plate image (the front vehicle number plate located on the front of the vehicle and the rear vehicle number plate located on the rear of the vehicle) extracted by the vehicle number plate recognition unit 3106. After the vehicle number plate is extracted by the vehicle number plate recognition unit 3106, the front and rear image selection unit 3114 waits for a certain period of time until the captured image is stored in the captured image storage unit 331.

[0075] Then, the front and rear image selection unit 3114 acquires a predetermined number of front and rear plan views from the captured image storage unit 331 from the plan view from which the vehicle number recognition unit 3106 has extracted the vehicle number. Here, the predetermined number of frames may be any number of frames. For example, the predetermined number of frames may be the number of frames from the front vehicle number image to the plan view that is large enough for the collector to recognize the vehicle number. Then, the front and rear image selection unit 3114 extracts the front vehicle number image from each of the acquired plan views.

[0076] Furthermore, the front and rear image selection unit 3114 is a cutting unit that cuts out the front view and the rear view from the captured images before and after the front view and the rear view cut out by the image cut out control unit 3107. After the front view or the rear view is cut out by the image cut out control unit 3107, the front and rear image selection unit 3114 waits for a certain period of time until the captured images are stored in the captured image storage unit 331.

[0077] Then, the front and rear image selection unit 3114 acquires a predetermined number of front and rear plan views from the captured image storage unit 331 from the plan view from which the image cut-out control unit 3107 has cut out the front view or rear view. Here, the predetermined number of frames may be any number of frames. For example, the predetermined number of frames may be the number of frames up to the plan view that is large enough for the collector to confirm the vehicle axle from the front view. The front and rear image selection unit 3114 then cuts out the front view from each of the acquired plan views.

[0078] The front and rear image display control unit 3115 is a control unit that controls the display of the vehicle number image and the front and rear views that are first displayed on the display of the output device 37. For example, when it detects that the collector has pressed the vehicle number image front and rear frame display button, the front and rear image display control unit 3115 acquires the vehicle number information in chronological order stored in the vehicle information storage unit 332. Then, the front and rear image display control unit 3115 controls the vehicle number image displayed on the display of the output device 37 to display the vehicle number image a predetermined number of frames before or after the vehicle number image.

[0079] The program storage unit 32 stores programs necessary for the control unit 31 to execute various processes. As a storage medium, for example, a combination of a non-volatile memory that can be written and read at any time, such as an EPROM, HDD, or SSD, and a non-volatile memory such as a ROM can be used.

[0080] The data storage unit 33 is a storage unit for storing data acquired and generated in the course of various processes performed by the control unit 31 by executing a program. As a storage medium, for example, it is a storage that uses a combination of a non-volatile memory that can be written and read at any time, such as an HDD or a memory card, and a volatile memory, such as a RAM.

[0081] The data storage unit 33 includes a captured image storage unit 331 , a vehicle information storage unit 332 , and a three-dimensional image storage unit 333 .

[0082] The captured image memory unit 331 is a memory unit that stores all of the captured images acquired by the image acquisition unit 3101 and the plan views corrected by the curvature correction unit 3102 from the time when the first shooting instruction (instruction to start shooting) is received until the time when the second shooting instruction (instruction to end shooting) is received.

[0083] The vehicle information storage unit 332 is used to store the vehicle number recognized by the vehicle number recognition unit 3106 .

[0084] The three-sided image storage unit 333 is a storage unit that stores the three-sided views of the vehicle cut out by the image cut-out control unit 3107. The communication interface 34 is a communication module that connects to the central vehicle confirmation assistance device 6, the toll collection device 2, and the lane server 5c by wire or wirelessly.

[0085] The input / output interface 35 is an interface that enables transmission and reception of information between the input device 36 and the output device 37. The input / output interface 35 may be integrated with the communication interface 34.

[0086] The input device 36 includes, for example, a keyboard, a pointing device, etc., through which a collector or an administrator who manages the vehicle confirmation support device 3 inputs various instructions such as image search to the vehicle confirmation support device 3. The input device 36 may also include a reader for reading data to be stored in the program storage unit 32 or the data storage unit 33 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.

[0087] The output device 37 includes a display for displaying output data to be presented from the vehicle inspection assistance device 3 to the collector or manager, and a printer for printing the same.

[0088] FIG. 5 is a diagram showing an example of a schematic configuration of the central vehicle confirmation assistance device 6 according to the embodiment. The central vehicle inspection support device 6 is a vehicle inspection support device 3 provided in the toll gate office of each toll gate.The central vehicle inspection support device 6 is one or more computers, and is equipped with a control unit 61, a program memory unit 62, a data memory unit 63, a communication interface 64, an input / output interface 65, an input device 66, and an output device 67.

[0089] The control unit 61 is a control unit that controls the central vehicle confirmation support device 6. The control unit 61 includes a hardware processor such as a central processing unit (CPU).

[0090] The control unit 61 includes a vehicle information search unit 611 , a three-dimensional view search unit 612 , and a communication control unit 613 .

[0091] The vehicle information search unit 611 is a search unit that searches for matching vehicle information from a vehicle information storage unit 631 (described later) in the data storage unit 63, and searches for vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include the vehicle type, the number of axles, etc., previously determined for the vehicle.

[0092] The three-planar view search unit 612 is a search unit that, when required by the vehicle confirmation support device 3, acquires three-planar images stored in a three-planar image storage unit 632 (described later) based on the vehicle number.

[0093] The communication control unit 613 is a communication control unit for communicating with the vehicle confirmation support device 3 installed in the toll booth office. In response to a transfer request from the vehicle confirmation support device 3, the communication control unit 613 controls the transmission of vehicle information acquired by the vehicle information search unit 611 and the transmission of three-dimensional images acquired by the plan search unit 612. In addition, the communication control unit 613 controls the reception of vehicle information and three-dimensional images of the vehicle from each vehicle confirmation support device 3.

[0094] The program storage unit 62 stores programs necessary for executing various processes. Therefore, the control unit 61 can realize various controls and operations by reading and executing the programs stored in the program storage unit 62.

[0095] The data storage unit 63 is a storage unit for storing data acquired and generated in the course of various processes performed by the control unit 61. The data storage unit 63 is a storage that uses a combination of a non-volatile memory that can be written and read at any time, such as an HDD or a memory card, and a volatile memory, such as a RAM (Random Access Memory), as a storage medium. The data storage unit 63 also includes a vehicle information storage unit 631 and a three-sided image storage unit 632.

[0096] The vehicle information storage unit 631 is used to store vehicle information received from the vehicle confirmation support device 3 and other central vehicle confirmation support devices 6 when a vehicle passes through an exit toll gate.

[0097] The three-sided image storage unit 632 is used to store three-sided views of a vehicle received from the vehicle confirmation support device 3 and the central vehicle confirmation support device 6 when the vehicle passes through the exit toll gate.

[0098] The communication interface 64 is a communication module that is connected by wire or wirelessly to the vehicle confirmation support device 3 provided in the toll gate office of each toll gate.

[0099] The input / output interface 65 is an interface that enables transmission and reception of information between the input device 66 and the output device 67. The input / output interface 65 may be integrated with the communication interface 64.

[0100] The input device 66 is, for example, a device through which an administrator who manages the central vehicle confirmation support device 6 inputs various instructions to the central vehicle confirmation support device 6, and includes a keyboard, a pointing device, etc. The input device 66 may also include a reader for reading data to be stored in the program storage unit 62 or the data storage unit 63 from a memory medium such as a USB memory, or a disk device for reading such data from a disk medium.

[0101] The output device 67 includes a display for displaying output data to be presented to the manager from the central vehicle confirmation support device 6, a printer for printing the output data, and the like.

[0102] (operation) First, the operation of this embodiment when applied will be described. In this embodiment, the processing operation of generating a display screen to be displayed to a collector based on a video captured by the wide-angle lens mounted camera 421 will be described.

[0103] 6 is a diagram showing an example of a series of steps at an exit toll gate regarding toll collection when this embodiment is applied. Here, the operation of a non-ETC vehicle using the toll collection device 2 will be mainly described.

[0104] As shown in FIG. 6(1), a non-ETC vehicle enters lane equipment 4 installed at the exit toll gate.

[0105] Next, as shown in Fig. 6 (2), the vehicle stops in front of the departure control device 425. Then, the driver inserts the toll ticket received at the entrance toll gate into the automatic toll collection machine 424. At this time, the vehicle confirmation support device 3 acquires an image of the vehicle taken by the wide-angle lens mounted camera 421.

[0106] At this time, as shown in FIG. 6(3), the vehicle confirmation assistance device 3 creates a three-dimensional view from the image captured by the wide-angle lens mounted camera 421, and displays the lane image and the three-dimensional view of the vehicle on the display of the output device 37.

[0107] Next, as shown in Fig. 6(4), the toll collector checks the vehicle type by referring to the display on the vehicle confirmation support device 3. Then, the driver of the vehicle and the toll collector talk to each other through an intercom or the like included in the automatic toll collection machine 424 to confirm the vehicle type, etc.

[0108] Then, as shown in FIG. 6(5), the toll collector inputs the confirmed vehicle type into the toll collection device 2.

[0109] Then, as shown in Fig. 6 (6), the toll collector says to the driver of the vehicle, "Please pay the toll at the automatic toll collection machine" or the like through an intercom or the like included in the automatic toll collection machine 424. In response, the driver pays the toll at the automatic toll collection machine 424. Once the payment of the toll is confirmed, the start control unit 425 is opened and the vehicle exits the lane.

[0110] FIG. 7 is a diagram showing an example of a display image displayed on the display of the output device 37. As shown in FIG. As shown in Fig. 7, the output device 37 displays lane images, vehicle number images (either or both of the front and rear vehicle numbers may be displayed, and in the example of Fig. 7, both the front and rear vehicle numbers are displayed), a bird's-eye view, a three-sided view, vehicle information, etc. Based on this information, the toll collector will determine the type of non-ETC vehicle. Note that the vehicle number image may be obtained not only from the front vehicle number image, but also from the rear of the vehicle, and displayed on the display of the output device 37.

[0111] In this embodiment, as shown in Fig. 7(f), a vehicle number plate image front and rear frame display button is provided for displaying the vehicle number plate image of the frames before and after the vehicle number plate image displayed on the output device 37. Furthermore, as shown in Fig. 7(g), a front / rear image front and rear frame display button is provided for displaying the three-plane images of the frames before and after the front view and the rear view of the three-plane views displayed on the output device 37.

[0112] For example, when it is detected that the vehicle number plate image front and rear frame display button has been pressed by the collector, the front and rear image display control unit 3115 controls the output device 37 to display the vehicle number plate image stored in the vehicle information storage unit 332, which is a predetermined number of frames before or after the currently displayed vehicle number plate image. Similarly, when it is detected that the front / rear image front and rear frame display button has been pressed by the collector, the front and rear image display control unit 3115 controls the output device 37 to display the front view and rear view stored in the vehicle information storage unit 332, which is a predetermined number of frames before or after the currently displayed front view and rear view.

[0113] FIG. 8 is a flowchart showing an example of a procedure for generating a display screen to be displayed on the display of the output device 37. When the vehicle confirmation support device 3 outputs a first shooting instruction (shooting start instruction) to the wide-angle lens mounted camera 421, the wide-angle lens mounted camera 421 starts shooting the lane and outputs the shot image to the vehicle confirmation support device 3.

[0114] This flowchart shows an example in which the wide-angle lens mounted camera 421 is disposed on the left side of the vehicle's traveling direction, i.e., the vehicle moves from right (upstream side) to left (downstream side) of the captured image. However, it is of course also applicable to a case in which the wide-angle lens mounted camera 421 is disposed on the right side of the vehicle's traveling direction, i.e., the vehicle moves from left (upstream side) to right (downstream side) of the captured image.

[0115] In step ST101, the image acquisition unit 3101 acquires a captured image output by the wide-angle lens mounted camera 421, and outputs the acquired captured image to the display control unit 3110 and the curvature correction unit 3102. Here, the image acquisition unit 3101 may store the acquired captured image in the captured image storage unit 331.

[0116] In step ST102, as shown in FIG. 7(a), the display control section 3110 outputs the acquired photographed image to the output device 37, and performs control so as to update the lane image displayed on the display of the output device 37.

[0117] In step ST103, the curvature correcting unit 3102 corrects the curvature of the captured image and converts it into a plan view. Here, the curvature correcting unit 3102 may store the plan view in the captured image storage unit 331.

[0118] FIG. 9 is a diagram showing an example of a plan view after the curvature has been corrected. As shown in FIG. 9, the distortion of an object in a captured image increases from the center to the edge of the image, so some distortion remains even after correction.

[0119] Returning to FIG. 8, in step ST104, the vehicle recognition section 3103 determines whether or not to recognize the vehicle.

[0120] FIG. 10 is a diagram showing an example in which the vehicle recognition unit 3103 recognizes a vehicle. 10, the vehicle recognition unit 3103 determines that a moving object is a vehicle when the leading edge of the moving object reaches line A (hereinafter referred to as "vehicle entry judgment line"), which is a line for recognizing whether or not a vehicle is entering and is set at the entrance end of the plan view, which is the direction from which the vehicle will enter. Here, the vehicle entry judgment line is a predetermined line set upstream (toward the entrance) from the center of the plan view. Returning to FIG. 8, in step ST104, if no moving object is captured on the vehicle entry judgment line, the vehicle recognition unit 3103 determines that no vehicle has entered, and the process returns to step ST101.

[0121] On the other hand, if a moving object is captured on the vehicle entry judgment line in the plan view, the vehicle recognition unit 3103 judges that a vehicle has been recognized, and the process proceeds to step ST105.

[0122] In step ST105, the vehicle recognition unit 3103 outputs a signal indicating that tracing processing of an object (vehicle) has started to the vehicle position determination unit 3104. Details of the tracing processing in step ST105 will be described later with reference to FIG.

[0123] In step ST106, the control unit 31 determines whether or not an instruction to end shooting has been received. If an instruction to end shooting has not been received, the process returns to step ST101. On the other hand, if an instruction to end shooting has been received, the instruction is output to the wide-angle lens mounted camera 421, and the process ends.

[0124] FIG. 11 is a flowchart for explaining in more detail the tracing process of step ST105 shown in FIG. In step ST201, when the vehicle position determination unit 3104 receives a signal indicating that tracing of the vehicle has started from the vehicle recognition unit 3103, the vehicle position determination unit 3104 determines where the vehicle is located on the island 40. At this time, before executing the process of step ST202, the control unit 31 acquires a captured image, updates a lane image, corrects curvature, and updates a bird's-eye view, that is, executes the processes of steps ST101 to ST103 and step ST201 in FIG.

[0125] FIG. 12 is a diagram showing an example of a bird's-eye view displayed on the display of the output device 37. As shown in FIG. Fig. 12 shows the location of the wide-angle lens camera 421 on the island 40 and the range captured by the wide-angle lens camera 421. The example of Fig. 12 shows an example of a bird's-eye view when a vehicle is moving from right to left.

[0126] 12(a) shows a bird's-eye view when a vehicle is recognized. The recognized vehicle (the 104th vehicle) is displayed on the right side of the screen. The number is assigned to the recognized vehicle by the vehicle recognition unit 3103 when the vehicle is recognized.

[0127] 12(b) shows a bird's-eye view of vehicles passing in front of the wide-angle lens camera 421. The recognized vehicle (the 104th vehicle) is displayed by moving to the center of the screen.

[0128] 12(c) shows a bird's-eye view of a vehicle parked in front of the launch control unit 425. The recognized vehicle (vehicle no. 104) is displayed slightly to the left of the center of the screen.

[0129] 12(d) shows an example of a bird's-eye view of a vehicle passing through the launch control unit 425. The recognized vehicle (the 104th vehicle) is displayed on the left side of the screen.

[0130] FIG. 12(e) shows an example in which a following vehicle (the 105th vehicle) is newly recognized.

[0131] As described above, the vehicle position determination unit 3104 determines the location of the vehicle based on the plan view. Then, the vehicle position determination unit 3104 generates information required to display the vehicle position on a bird's-eye view based on the determination result, and outputs the information to the output device 37.

[0132] Returning to FIG. 11, in step ST202, the vehicle recognition unit 3103 judges whether the leading edge of the vehicle is on a line used when cutting out a vehicle license plate image of the vehicle in the plan view (line B in FIG. 13 described later, hereinafter referred to as the "vehicle license plate recognition reference line"). Here, the vehicle license plate recognition reference line is a line set downstream (toward the exit) of the vehicle entry detection line. If the vehicle has not advanced to the vehicle license plate recognition reference line, the process returns to the judgment process of step ST202.

[0133] Here, when it is determined that the leading edge of the vehicle has reached the vehicle number recognition reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has reached the vehicle number recognition reference line to the vehicle number recognition unit 3106. Then, the process proceeds to step ST203.

[0134] In step ST203, the vehicle number recognition unit 3106 cuts out a vehicle number image of the vehicle. The vehicle number recognition unit 3106 cuts out a vehicle number image including the license plate arranged on the front of the vehicle in the plan view. Then, the recognized vehicle number image is output to the display control unit 3110.

[0135] FIG. 13 is a diagram showing an example of cutting out a vehicle license plate number image of a vehicle from a plan view. As shown in Figure 13, when the front end of a vehicle approaches the vehicle number recognition reference line (line B in Figure 13), the vehicle number recognition unit 3106 cuts out a vehicle number image including the license plate located on the front of the vehicle in the plan view.

[0136] 11, in step ST204, the front and rear image selection unit 3114 waits for a certain period of time. That is, it waits until a captured image of a vehicle proceeding beyond the vehicle number recognition reference line is stored in the captured image storage unit 331.

[0137] In step ST205, the front and rear image selection unit 3114 cuts out front vehicle number plate images from the captured images before and after the vehicle number plate image cut out in step ST203. The front and rear image selection unit 3114 acquires a predetermined number of front and rear plan views from the plan view from which the vehicle number plate was cut out in step ST203, from the captured image storage unit 331. Then, the front and rear image selection unit 3114 cuts out front vehicle number plate images from each of the acquired plan views. Then, the front and rear image selection unit 3114 stores the cut out vehicle number plate images in the vehicle information storage unit 332 in chronological order.

[0138] In step ST206, the display control unit 3110 updates the vehicle number image. The display control unit 3110 outputs the vehicle number image received from the vehicle number recognition unit 3106 to the output device 37. That is, the display control unit 3110 controls the output device 37 to display the vehicle number extracted in step ST203 on the display.

[0139] Here, the control unit 31 operating as the acquisition unit may receive the input of the vehicle number recognized by the collector from the vehicle number image. That is, the control unit 31 operating as the acquisition unit acquires the vehicle number.

[0140] Then, the communication control unit 3111 transmits the acquired vehicle number to the central vehicle confirmation support device 6 through the communication interface 34. The vehicle information search unit 611 of the central vehicle confirmation support device 6 may search for a matching vehicle number from the vehicle information storage unit 631 of the data storage unit 63, and acquire vehicle information corresponding to the relevant vehicle. Here, the vehicle information may include the vehicle type, number of axles, etc., previously determined for the vehicle.

[0141] Then, the vehicle information search unit 611 outputs the vehicle information to the communication control unit 613. Furthermore, the tri-planar view search unit 612 may search for a tri-planar view corresponding to the vehicle number from the tri-planar image storage unit 632, and acquire the corresponding tri-planar view. Then, the tri-planar view search unit 612 may output the tri-planar view to the communication control unit 613.

[0142] The communication control unit 613 may transmit the vehicle information to the vehicle confirmation support device 3 through the communication interface 64. The display control unit 3110 may perform control so that the received vehicle information and three-sided diagram are displayed on the output device 37.

[0143] As described above, if the vehicle confirmation support device 3 has the function of the vehicle information search unit 611 of the central vehicle confirmation support device 6, the vehicle confirmation support device 3 may of course perform the above processing. For example, the vehicle information search unit 3112 may search for a matching vehicle number from the vehicle information storage unit 332 and obtain vehicle information corresponding to the relevant vehicle.

[0144] The three-plan view search unit 3113 may search for a three-plan view corresponding to the vehicle number from the three-plan image storage unit 333 and acquire the corresponding three-plan view. Then, the vehicle information search unit 3112 and the three-plan view search unit 3113 each output the acquired images to the display control unit 3110. The display control unit 3110 controls the display of the output device 37 to display them.

[0145] In this way, by displaying information about past vehicles (vehicle information and three-sided drawings) on the display of output device 37, it becomes easier for the collector to determine the type of vehicle.

[0146] Here, the control unit 31 executes the processes of obtaining a captured image, updating a lane image, correcting curvature, and updating a bird's-eye view, that is, steps ST101 to ST103 and ST201 in FIG.

[0147] Next, in step ST207, the vehicle recognition unit 3103 judges whether the leading edge of the vehicle is on a line (line C in FIG. 14, described later, hereinafter referred to as the "front cut-out reference line") used when cutting out the front of the vehicle in the plan view. Here, the front cut-out reference line is a line set downstream of the vehicle entrance detection line and the vehicle number recognition reference line.

[0148] If the vehicle has not reached the front cut-out reference line, the process returns to the determination process in step ST207.

[0149] When it is determined that the leading edge of the vehicle has reached the front cut-out reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has reached the front cut-out reference line to the image cut-out control unit 3107. Then, the process proceeds to step ST208.

[0150] In step ST208, the image cutout control unit 3107 cuts out the front view. The image cutout control unit 3107 determines that the vehicle has advanced to the optimum position for cutting out the front view of the vehicle, and cuts out the front view of the vehicle. The image cutout control unit 3107 outputs the cut-out front view to the display control unit 3110.

[0151] FIG. 14 is a diagram showing an example of cutting out a front view of a vehicle from a plan view. When the leading edge of the vehicle approaches the front cut-out reference line (line C in FIG. 14) as shown in FIG. 14, the image cut-out control unit 3107 cuts out a front view of the vehicle. Here, the front view of the vehicle may be either area C1 or area C2.

[0152] Area C1 is an area in which the front view of the vehicle can be captured regardless of the size of the vehicle. Therefore, by cropping out the area like area C1, the size of the vehicle can be compared with other vehicles. Area C2 is an area in which the front of the vehicle is enlarged as much as possible. For example, area C2 may be cropped into a square in accordance with the longer side or longer side of the front of the vehicle. Therefore, by cropping out the area like area C2, it becomes easier to capture the characteristics of the front of the vehicle.

[0153] The receipt clerk may be able to switch between cutting out area C1 and cutting out area C2. In other words, the receipt clerk may set which area to cut out before this processing flow starts.

[0154] 11, in step ST209, the front and rear image selection unit 3114 waits for a certain period of time. That is, it waits until a captured image of a vehicle proceeding beyond the front cut-out reference line is stored in the captured image storage unit 331.

[0155] In step ST210, the front / rear image selection unit 3114 cuts out a front view from the captured images before and after the front view cut out in step ST208. The front / rear image selection unit 3114 acquires a predetermined number of frames of front and rear plan views from the plan view from which the front view was cut out in step ST208, from the captured image storage unit 331. Then, the front / rear image selection unit 3114 cuts out a front view from each of the acquired plan views. Then, the front / rear image selection unit 3114 stores the cut-out front views in the vehicle information storage unit 332 in chronological order.

[0156] In step ST211, the display control section 3110 updates the three views. The display control section 3110 controls the output device 37 to display the front view extracted in step ST208 on its display.

[0157] FIG. 15 is a diagram showing an example of a three-sided view displayed on the display of the output device 37. As shown in FIG. As shown in Fig. 15(a), when the front view is extracted in step ST210, only the front view is displayed among the three views shown on the display of the output device 37. Figs. 15(b) and (c) will be described later.

[0158] Here, the control unit 31 executes the processes of obtaining a captured image, updating a lane image, correcting curvature, and updating a bird's-eye view, that is, steps ST101 to ST103 and ST201 in FIG.

[0159] Next, in step ST212, the vehicle recognition unit 3103 judges whether the center of the vehicle has progressed to the center position of the plan view (hereinafter referred to as "first position"). Here, the position where the center of the vehicle is at the center position of the plan view refers to a position where the distance between the front end of the vehicle and the front end of the plan view is the same as the distance between the rear end of the vehicle and the rear end of the plan view.

[0160] If the vehicle has not reached the front cut-out reference line, the process returns to the determination process in step ST212.

[0161] When it is determined that the vehicle is at the first position, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has proceeded to the first position to the image cutout control unit 3107. Then, the process proceeds to step ST213.

[0162] In step ST213, the image cutout control unit 3107 cuts out a side view of the vehicle at the best position. The image cutout control unit 3107 cuts out the side view of the vehicle, assuming that the first position is the best position for cutting out the side view of the vehicle. The image cutout control unit 3107 outputs the cut-out side view to the display control unit 3110.

[0163] FIG. 16 is a diagram showing an example of cutting out a side view of a vehicle from a plan view. As shown in Figure 16, a side view of the vehicle is cut out at a position where the distance from the front end of the plan view to the front end of the vehicle (indicated by d in the example of Figure 16) is the same as the distance from the rear end of the plan view to the rear end of the vehicle (indicated by e in the example of Figure 16).

[0164] Here, the image cutout control unit 3107 may acquire plan views before and after the plan view from which the side view was cut out for a predetermined time. Here, the predetermined time may be, for example, several frames. Then, the image cutout control unit 3107 may cut out a side view from each of the acquired plan views. Then, the image cutout control unit 3107 may store the cut-out side views in the three-plane image storage unit 333.

[0165] Furthermore, the axle number detection unit 3108 may detect the number of axles of the vehicle from the side view using a common method. Furthermore, the lift-up detection unit 3109 may detect whether the tires are lifted up from the side view using a common method.

[0166] In step ST214, the display control unit 3110 outputs the side view to the output device 37. For example, referring to Fig. 15(b), a side view is displayed in addition to a front view in the three-sided view on the display of the output device 37. The display control unit 3110 may also output the number of axes detected by the axis number detection unit 3108 and the lift-up detection unit 3109 to the display of the output device 37.

[0167] Before executing the process of step ST215, the control unit 31 acquires a captured image, updates a lane video, corrects curvature, and updates a bird's-eye view, that is, executes the processes of steps ST101 to ST103 and ST201 in FIG.

[0168] Then, in step ST215, the vehicle recognition unit 3103 determines whether the rear end of the vehicle is ahead of the line used when cutting out the rear view of the vehicle from the plan view (the line f in Figure 17 described later, hereinafter referred to as the "rear cut-out reference line"). If the vehicle has not reached the rear cut-out reference line, the process returns to the determination process in step ST215.

[0169] When it is determined that the rear end of the vehicle has advanced up to the rear surface cut-out reference line in the plan view, the vehicle recognition unit 3103 outputs a signal indicating that the vehicle has advanced up to the rear surface cut-out reference line to the image cut-out control unit 3107. Then, the process proceeds to step ST216.

[0170] In step ST216, the display control unit 3110 updates the three-sided view. The display control unit 3110 outputs the rear view of the vehicle to the output device 37. That is, the display control unit 3110 controls the display of the output device 37 to display the rear view.

[0171] In step ST217, the image cutout control unit 3107 cuts out the rear view. The image cutout control unit 3107 determines that the vehicle has advanced to an optimal position for cutting out the rear view of the vehicle, and cuts out the rear view of the vehicle. The image cutout control unit 3107 outputs the cut-out rear view to the display control unit 3110.

[0172] FIG. 17 is a diagram showing an example of cutting out a rear view of a vehicle from a plan view. As shown in FIG. 17, when the rear end of the vehicle approaches the rear view cropping reference line (line f in FIG. 17), the image cropping control unit 3107 crops out a rear view of the vehicle.

[0173] 11, in step ST218, the front and rear image selection unit 3114 waits for a certain period of time. That is, it waits until a captured image of the vehicle traveling beyond the rear cut-out reference line is stored in the captured image storage unit 331.

[0174] In step ST219, the front / rear image selection unit 3114 cuts out rear views from the captured images before and after the rear view cut out in step ST217. The front / rear image selection unit 3114 acquires a predetermined number of front and rear plan views from the plan view from which the rear view was cut out in step ST217, from the captured image storage unit 331. Here, the predetermined number of frames may be the same as or different from the number of frames when the front view was cut out. Then, the front / rear image selection unit 3114 cuts out rear views from each of the acquired plan views. Then, the front / rear image selection unit 3114 stores the cut-out front views in chronological order in the vehicle information storage unit 332.

[0175] In step ST220, the display control unit 3110 updates the three-sided view. The display control unit 3110 outputs the rear view of the vehicle to the output device 37. That is, the display control unit 3110 controls the display of the output device 37 to display the rear view.

[0176] As shown in FIG. 15(c), in step ST220, all three views, that is, the front view, the side view, and the rear view, are displayed.

[0177] Furthermore, the display control unit 3110 controls the display of the output device 37 to display the received vehicle number image and a three-view diagram including a front view, a side view, and a rear view. This allows the collector to check the vehicle model in a three-view diagram that is easier to recognize.

[0178] Here, the communication control unit 3111 may transmit the tri-planar view to the central vehicle confirmation support device 6 via the communication interface 34. The tri-planar view search unit 612 of the central vehicle confirmation support device 6 may store the received tri-planar view in the tri-planar image storage unit 632. The tri-planar view search unit 612 may acquire the tri-planar image stored in the tri-planar image storage unit 632 when needed by another vehicle confirmation support device 3. Then, the communication control unit 613 may transmit the acquired tri-planar image to the other vehicle confirmation support device 3.

[0179] Also, although not shown in this flowchart, the vehicle number recognition unit 3106 may cut out a rear vehicle number image behind the vehicle using a method similar to that described above, either before or after the process of cutting out the rear view in step ST219.

[0180] FIG. 18 is a diagram showing an example of a vehicle number plate image displayed by the front and rear image display control unit 3115. As shown in FIG. Fig. 18(a) is a diagram showing an example of a vehicle number image that is first displayed on the display of the output device 37. As shown in Fig. 18(a), raindrops or the like may adhere to the wide-angle lens mounted camera 421, making it impossible for the collector to recognize the vehicle number. In this case, the collector presses the vehicle number image front and rear frame display button shown in Fig. 7 that is associated with the vehicle number image.

[0181] When the front and rear image display control unit 3115 detects that the collector has pressed the vehicle number image front and rear frame display button, it acquires the vehicle number information in chronological order stored in the vehicle information storage unit 332, and controls to display the vehicle number image in which the vehicle number image in FIG. 18(b) or FIG. 18(c) is moved forward or backward in time. That is, the front and rear image display control unit 3115, for example, shows an example in which the collector has pressed the right arrow of the front and rear frame display button in FIG. 18(b), and shows an example in which the collector has pressed the left arrow of the front and rear frame display button in FIG. In this way, even if the initially displayed vehicle number image is unclear due to raindrops or the like, the collector can correctly recognize the vehicle number by moving the vehicle number image forward or backward.

[0182] FIG. 19 is a diagram showing an example of the front view and the rear view displayed by the front and rear image display control unit 3115. As shown in FIG. FIG. 19(a) is a diagram showing an example of the front view and rear view that are first displayed on the display of the output device 37. As shown in FIG. 19(a), the light of the vehicle hits the wide-angle lens mounted camera 421, and as a result of a large change in the amount of light, halation occurs, and the axles are not shown in the front view and rear view of the vehicle. For example, when the front and rear image display control unit 3115 detects that the collector has pressed the front and rear frame display button shown in FIG. 7, it acquires the front view and rear view in the chronological order stored in the vehicle information storage unit 332, and controls to display the front view and rear view of FIG. 19(b) or FIG. 19(c) moved forward or backward in time. Here, Fig. 19(b) shows an example in which the collector presses the right arrow of the front and rear frame display button, and Fig. 19(c) shows an example in which the collector presses the left arrow of the front and rear frame display button. In this way, even if the front and rear views displayed first are unclear due to the influence of the vehicle's lights, etc., the collector can correctly recognize the number of axles and the type of vehicle by moving the front and rear views back and forth.

[0183] As described above, the front and rear image selection unit 3114 cuts out a predetermined number of frames of front and rear vehicle images of the vehicle image (vehicle number plate image and three-sided view) for identifying the vehicle model cut out by the image cut-out control unit 3107. Then, when it detects that the image front and rear frame display buttons (vehicle number plate image front and rear frame display button and front and rear image front and rear frame display button) have been pressed, the front and rear image display control unit 3115 controls the vehicle image cut out by the front and rear image selection unit 3114 to be displayed on the display of the output device 37.

[0184] Before executing the process of step ST221, the control unit 31 acquires a captured image, updates a lane video, corrects curvature, and updates a bird's-eye view, that is, executes the processes of steps ST101 to ST103 and ST201 in FIG.

[0185] Then, in step ST221, the vehicle recognition unit 3103 judges whether the rear end of the vehicle has passed a line (line g in FIG. 20, described later, hereinafter referred to as the "vehicle exit detection line") that is used to judge whether the vehicle in the plan view has exited the toll gate. Here, the vehicle exit detection line is a predetermined line that is set downstream of the rear cutout reference line and at the exit side end of the plan view.

[0186] FIG. 20 is a plan view showing an example of a vehicle exiting a toll booth. As shown in FIG. 20, when the rear end of the vehicle approaches the vehicle exit detection line (line g in FIG. 20), the vehicle recognition unit 3103 determines that the vehicle has exited the toll gate.

[0187] Here, if the vehicle has not proceeded to the vehicle exit detection line, the process returns to the determination process of step ST221.

[0188] On the other hand, in step ST221, if the vehicle has proceeded to the vehicle exit detection line, the process proceeds to step ST222.

[0189] In step ST222, the vehicle recognition section 3103 ends the tracing of the vehicle, and then proceeds to step ST106 in FIG.

[0190] (Effects of the embodiment) According to the embodiment, when the information for identifying the vehicle type cannot be obtained from the vehicle image (vehicle number image and three-sided view) due to the external environment, the collector presses the image front and rear frame display button to display images before and after the displayed vehicle image. This allows the collector to correctly recognize the information for identifying the vehicle type (vehicle number or number of axles), making it easier for the collector to determine the vehicle type.

[0191] [Modification of the embodiment] FIG. 21 is a diagram showing an example of a display screen that is initially displayed on the display of the output device 37 in the modified example of the embodiment. In the above embodiment, for example, when it is detected that the vehicle number plate image front and rear frame display button is pressed, the front and rear image display control unit 3115 changes both the front and rear of the vehicle number plate image.

[0192] However, for example, if the front vehicle number plate image is recognizable but the rear vehicle image is unrecognizable due to raindrops or the like, only the rear vehicle number plate image may be changed to a recognizable image. Alternatively, only the front vehicle number plate image or the rear vehicle number plate image may be acquired. Therefore, in a modified embodiment, as shown in FIG. 21, vehicle number plate image front and rear frame display buttons are provided on both the front and rear of the vehicle number plate image. This allows the display control unit 3110 to change the display of only either the front or rear in response to pressing of either display button.

[0193] If the vehicle number cannot be recognized on either the front or rear of the vehicle number image due to raindrops or the like, the collector can change only the unrecognizable image to a vehicle number image in which the vehicle number can be recognized. Alternatively, the collector can adjust the front and rear images respectively to change them to the most easily recognizable front image and the most easily recognizable rear image. This allows the collector to more accurately recognize the vehicle number of the vehicle.

[0194] Similarly, as shown in Fig. 21, a front / rear image front and rear frame display button is provided for each of the front and rear views in the three-plane view, so that the front and rear image display control unit 3115 can change the display to only the front view or only the rear view in response to pressing of either display button.

[0195] If the number of axles of a vehicle cannot be recognized from the front or rear view due to halation or the like, the collector can change only the unrecognized image to an image in which the number of axles can be recognized. Alternatively, the collector can adjust the front and rear view images, respectively, to change them to the front and rear views in which the number of axles is easiest to recognize. This allows the collector to more accurately recognize the number of axles of the vehicle.

[0196] [Other embodiments] It should be noted that the present invention is not limited to the above-described embodiment.

[0197] For example, the hardware and software configurations constituting the vehicle confirmation support device 3 may be configured within the central vehicle confirmation support device 6. In other words, the processing executed by the vehicle confirmation support device 3 may be executed within the central vehicle confirmation support device 6.

[0198] Also, for example, the hardware configuration and software configuration constituting the central vehicle confirmation support device 6 may be configured within the central toll collection system 1 or the toll collection device 2. In other words, the processing executed by the central vehicle confirmation support device 6 may be executed by the central toll collection system 1 or the toll collection device 2.

[0199] Also, for example, it is not necessary to execute all of the operations in the flowcharts described with reference to FIG. 8 and FIG.

[0200] For example, when the collector is able to identify the type of vehicle, he or she may press a button on the input device 36 indicating that processing is to be stopped.

[0201] For example, if the vehicle type can be identified from only the front and side views and the rear view is not required, the collector may press the button to stop the process. In this case, the vehicle tracing may be terminated without executing the process from step ST215 onwards shown in Figure 11, and the process may proceed to step ST106.

[0202] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0203] 1. Central toll collection system 2. Toll collection device 3. Vehicle identification support device 31...Control unit 3101…Image acquisition unit 3102…curvature correction unit 3103…Vehicle Recognition Unit 3104…Vehicle position determination unit 3105...Vehicle commander judgment section 3106…Vehicle Number Recognition Department 3107...Image cutout control unit 3108…Axis number detector 3109...Lift-up detector 3110: Display control unit 3111…Communication control section 3112…Vehicle Information Search Department 3113…3-Drawing Search Section 3114…Front and rear image selection section 3115…Front and rear image display control unit 32...Program memory section 33...Data storage unit 331... Photographed image storage unit 332...Vehicle information storage unit 333...3-sided image storage section 34…Communication interface 35... Input / Output Interface 36...Input device 37...Output device 4…Lane equipment 40…Island 401...Interface aggregation unit 411…First antenna 413…Recovery antenna 421…Wide-angle lens camera 422…Intercom 423…Roadside indicator 424…Automatic toll collection machine 425…Launch control unit 4251…Launch control bar 426…Lane monitoring camera S1…Vehicle detector S2: Vehicle detector S4…Vehicle detector 5a…ETC central device 5b…Tollbooth server 5c…Lane Server 6...Central vehicle confirmation support device 61...Control unit 611…Vehicle Information Search Department 612...3-Drawing Search Section 613…Communication control unit 62...Program memory section 63...Data storage unit 631... Vehicle information storage unit 632...3-sided image storage section 64…Communication interface 65... Input / Output Interface 66...Input device 67...Output device

Claims

1. A storage unit that stores images captured by a wide-angle lens-equipped camera; a curvature correction unit that corrects the curvature of a captured image to generate a plan view; a cut-out control unit that cuts out a vehicle image for identifying a vehicle model from the plan view; a front and rear image selection unit that cuts out a predetermined number of frames of the vehicle image before and after the cut-out vehicle image; a display control unit that controls an output device to display the extracted vehicle image and image front and rear frame display buttons; a front and rear image display control unit that, when detecting that the image front and rear frame display button is pressed, controls the output device to display the vehicle image cut out by the front and rear image selection unit; A vehicle confirmation assistance device comprising:

2. the vehicle image is a vehicle number plate image including a front vehicle number plate image arranged on a front surface of the vehicle and a rear vehicle number plate image arranged on a rear surface of the vehicle; When detecting that the image front / rear frame display button corresponding to the vehicle number plate image is pressed, the front / rear image display control unit controls to display a front vehicle number plate image and a rear vehicle number plate image a predetermined number of frames ahead or behind the displayed front vehicle number plate image and the rear vehicle number plate image. The vehicle confirmation assistance device according to claim 1.

3. the vehicle image is a front vehicle number image disposed on the front side of the vehicle, When detecting that the image front / rear frame display button corresponding to the front vehicle number plate image is pressed, the front / rear image display control unit controls to display a front vehicle number plate image a predetermined number of frames before or after the displayed front vehicle number plate image. The vehicle confirmation assistance device according to claim 1.

4. the vehicle image is a rear vehicle number image disposed on the front side of the vehicle, When detecting that the image front / rear frame display button corresponding to the rear vehicle number plate image is pressed, the front / rear image display control unit controls to display a rear vehicle number plate image a predetermined number of frames ahead or behind the displayed rear vehicle number plate image. The vehicle confirmation assistance device according to claim 1.

5. The vehicle image is a three-view image including a front view, a side view, and a rear view of the vehicle; the front and rear image display control unit, when detecting that the image front and rear frame display buttons corresponding to the front view and the rear view are pressed, controls to display the front view and the rear view that are before or after a predetermined number of frames for both the displayed front view and the rear view; The vehicle confirmation assistance device according to claim 1.

6. the vehicle image includes a front view of the vehicle; When detecting that the image front / rear frame display button corresponding to the front view is pressed, the front and rear image display control unit controls to display a front view that is before or after the displayed front view by a predetermined number of frames. The vehicle confirmation assistance device according to claim 1.

7. the vehicle image includes a rear view of the vehicle; When detecting that the image front / rear frame display button associated with the rear view is pressed, the front / rear image display control unit controls to display a rear view that is before or after a predetermined number of frames with respect to the displayed rear view. The vehicle confirmation assistance device according to claim 1.

8. A vehicle confirmation assistance method executed by a vehicle confirmation assistance device, comprising: storing an image captured by a wide-angle lens mounted camera in a storage unit of the vehicle confirmation assistance device; a curvature correction unit corrects the curvature of the captured image to generate a plan view; a cut-out control unit cuts out a vehicle image for identifying a vehicle model from the plan view; a front and rear image selection unit for cutting out a predetermined number of frames of the vehicle image before and after the cut-out vehicle image; a display control unit controls an output device to display the extracted vehicle image and a front and rear frame display button of the image; a front and rear image display control unit, when detecting that the image front and rear frame display button is pressed, controls the output device to display the vehicle image cut out by the front and rear image selection unit; A vehicle confirmation assistance method comprising:

Citation Information

Patent Citations

  • Vehicle-photographing device and electronic toll collection system, and vehicle-photographing method

    JP2008250677A

  • License plate reader and license plate reading method

    JP2011060138A

  • Image display device, toll collection system, image display method, and program

    JP2019087159A

  • Same vehicle determination device, same vehicle determination method, and program

    JP2020154959A