Image acquisition device and image acquisition method

Abstract

To provide an image acquisition device capable of acquiring an image from which plate information can be recognized and capable of acquiring a selection image from which a vehicle can be identified even in a case where the plate information cannot be recognized from the image.SOLUTION: An image acquisition device comprises: a first detection section which detects a plurality of pieces of information included in a car registration plate represented in an inputted acquisition image; a first acquisition section by which, regarding a predetermined number of acquisition images inputted continuously, in a case where a detection ratio in which a plurality of pieces of information are matched and detected is equal to or higher than a reference ratio, one acquisition image in a plurality of acquisition images from which a plurality of pieces of information are matched and detected is selected, a segmentation region around the car registration plate represented in the acquisition image is segmented, and a first candidate image is acquired; a second acquisition section which acquires an acquisition image which is inputted when a detection signal is inputted as a second candidate image; and a selection section which selects the first candidate image as a selection image in a case where the first candidate image is acquired, and selects the second candidate image as a selection image in a case where the first candidate image is not acquired.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image acquisition device and an image acquisition method. [Background technology]

[0002] Conventionally, plate information shown on the vehicle registration plate (number plate) has been used in the settlement process for parking fees when a vehicle is parked in a parking lot.

[0003] In this settlement process, a camera captures an image of the vehicle when it enters, recognizes the plate information shown on the vehicle registration plate shown in the image, and records this plate information in association with the time the vehicle entered (see, for example, Patent Document 1).

[0004] The plate information includes four pieces of information: area name, classification number, hiragana, and serial number. This information is used to associate with the entry time.

[0005] Before leaving the parking lot, the user uses the payment machine to complete the payment process. During the payment process, the user searches for an image representing their vehicle based on the plate information and selects the image representing their vehicle displayed on the payment machine's display, thereby identifying the entry time associated with the vehicle registration number.

[0006] However, since the images acquired by the camera are affected by factors such as weather and the relative position of the camera relative to the vehicle registration plate, it has sometimes been impossible to correctly recognize plate information from the images.

[0007] Therefore, an image of a vehicle entering a parking lot is acquired, and this image is stored as a selected image associated with the entry time. During the payment process, the selected image is displayed based on the entry time. During the payment process, the user selects a selected image showing their vehicle displayed on the payment machine's display, thereby identifying the entry time associated with the selected image. The selected image is also used to recognize plate information. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 11-110695 Summary of the Invention [Problem to be solved by the invention]

[0009] Such selected images must show a large representation of the vehicle in the center of the image, show the vehicle registration plate along with the front grille and hood, and not show the user in a recognizable manner.

[0010] As a method for acquiring a selected image that satisfies the above conditions, it has been proposed to acquire an image when the vehicle registration plate is in a predetermined area within the image.

[0011] As another method for acquiring a selected image that satisfies the above conditions, it has been proposed to determine the timing for acquiring the selected image based on a detection signal from a vehicle sensor that detects a vehicle.

[0012] However, there is thought to be room for improvement in obtaining a selected image from which the vehicle registration number can be recognized.

[0013] Furthermore, even if the plate information cannot be recognized from the image, it is desired that the user be able to obtain a selected image that allows the user to identify his or her own vehicle.

[0014] An object of the present disclosure is to provide an image acquisition device that can acquire an image in which plate information can be recognized, and can also acquire a selected image that allows a user to identify their own vehicle even if the plate information cannot be recognized from the image. [Means for solving the problem]

[0015] According to one embodiment of the present disclosure, there is provided an image acquisition device comprising: an input unit that inputs acquired images of a predetermined vehicle acquired at image acquisition times having a predetermined cycle and a detection signal indicating that the vehicle has been detected; a first detection unit that detects multiple pieces of information included in vehicle registration plates shown in the acquired images input by the input unit; a first acquisition unit that, when a detection rate at which the first detection unit detects multiple pieces of information matching for a predetermined number of acquired images continuously input by the input unit is equal to or greater than a reference rate, selects one of the acquired images in which the multiple pieces of information matching are detected and cuts out an area centered on the vehicle registration plate shown in the acquired image to acquire a first candidate image; a second acquisition unit that, when the detection signal is input from the input unit, acquires the acquired image input by the input unit as a second candidate image; and a selection unit that selects the first candidate image as a selected image if the first candidate image is acquired by the first acquisition unit and selects the second candidate image as the selected image if the first candidate image is not acquired by the first acquisition unit.

[0016] In this image acquisition device, it is preferable that the device has a modification unit that changes the cut-out area based on information representing the size of the vehicle from among the multiple pieces of information detected by the first detection unit, and the first acquisition unit cuts out the cut-out area modified by the modification unit, centered on the vehicle registration plate shown in the acquired image, and acquires the first candidate image.

[0017] Furthermore, in this image acquisition device, when multiple first candidate images are acquired by the first acquisition unit, it is preferable that the selection unit acquires as the selected image, from among the multiple first candidate images, an image in which the ratio of the length to the width of the vehicle registration plate represented in each of the multiple first candidate images is closest to the ratio of the length to the width of the vehicle registration plate when an image including the vehicle registration plate is acquired from the front.

[0018] Furthermore, this image acquisition device has a determination unit that determines whether a vehicle registration plate is included in a specified area of ​​the acquired image input by the input unit, a second detection unit that detects multiple pieces of information included in the vehicle registration plate shown in the acquired image when the determination unit determines that the vehicle registration plate is included, and a third acquisition unit that acquires the acquired image that was the target of detection by the second detection unit as a third candidate image when the number of pieces of information detected by the second detection unit is equal to or greater than a reference number, and it is preferable that the selection unit selects the first candidate image as the selected image when the first candidate image is acquired by the first acquisition unit, and selects the third candidate image as the selected image when the first candidate image is not acquired by the first acquisition unit and the third candidate image is acquired by the third acquisition unit at the selected image determination time that is determined based on the time the detection signal is input from the input unit, and selects the second candidate image as the selected image when the first candidate image is not acquired by the first acquisition unit and the third candidate image is not acquired by the third acquisition unit at the selected image determination time.

[0019] Furthermore, in this image acquisition device, when the detection rate at which all information contained in the vehicle registration plate is detected by the first detection unit as matching for a predetermined number of acquired images input consecutively by the input unit is equal to or higher than a reference rate, it is preferable that the first acquisition unit selects one of the multiple acquired images in which all of the information is detected as matching, cuts out an extraction area centered on the vehicle registration plate shown in the acquired image, and acquires a first candidate image.

[0020] According to another embodiment of the present disclosure, there is provided an image acquisition method, wherein an image acquisition device inputs acquired images of a predetermined vehicle acquired at image acquisition times having a predetermined cycle and a detection signal indicating that the vehicle has been detected, detects a plurality of pieces of information included in the vehicle registration plate shown in the input acquired images, selects one of the acquired images in which the plurality of pieces of information match when a detection rate at which the plurality of pieces of information match is equal to or greater than a reference rate for a predetermined number of consecutively input acquired images, extracts an extraction region centered on the vehicle registration plate shown in the acquired image, and acquires a first candidate image, acquires the input acquired image as a second candidate image when the detection signal is input, selects the first candidate image as a selected image when the first candidate image is acquired, and selects the second candidate image as a selected image when the first candidate image is not acquired. [Effects of the Invention]

[0021] The image acquisition device according to the present disclosure has the effect of acquiring an image in which plate information can be recognized, and also enabling a user to acquire a selected image in which the user can identify his or her own vehicle. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a schematic diagram of a settlement processing system in which the number recognition device of this embodiment is implemented. [Figure 2] FIG. 2 is a hardware configuration diagram of the number recognition device. [Figure 3] 10 is an example of an operational flowchart of a first candidate image acquisition process of the number recognition device. [Figure 4] FIG. 10 is a diagram illustrating acquisition of a first candidate image. [Figure 5] 10 is an example of an operational flowchart of a second candidate image acquisition process of the number recognition device. [Figure 6] 10 is an example of an operational flowchart of a third candidate image acquisition process of the number recognition device. [Figure 7] 10 is an example of an operational flowchart of an image selection process of the number recognition device. [Figure 8] (A) shows an example of an operational flowchart explaining the settlement processing of the settlement processing system, (B) is a diagram explaining the input processing of a search method, (C) is a diagram explaining the input processing of search conditions, (D) is a diagram explaining the input processing of other search conditions, and (E) is a diagram explaining the input processing of a selected image. DETAILED DESCRIPTION OF THE INVENTION

[0023] FIG. 1 is a schematic diagram of a settlement processing system 1 in which a number recognition device 10 of this embodiment is implemented. The settlement processing system 1 has a server 40, a communication network 50, and devices 31 to 38 located in parking lots 30 and the like. The settlement processing system 1 of this embodiment executes settlement processing for parking fees in at least one parking lot 30. The number recognition device 10 is an example of an image capture device.

[0024] Parking lot 30 has a number recognition device 10, a camera 31, an entrance gate device 32, and a vehicle detection sensor 33, which are arranged at the entrance gate, and a number recognition device 35, a camera 36, ​​an exit settlement device 37, and a vehicle detection sensor 38, which are arranged at the exit gate of parking lot 30. In addition, payment processing system 1 has a fee settlement machine 34 for settling parking fees for parking lot 30. Fee settlement machine 34 may be arranged in parking lot 30. Alternatively, fee settlement machine 34 may be arranged in a commercial facility adjacent to parking lot 30.

[0025] Although only one parking lot 30 is shown in FIG. 1, the payment processing system 1 may execute payment processing at multiple parking lots.

[0026] The number recognition device 10, the entrance gate device 32, the fare adjustment machine 34, the number recognition device 35, and the exit settlement device 37 are communicably connected to a server 40 via a communication network 50. Note that the number recognition device 10, the entrance gate device 32, the fare adjustment machine 34, the number recognition device 35, the exit settlement device 37, and the server 40 may be communicably connected by wire or wirelessly.

[0027] The fee adjustment machine 34 and the exit adjustment device 37 communicate with the server 40 to execute the settlement process. The entrance gate device 32 and the exit adjustment device 37 communicate with the server 40 to control the opening and closing operations of the gates. The vehicle detection sensor 33 detects vehicles located near the entrance gate, and the vehicle detection sensor 38 detects vehicles located near the exit gate. The vehicle detection sensor 33 and the vehicle detection sensor 38 each transmit a detection signal indicating that a vehicle has been detected to the server 40 via the communication network 50.

[0028] At the entrance gate of the parking lot 30, the camera 31 captures camera images at image capture times having a predetermined cycle. The camera 31 is installed so that it can capture images of vehicles approaching the entrance gate from a distant location and images of vehicles detected by the vehicle detection sensor 33. The camera image is an example of a captured image. The camera 31 outputs the camera image and the image capture time to the license plate number recognition device 10.

[0029] The license plate recognition device 10 detects plate information contained in the vehicle registration plate displayed in the camera image. The plate information includes four types of plate information: area name, classification number, hiragana, and serial number. The area name represents the place name of the transport bureau and motor vehicle inspection and registration office that have jurisdiction over the vehicle user's base of operations. The classification number represents the vehicle's use and model. The hiragana represents distinguishing characters that indicate the vehicle's use. The serial number, combined with the area name, classification number, and hiragana, represents a unique number that identifies the vehicle.

[0030] The number recognition device 10 also acquires a selected image selected by a user when paying the parking fee. To this end, the number recognition device 10 executes a first candidate image acquisition process, a second candidate image acquisition process, and a third candidate image acquisition process.

[0031] In the first candidate image acquisition process, if the detection rate at which multiple pieces of plate information are detected as matching is equal to or higher than a reference rate for a predetermined number of camera images continuously input from the camera 31, the number recognition device 10 selects one of the camera images in which multiple pieces of plate information are detected as matching. A new first candidate image acquisition process is started each time the entrance gate opens or closes.

[0032] For example, if all four pieces of information including area name, classification number, hiragana, and serial number match for three or more camera images out of a predetermined number of five consecutively input camera images, the number recognition device 10 selects one of these three or more camera images.

[0033] Then, the license plate recognition device 10 extracts an extraction area centered on the vehicle registration plate shown in the selected camera image, and acquires a first candidate image (see FIG. 4). The extraction area may be extracted from the camera image with the center of gravity of the vehicle registration plate as its center.

[0034] The cut-out area can be determined to be compatible with many vehicle models by ensuring that (1) the vehicle is shown large in the center of the image, (2) the vehicle registration plate is shown along with the front grille and hood, and (3) the user, the driver, is not shown identifiably.

[0035] Here, the vehicle registration number may not be correctly recognized from the camera image because the camera image is affected by factors such as weather and the positional relationship between the vehicle registration plate and the camera 31. Therefore, the number recognition device 10 performs a second candidate image acquisition process.

[0036] In the second candidate image acquisition process, the license plate recognition device 10 determines whether or not a vehicle registration plate is included in a predetermined area of ​​each camera image output from the camera 31. The predetermined area is preferably determined so that the camera image when the vehicle registration plate is included in this area satisfies the above-mentioned conditions (1) to (3).

[0037] If a vehicle registration plate is included in a predetermined area of ​​the camera image, the license plate recognition device 10 detects plate information included in the vehicle registration plate shown in the camera image.

[0038] If the number of detected plate information pieces is equal to or greater than the reference number, the number recognition device 10 acquires the camera image that was the subject of this detection as a second candidate image. The second candidate image is an example of a third candidate image. The reference number can be set to, for example, 2.

[0039] The process of acquiring the second candidate image is performed when the vehicle is closer to the entrance gate than when the first candidate image acquisition process was started. Therefore, the vehicle is depicted relatively large in the second candidate image. On a vehicle registration plate, the serial number is depicted in larger characters than the other plate information, so this serial number can usually be detected as one of two or more pieces of information.

[0040] As described above, the camera image is affected by factors such as weather and the positional relationship between the vehicle registration plate and the camera 31, so the vehicle registration number may not be correctly recognized from the camera image. As a result, the number of detected plate information pieces may be less than the reference number. Therefore, the license plate recognition device 10 performs a third candidate image acquisition process.

[0041] In the third candidate image acquisition process, the number recognition device 10 acquires, as a third candidate image, a camera image input by the camera 31 when a detection signal indicating that a vehicle has been detected by the vehicle detection sensor 33 is input. The third candidate image is an example of a second candidate image.

[0042] The process of acquiring this third candidate image may be performed when the vehicle is closer to the entrance gate than when the second candidate image was acquired, so the third candidate image will depict the vehicle at the same size or larger than the second candidate image.

[0043] When the first candidate image is acquired, the license plate recognition device 10 selects the first candidate image as the selected image. This makes it possible to recognize multiple pieces of plate information contained in the vehicle registration plate, and to acquire an image that can be used for payment processing based on both the entry time and the plate information.

[0044] Furthermore, if the first candidate image is not acquired and the second candidate image is acquired at the selected image determination time, which is determined based on the time when a detection signal indicating that a vehicle has been detected by the vehicle detection sensor 33 is input, the number recognition device 10 selects the second candidate image as the selected image. This is expected to make it possible to acquire a selected image from which at least a serial number can be recognized, even in a situation where only a small amount of plate information is detected from the camera image.

[0045] Furthermore, if the first candidate image and the second candidate image are not acquired at the selected image determination time, the number recognition device 10 selects the third candidate image as the selected image. This allows the number recognition device 10 to acquire the selected image even if plate information is not detected.

[0046] As described above in detail, the number recognition device 10 of this embodiment can acquire camera images in which plate information can be recognized, and can also acquire selected images in which the user can identify their own vehicle even if the plate information cannot be recognized from the image.

[0047] 2 is a hardware configuration diagram of the number recognition device 10. The number recognition device 10 executes detection processing, acquisition processing, selection processing, change processing, and judgment processing. To this end, the number recognition device 10 has a communication interface (IF) 21, a memory 22, and a processor 23. The communication IF 21 and the memory 22 are connected to the processor 23 via a signal line 24.

[0048] The communication IF 21 has an interface circuit for connecting the number recognition device 10 to the communication network 50. The communication IF 21 is configured to be able to communicate with the server 40 via the communication network 50. The communication IF 21 is also configured to be able to communicate with the camera 31. The communication IF 21 is an example of an input unit. The communication IF 21 inputs a camera image and the image acquisition time from the camera 31. The communication IF 21 also inputs a detection signal output by the vehicle detection sensor 33 via the entrance gate device 32, the server 40, and the communication network 50.

[0049] The memory 22 includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory, and stores computer programs of applications used in information processing executed by the processor 23 and various data.

[0050] All or part of the functions of the number recognition device 10 are functional modules implemented by, for example, a computer program running on the processor 23. The processor 23 includes a detection unit 231, an acquisition unit 232, a selection unit 233, a change unit 234, and a determination unit 235. The detection unit 231 is an example of a first detection unit and a second detection unit. The acquisition unit 232 is an example of a first acquisition unit, a second acquisition unit, and a third acquisition unit. Alternatively, the functional modules included in the processor 23 may be dedicated arithmetic circuits provided in the processor 23. The processor 23 includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit.

[0051] Next, the first candidate image acquisition process of the number recognition device 10 will be described below with reference to Fig. 3. Fig. 3 is an example of an operational flowchart of the first candidate image acquisition process of the number recognition device 10. In the first candidate image acquisition process shown in Fig. 3, a new process is started every time the entrance gate is opened or closed. Note that the initial process is started regardless of whether the entrance gate is open or closed.

[0052] First, the detection unit 231 determines whether a new camera image has been input via the communication IF 21 (step S101). If a camera image has not been input (step S101-No), the process returns to before step S101. For example, a camera image can be identified using the image acquisition time.

[0053] On the other hand, if a camera image has been input (step S101-Yes), the detection unit 231 determines whether or not the camera image includes a vehicle number registration table (step S102).

[0054] The detection unit 231 has a classifier that receives a camera image as input and is trained to detect vehicle license plate numbers. The classifier receives a camera image as input and detects the area of ​​the vehicle license plate number included in the camera image.

[0055] The classifier is, for example, a deep neural network (DNN) having multiple layers connected in series from the input side to the output side. Images containing vehicle registration plates are used as training data and input to the DNN in advance to perform learning, so that the DNN operates as a classifier that detects the area of ​​the vehicle registration plate. Alternatively, a machine learning model such as a support vector machine or a random forest may be used as the classifier.

[0056] If the vehicle license plate number registration plate is not included in the camera image (step S102-No), the process returns to before step S101.

[0057] On the other hand, if the vehicle registration plate is included in the camera image (step S102-Yes), the detection unit 231 detects the plate information included in the vehicle registration plate shown in the camera image (step S103).

[0058] The detection unit 231 has a classifier that is trained to receive an image containing a vehicle registration plate and detect multiple pieces of plate information contained in the vehicle registration plate. This classifier receives a camera image and recognizes multiple pieces of plate information contained in the vehicle registration plate contained in the camera image. The plate information includes four pieces of information: area name, classification number, hiragana, and serial number.

[0059] The detection unit 231 may recognize four pieces of plate information, namely, area name, classification number, hiragana, and serial number, from the camera image. The detection unit 231 may also recognize some of the four pieces of plate information. The detection unit 231 may also not recognize any of the plate information.

[0060] In step S102, the detection unit 231 may detect the vehicle registration plate from the camera image and also recognize a plurality of pieces of plate information included in the vehicle registration plate.

[0061] Next, the acquisition unit 232 determines whether or not the detection process has been performed on a predetermined number of camera images that have been input consecutively (step S104). The predetermined number can be, for example, five. The acquisition unit 232 determines whether or not the number of detection processes that have been performed consecutively by the detection unit 231 has reached a predetermined number. When the fifth or subsequent camera image is acquired since the initial acquisition, the number of detection processes has always reached the predetermined number.

[0062] If the detection process has not been performed on the predetermined number of camera images (step S104-No), the process returns to before step S101.

[0063] On the other hand, if the detection process has been performed on a predetermined number of camera images (step S104-Yes), the acquisition unit 232 determines whether the detection rate at which multiple plate information is detected as matching by the detection unit 231 for the predetermined number of camera images input consecutively is equal to or greater than a reference rate (step S105).

[0064] In this embodiment, the acquisition unit 232 determines whether the detection rate at which all four pieces of plate information included in a vehicle registration plate are detected as matching by the detection unit 231 for a predetermined number of consecutively input camera images is equal to or greater than a reference rate. For example, the reference rate can be a majority rate. If the predetermined number is 5, the reference rate can be set to 3 / 5. In this case, the detection rate at which all four pieces of plate information are detected as matching for five camera images is equal to or greater than 3 / 5.

[0065] If the ratio is not equal to or greater than the reference ratio (step S105-No), the process returns to before step S101.

[0066] On the other hand, if the ratio is equal to or greater than the reference ratio (step S105—Yes), the acquisition unit 232 selects one of the camera images in which multiple plate information matches, cuts out a cutout area centered on the vehicle registration plate shown in the camera image, acquires a first candidate image (step S106), and ends the series of processes. For example, the acquisition unit 232 selects the camera image that was last targeted for plate information detection as the image for cutting out the cutout area from the multiple camera images in which multiple plate information matches. The camera image that was last targeted for plate information detection is considered to show the vehicle most clearly in the camera image because the vehicle was closest to the camera 31 when the camera image was acquired. Therefore, it is preferable as the camera image for acquiring the selected image. Note that after acquiring one first candidate image, the process may return to before step S101. In this case, multiple first candidate images may be acquired.

[0067] The first candidate image is an image that can recognize all four pieces of plate information contained in the vehicle registration plate, and is compatible with payment processing based on both the entry time and the plate information.

[0068] In the above explanation, it was determined whether the detection rate at which all four pieces of information contained in the vehicle registration plate are detected as matching for a predetermined number of camera images input consecutively is above a standard rate, but it may also be determined whether the detection rate at which two or three pieces of plate information contained in the vehicle registration plate are detected as matching is above a standard rate.

[0069] 4 is a diagram illustrating the acquisition of the first candidate image. The acquisition unit 232 cuts out a partial image of a predetermined cutout area 501 based on the center of gravity of the area of ​​the vehicle license plate number registration plate included in the camera image 500, and generates a first candidate image 510.

[0070] This cut-out area can be determined to be compatible with many vehicle models by ensuring that (1) the vehicle is shown large in the center of the image, (2) the vehicle registration plate is shown along with the front grille and hood, and (3) the user, the driver, is not shown identifiably.

[0071] By transmitting information about the clipped region of the camera image, the amount of communication transmitted from the number recognition device 10 to the server 40 can be reduced.

[0072] Here, the cutout area may be a fixed area, or may be changed based on the size of the vehicle shown in the camera image.

[0073] The change unit 234 changes the cut-out area based on information indicating the size of the vehicle among the plurality of pieces of plate information detected by the detection unit 231. Among the plurality of pieces of plate information, the classification number indicates the vehicle's use / model, and indicates whether it is a standard, small, special-purpose vehicle, or large special-purpose vehicle. The detection unit 231 recognizes whether the classification number indicates a standard, small, special-purpose vehicle, or large special-purpose vehicle.

[0074] The change unit 234 changes the cut-out area so that it is larger when the classification number indicates a special-purpose vehicle or a large special-purpose vehicle than when the classification number indicates a standard or small vehicle. This allows the user to acquire a selected image that makes it easier to identify their vehicle. For example, if the area of ​​the cut-out area when the classification number indicates a standard or small vehicle is 1.0, the area of ​​the cut-out area when the classification number indicates a special-purpose vehicle or a large special-purpose vehicle may be 1.2 to 1.5 times larger.

[0075] The acquisition unit 232 cuts out the cut-out area changed by the change unit 234, and acquires the first candidate image.

[0076] Next, the second candidate image acquisition process of the number recognition device 10 will be described below with reference to Fig. 5. Fig. 5 is an example of an operational flowchart of the second candidate image acquisition process of the number recognition device 10.

[0077] First, the detection unit 231 determines whether a new camera image has been input via the communication IF 21 (step S201). If a camera image has not been input (step S201-No), the process returns to before step S201. For example, a camera image can be identified using the image acquisition time.

[0078] On the other hand, if a camera image has been input (step S201-Yes), the determination unit 235 determines whether or not a vehicle number registration table is included in a predetermined area of ​​the camera image (step S202).

[0079] The specified area can be determined to be compatible with many vehicle models by (1) showing the vehicle prominently in the center of the image, (2) showing the vehicle registration plate along with the front grille and hood, and (3) not showing the user, the driver, in a identifiable manner.

[0080] If the vehicle license plate number registration plate is not included in the predetermined area of ​​the camera image (step S202-No), the process returns to before step S201.

[0081] On the other hand, if the vehicle registration plate is included in the predetermined area of ​​the camera image (step S202-Yes), the detection unit 231 detects the plate information included in the vehicle registration plate shown in the camera image (step S203).

[0082] In step S202, the detection unit 231 may detect the vehicle registration plate from the camera image and also detect plate information included in the vehicle registration plate.

[0083] Next, the acquisition unit 232 determines whether the number of plate information detected by the detection unit 231 is equal to or greater than a reference number (step S204). The reference number can be set to 2. On a vehicle registration plate, the serial number is written in larger characters than the other plate information, so the serial number can usually be detected as one of the two or more pieces of information.

[0084] If the number of plate information pieces is equal to or greater than the reference number (step S204-Yes), the acquisition unit 232 acquires the camera image that was detected by the detection unit 231 as a second candidate image (step S205), and ends the series of processes. Thereafter, the second candidate image acquisition process shown in Fig. 5 is started every time a new vehicle license plate registration plate is detected in an area other than the predetermined area of ​​the camera image. As a result, an image of the same vehicle as the vehicle shown in the first candidate image is acquired as the second candidate image.

[0085] The second candidate image can be expected to be acquired as an image in which at least the serial number can be recognized, even in a situation where the number of pieces of plate information detected from the camera image is small.

[0086] Next, the third candidate image acquisition process of the number recognition device 10 will be described below with reference to FIG. 6. FIG. 6 is an example of an operational flowchart of the third candidate image acquisition process of the number recognition device 10. The third candidate image acquisition process shown in FIG. 6 is started each time the current time reaches a selected image determination time determined based on the time when a detection signal is output from the vehicle detection sensor 33. This ensures a long time to acquire a camera image for acquiring the first candidate image. Alternatively, the third candidate image acquisition process may be started a predetermined time after the current time reaches the selected image determination time so that the next third candidate image acquisition process starts after the vehicle has sufficiently left the detection range of the vehicle sensor 33. This allows an image of the same vehicle as the vehicle depicted in the first candidate image to be acquired as the third candidate image. The selected image determination time will be described later.

[0087] First, the acquisition unit 232 determines whether or not a detection signal indicating that a vehicle has been detected has been input via the communication IF 21 (step S301). If a detection signal has not been input (step S301-No), the process returns to before step S301.

[0088] When the vehicle detection sensor 33 detects a vehicle, it outputs a detection signal to the entrance gate device 32, and the entrance gate device 32 transmits the detection signal and the detection signal acquisition time to the server 40 via the communication network 50. The server 40 transmits the detection signal and the detection signal acquisition time to the license plate number recognition device 10 via the communication network 50. The detection signal may also be input directly from the vehicle detection sensor 33 to the license plate number recognition device 10.

[0089] On the other hand, if a detection signal has been input (step S301-Yes), the acquisition unit 232 determines whether or not a camera image has been input via the communication IF 21 (step S302). If a camera image has not been input (step S302-No), the process returns to before step S302.

[0090] On the other hand, if a camera image has been input (step S302-Yes), the acquisition unit 232 acquires the input camera image as a third candidate image via the communication IF 21 (step S303), and ends the series of processes.

[0091] The third candidate image is acquired based on a detection signal of a vehicle detected by the vehicle detection sensor 33, so that even if the first and second candidate images cannot be acquired, the third candidate image can be acquired. This allows the number recognition device 10 to reliably acquire the selected image.

[0092] Next, the image selection process for acquiring a selected image of the number recognition device 10 will be described below with reference to Fig. 7. Fig. 7 is an example of an operational flowchart of the image selection process of the number recognition device 10. The image selection process shown in Fig. 7 is started anew every time the entrance gate opens or closes.

[0093] 101 First, the selection unit 233 determines whether or not a first candidate image has been acquired (step S401). If a first candidate image has been acquired (step S401-Yes), the selection unit 233 selects the first candidate image as a selected image (step S402). The selection unit 233 generates image identification information that identifies the selected image.

[0094] Next, the selection unit 233 transmits the selected image, which is the first candidate image, the image identification information, the plurality of plate information detected from the camera image cut out of the first candidate image, and the image acquisition time when the camera image was acquired to the server 40 via the communication network 50 (step S403), thereby completing the series of processes. The server 40 stores the selected image, the image identification information, the plurality of plate information, and the entry time, which is the image acquisition time, in association with each other. In this specification, the entry time includes the date and time. The number recognition device 10 also transmits the selected image and the image identification information to the fee adjustment machine 34 and the exit adjustment device 37 via the communication network 50.

[0095] On the other hand, if the first candidate image has not been acquired (step S401-No), the selection unit 233 determines whether the current time has reached the selected image determination time, which is determined based on the time when the detection signal is input from the vehicle detection sensor 33 (step S404).

[0096] The server 40 generates a selected image determination time based on the detection signal acquisition time and transmits it to the number recognition device 10 via the communication network 50. The selected image determination time can be generated as a time that is a predetermined time after the detection signal acquisition time. The selection unit 233 determines the current time using the selected image determination time received from the server 40. Note that the route by which the selected image determination time is notified to the selection unit 233 is not limited to the one described above. Alternatively, the detection signal may be directly input to the number recognition device 10, and the selected image determination time may be generated in the selection unit 233.

[0097] If the current time has not reached the selected image determination time (step S404-No), the process returns to before step S401. Note that if the number recognition device 10 has not received the selected image determination time, the selection unit 233 determines that the current time has not reached the selected image determination time.

[0098] On the other hand, if the current time has reached the selected image determination time (step S404-Yes), the selection unit 233 determines whether or not a second candidate image has been acquired (step S405). If a second candidate image has been acquired (step S405-Yes), the selection unit 233 selects the second candidate image as the selected image (step S406). The selection unit 233 generates image identification information that identifies the selected image.

[0099] Next, the selection unit 233 transmits the selected image, which is the second candidate image, the image identification information, the plurality of plate information detected from the second candidate image, and the image acquisition time when the second candidate image was acquired to the server 40 via the communication network 50 (step S403), thereby completing the series of processes. The server 40 associates and stores the selected image, the image identification information, the plurality of plate information, and the entry time, which is the image acquisition time. The selection unit 233 also transmits the selected image and the image identification information to the fee adjustment machine 34 and the exit adjustment device 37 via the communication network 50.

[0100] On the other hand, if the second candidate image has not been acquired (step S405-No), the selection unit 233 selects the third candidate image as the selected image (step S407). The selection unit 233 generates image identification information for identifying the selected image.

[0101] Next, the selection unit 233 transmits the selected image, which is the third candidate image, the image identification information, and the image acquisition time when the third candidate image was acquired to the server 40 via the communication network 50 (step S403), and ends the series of processes. The server 40 associates and stores the selected image, the image identification information, and the entry time, which is the image acquisition time. The selection unit 233 also transmits the selected image and the image identification information to the fee adjustment machine 34 and the exit adjustment device 37 via the communication network 50.

[0102] When the entrance time is stored, the server 40 controls the entrance gate device 32 to open the gate. The user drives the vehicle into the parking lot and parks the vehicle.

[0103] In the above-described operation example, when the first candidate image is acquired, the selection unit 233 immediately acquires the first candidate image as the selected image, but the selection unit 233 may acquire the selected image based on the priority order of the first candidate image, the second candidate image, and the third candidate image at the selected image determination time. In this case, when the first candidate image is acquired, the first candidate image becomes the selected image.

[0104] When a selected image is acquired at the selected image determination time, the acquisition unit 232 may acquire multiple first candidate images if the detection rate at which multiple pieces of information match is equal to or higher than a reference rate for a predetermined number of camera images input consecutively between the start of the first candidate image acquisition process and the selected image determination time.

[0105] When multiple first candidate images are acquired, the selection unit 233 may select as the selected image an image that is estimated to have been acquired from a more frontal perspective based on the ratio between the vertical and horizontal directions of the vehicle registration plate represented in the multiple first candidate images.

[0106] When viewed from the front, a vehicle registration plate has a generally horizontally elongated rectangular shape. The ratio between the vertical and horizontal directions of the vehicle registration plate when an image including the vehicle registration plate is acquired from the front is set as the standard ratio. The selection unit 233 acquires, from among the multiple first candidate images, the image in which the ratio between the vertical and horizontal directions of the vehicle registration plate depicted in each image is closest to the standard ratio as the selected image.

[0107] This allows the license plate number recognition device 10 to provide the user with a selected image that allows the user to more clearly identify his or her vehicle.

[0108] In the above-described operational example, if the third candidate image is selected as the selected image, the automobile registration plate shown in this third candidate image can be said to be an image in which plate information is difficult to recognize. Therefore, the third candidate image selected as the selected image is preferably used as training data when machine learning a classifier. By using the third candidate image selected as the selected image as training data, it is expected that a classifier can be created that can recognize plate information from images in which plate information was previously difficult to recognize.

[0109] 8(A) to 8(E), the following describes the settlement process of the settlement processing system 1. A user performs the settlement process for the parking fee at the fee settlement machine 34 or the exit settlement device 37.

[0110] First, the user inputs a search method for searching for his / her vehicle at the toll adjustment machine 34 or the exit adjustment device 37 (step S501). Figure 8(B) is a diagram explaining the search method input process. The user selects the entry time (search by time) or the serial number of the plate information (search by number) as the search method.

[0111] Next, the user inputs search conditions into the fare adjustment machine 34 or the exit adjustment device 37 according to the selected search method (step S502). If entry time is selected as the search method, the user inputs the entry time. If serial number is selected as the search method, the user inputs the serial number. The fare adjustment machine 34 or the exit adjustment device 37 transmits the entry time or serial number to the server 40 via the communication network 50.

[0112] Figure 8(C) is a diagram explaining the process of inputting search conditions when entry time is selected as the search method, and Figure 8(D) is a diagram explaining the process of inputting search conditions when serial number is selected as the search method.

[0113] The server 40 transmits image identification information identifying one or more selected images to the fee adjustment machine 34 or the exit adjustment device 37 via the communication network 50 based on the entry time or the serial number. For example, the server 40 transmits image identification information identifying selected images of vehicles that entered the parking lot 30 within 30 minutes before and after the input entry time. The server 40 also transmits image identification information identifying selected images associated with a serial number that matches the received serial number. Note that the period of 30 minutes is an example and is not limited to this.

[0114] If the serial number input by the user is not stored, the server 40 notifies the fee adjustment machine 34 or the exit adjustment device 37 to display the entry time as a search method.

[0115] Next, the fee adjustment machine 34 or the exit adjustment device 37 displays one or more selected images corresponding to the image identification information received from the server 40 from among the plurality of selected images stored therein (step S503).

[0116] The user selects an image depicting his / her vehicle from the displayed selection images (step S504). The toll adjustment machine 34 or the exit adjustment device 37 transmits image identification information identifying the selected selection image to the server 40 via the communication network 50.

[0117] The server calculates the parking fee based on the entry time associated with the selected image identified by the image identification information and the time of payment, and transmits the calculated fee to the fee payment machine 34 or the exit payment device 37 via the communication network 50.

[0118] If the user has a discount coupon for the parking fee, the server 40 may calculate the parking fee based on the discount information indicated on the discount coupon.

[0119] Next, the fee adjustment machine 34 or the exit payment device 37 displays the parking fee (step S505).

[0120] The user performs the parking fee settlement process at the fee settlement machine 34 or the exit settlement device 37 (step S506), and the series of processes ends.

[0121] When a vehicle leaves parking lot 30, an image captured by camera 36 is input into license plate recognition device 35, which then transmits the plate information contained in the detected vehicle registration plate to server 40 via communication network 50. When server 40 determines that the settlement process for the received plate information has been completed and receives a detection signal from vehicle detection sensor 38 indicating that a vehicle has been detected, it controls exit settlement device 37 to open the exit gate. The user drives the vehicle to exit parking lot 30.

[0122] As described above in detail, the number recognition device of this embodiment can obtain an image in which plate information can be recognized, and if the plate information cannot be recognized from the image, the user can obtain a selected image in which their own vehicle can be identified.

[0123] In the present invention, the image acquisition device and image acquisition method of the above-described embodiments can be modified as appropriate without departing from the spirit of the present invention. Furthermore, the technical scope of the present invention is not limited to those embodiments, but extends to the inventions set forth in the claims and their equivalents.

[0124] For example, in the above-described embodiment, the image acquisition device has a process for acquiring the first, second, and third candidate images, but the image acquisition device may have only a process for acquiring the first and third candidate images (as an example of the second candidate image). In this case, if the first candidate image is acquired, the image acquisition device selects the first candidate image as the selected image, and if the first candidate image is not acquired, the image acquisition device selects the third candidate image as the selected image.

[0125] In the above-described embodiment, the image recognition device performs the image acquisition process, but this image acquisition process may be performed by a server. The server receives camera images from the camera and performs each process, including the input process, detection process, acquisition process, and selection process, which were previously performed by the image recognition device. [Explanation of symbols]

[0126] 1. Payment system 10 Number recognition device 21 Communication Interface 22 Memory 23 processors 231 Detector 232 Acquisition Department 233 Selection Section 234 Changes 235 Judgment section 24 signal line 30 Parking 31 Camera 32 Entrance gate device 33 Vehicle detection sensor 34 Fare payment machine 35 Number Recognition Device 36 Camera 37 Exit gate device 38 Vehicle detection sensor 40 servers 50 Communication Network 60 vehicles

Claims

1. an input unit that inputs images of a predetermined vehicle acquired at image acquisition times having a predetermined cycle and a detection signal indicating that the vehicle has been detected; a first detection unit that detects a plurality of pieces of information included in the vehicle registration plate shown in the acquired image input by the input unit; a first acquisition unit that, when a detection rate at which the plurality of pieces of information are detected as matching by the first detection unit is equal to or higher than a reference rate for a predetermined number of the acquired images continuously input by the input unit, selects one of the acquired images for which the plurality of pieces of information are detected as matching, cuts out a cutout area centered on the automobile registration plate shown in the acquired image, and acquires a first candidate image; a second acquisition unit that acquires the acquired image input by the input unit as a second candidate image when the detection signal is input from the input unit; When the first candidate image is acquired by the first acquisition unit, the first candidate image is selected as a selected image, and a selection unit that selects the second candidate image as a selected image when the first acquisition unit does not acquire the first candidate image; An image acquisition device having:

2. a change unit that changes the cutout area based on information that indicates a size of the vehicle among the plurality of pieces of information detected by the first detection unit, The image acquisition device according to claim 1 , wherein the first acquisition unit acquires the first candidate image by extracting the extraction area changed by the modification unit, the extraction area being centered on a vehicle registration plate shown in the acquired image.

3. 2. The image acquisition device of claim 1, wherein when a plurality of first candidate images are acquired by the first acquisition unit, the selection unit acquires as the selected image an image from among the plurality of first candidate images in which the ratio of the length to the width of the vehicle registration plate represented in each of the plurality of first candidate images is closest to the ratio of the length to the width of the vehicle registration plate when an image including the vehicle registration plate from the front is acquired.

4. a determination unit that determines whether a vehicle registration plate is included in a predetermined area of ​​the acquired image input by the input unit; a second detection unit that detects the plurality of pieces of information included in the vehicle registration plate represented in the acquired image when the determination unit determines that the vehicle registration plate is included; a third acquisition unit that acquires the acquired image that is the detection target of the second detection unit as a third candidate image when the number of pieces of information detected by the second detection unit is equal to or greater than a reference number; and The selection unit When the first candidate image is acquired by the first acquisition unit, the first candidate image is selected as a selected image; and If the first candidate image is not acquired by the first acquisition unit and the third candidate image is acquired by the third acquisition unit at a selected image determination time that is determined based on the time when the detection signal is input from the input unit, select the third candidate image as a selected image; and The image acquisition device according to claim 1, wherein if the first candidate image is not acquired by the first acquisition unit and the third candidate image is not acquired by the third acquisition unit at the selected image determination time, the second candidate image is selected as the selected image.

5. 5. The image acquisition device of claim 1, wherein, when the detection rate at which all information contained in a vehicle registration plate is detected as matching by the first detection unit for the predetermined number of acquired images input consecutively by the input unit is equal to or higher than a reference rate, the first acquisition unit selects one of the acquired images in which all information is detected as matching, cuts out an area centered on the vehicle registration plate represented in the acquired image, and acquires the first candidate image.

6. The image capture device inputting images of a predetermined vehicle acquired at image acquisition times having a predetermined cycle and a detection signal indicating that the vehicle has been detected; Detecting a plurality of pieces of information contained in the vehicle registration plate represented in the input acquired image; When the detection rate at which the plurality of pieces of information are detected as matching is equal to or higher than a reference rate for a predetermined number of consecutively input acquired images, one of the acquired images at which the plurality of pieces of information are detected as matching is selected, and an acquisition area is cut out centered on the vehicle registration plate shown in the acquired image, thereby acquiring a first candidate image; When the detection signal is input, the input acquired image is acquired as a second candidate image; If the first candidate image is acquired, select the first candidate image as a selected image, and if the first candidate image is not acquired, select the second candidate image as a selected image. Image acquisition method to perform that.

Images