Identification device, identification method, and program

Abstract

To provide an identification device capable of suppressing a decrease in individual identification accuracy of a vehicle due to erroneous recognition of license plate information.SOLUTION: An identification device comprises: a first acquisition unit that acquires a first front-shooting image and a first rear-shooting image from a first front-shooting camera and a first rear-shooting camera of an entrance toll gate; a first image recognition unit that generates first license plate information from the first front-shooting image and the first rear-shooting image; a second acquisition unit that acquires a second front-shooting image and a second rear-shooting image from the second front-shooting camera and the second rear-shooting camera of the exit toll gate; a second image recognition unit that generates second license plate information from the second front-shooting image and the second rear-shooting image; and a selection unit that selects first license plate information related to the same vehicle as a vehicle arriving at the exit toll gate on the basis of the comparison between the first license plate information and the second license plate information.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an identification device, an identification method, and a program. [Background technology]

[0002] A technology for reading a vehicle's license plate is used to determine the vehicle model, identify the individual vehicle, etc. For example, Patent Document 1 describes a technology for determining the vehicle model by using license plate information read from either the license plate on the front or rear side of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-61506 A Summary of the Invention [Problem to be solved by the invention]

[0004] A system is being considered that identifies individual vehicles using license plate information acquired at the entrance and exit toll gates of a toll road, and charges each vehicle a toll according to the distance it has traveled (the distance from the entrance toll gate to the exit toll gate). However, systems used at conventional toll gates only use license plate information from the front of the vehicle. This makes it difficult to identify that the vehicles arriving at the entrance toll gate and the exit toll gate are the same vehicle if the license plate information is misrecognized at the entrance toll gate or the exit toll gate. This may make it impossible to identify the section the vehicle has traveled and charge the toll.

[0005] The present disclosure has been made in consideration of such problems, and provides an identification device, an identification method, and a program that can suppress a decrease in the accuracy of individual vehicle identification due to erroneous recognition of license plate information. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, an identification device includes a first acquisition unit that acquires, from a first front-view camera and a first rear-view camera at an entrance toll booth, a first front image including a front side license plate of a vehicle that has arrived at the entrance toll booth, and a first rear image including a rear side license plate of the vehicle, respectively; a first image recognition unit that generates first license plate information by integrating character strings of the license plate read from each of the first front-view image and the first rear-view image; and a second image recognition unit that acquires, from a second front-view camera and a second rear-view camera at an exit toll booth, a first image recognition unit that generates first license plate information by integrating character strings of the license plate read from each of the first front-view image and the first rear-view image. the second acquisition unit acquiring a second front image including the front license plate of the vehicle and a second rear image including the rear license plate of the vehicle; a second image recognition unit generating second license plate information by combining license plate character strings read from each of the second front image and the second rear image; and a selection unit selecting, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

[0007] According to one aspect of the present disclosure, the identification method includes the steps of: acquiring a first front image including the front license plate of a vehicle that has arrived at the entrance toll gate, and a first rear image including the rear license plate of the vehicle, from a first front camera and a first rear camera at the entrance toll gate, respectively; generating first license plate information by integrating license plate character strings read from each of the first front image and the first rear image; acquiring a second front image including the front license plate of the vehicle that has arrived at the exit toll gate, and a second rear image including the rear license plate of the vehicle, from a second front camera and a second rear image at the exit toll gate, respectively; generating second license plate information by integrating license plate character strings read from each of the second front image and the second rear image; and selecting, from the first license plate information, first license plate information associated with the same vehicle as the vehicle that has arrived at the exit toll gate, based on a comparison between the first license plate information and the second license plate information.

[0008] According to one aspect of the present disclosure, the program causes a computer of the identification device to execute the following steps: acquiring a first pre-photographed image including the front license plate of a vehicle that has arrived at the entrance toll gate, and a first rear-view image including the rear license plate of the vehicle, from each of a first pre-photographed camera and a first rear-view image including the rear license plate of the vehicle; generating first license plate information by integrating license plate character strings read from each of the first pre-photographed image and the first rear-view image; acquiring a second pre-photographed image including the front license plate of the vehicle that has arrived at the exit toll gate, and a second rear-view image including the rear license plate of the vehicle, from each of a second pre-photographed camera and a second rear-view image including the rear license plate of the vehicle, from each of a second pre-photographed camera and a second rear-view image including the rear license plate of the vehicle, generating second license plate information by integrating license plate character strings read from each of the second pre-photographed image and the second rear-view image; and selecting, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate, based on a comparison between the first license plate information and the second license plate information. Effect of the Invention

[0009] According to the identification device, identification method, and program disclosed herein, it is possible to suppress a decrease in the accuracy of individual vehicle identification due to erroneous recognition of license plate information. [Brief description of the drawings]

[0010] [Figure 1] 1 is a diagram showing an overall configuration of an identification system according to a first embodiment of the present disclosure. [Diagram 2] 1 is a diagram illustrating a functional configuration of a recognition device according to a first embodiment of the present disclosure. [Diagram 3] 4 is a first flowchart showing an example of processing of the identification device according to the first embodiment of the present disclosure. [Figure 4] FIG. 2 is a diagram showing an example of first license plate information according to the first embodiment of the present disclosure. [Diagram 5]11 is a second flowchart showing an example of processing of the identification device according to the first embodiment of the present disclosure. [Figure 6] FIG. 2 is a diagram for explaining a function of the identification device according to the first embodiment of the present disclosure. [Figure 7] FIG. 11 is a diagram for explaining a function of a recognition device according to a second embodiment of the present disclosure. [Figure 8] 13 is a diagram for explaining a function of a recognition device according to a third embodiment of the present disclosure. FIG. [Figure 9] FIG. 13 is a diagram illustrating a functional configuration of an identification device according to a modified example of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] <First embodiment> Hereinafter, a classification system 1 and a classification device 10 according to a first embodiment of the present disclosure will be described with reference to FIGS.

[0012] (Overall composition) FIG. 1 is a diagram showing an overall configuration of an identification system according to a first embodiment of the present disclosure. As shown in Figure 1, the identification system 1 includes a first front camera 21 and a first rear camera 22 installed at the entrance toll gate EN, a second front camera 31 and a second rear camera 32 installed at the exit toll gate EX, and an identification device 10.

[0013] The first pre-photographing camera 21 is installed on a gantry at a predetermined position on lane L1 of the entrance toll gate EN. The first pre-photographing camera 21 photographs the front license plate NP1 of vehicle A traveling on lane L1 from above lane L1. The image photographed by the first pre-photographing camera 21 (first pre-photographed image F1) is transmitted to the identification device 10 via a communication line.

[0014] The first rear camera 22 is installed on a gantry upstream (-X side) in the lane direction from the first front camera 21. The first rear camera 22 captures an image of the rear license plate NP2 of the vehicle A traveling on the lane L1 from above the lane L1. The image captured by the first rear camera 22 (first rear image R1) is transmitted to the identification device 10 via a communication line.

[0015] The second front-view camera 31 is installed on a gantry at a predetermined position on lane L2 at the exit toll gate EX. The second front-view camera 31 photographs the front license plate NP1 of vehicle A traveling on lane L2 from above lane L2. The image captured by the second front-view camera 31 (second front-view image F2) is transmitted to the identification device 10 via a communication line.

[0016] The second rear camera 32 is installed on a gantry upstream (-X side) in the lane direction from the second front camera 31. The second rear camera 32 captures an image of the rear license plate NP2 of the vehicle A traveling on the lane L2 from above the lane L2. The image captured by the second rear camera 32 (second rear image R2) is transmitted to the identification device 10 via a communication line.

[0017] For example, the first front camera 21 and the first rear camera 22 constantly capture images of the lane L1, and when a license plate of the vehicle A is detected from the captured image by a predetermined image processing of the vehicle A, the image is transmitted to the identification device 10. The first front camera 21 and the first rear camera 22 may also capture the first front image F1 and the first rear image R1, for example, when a vehicle detector (not shown) detects that the head or tail of the vehicle A is at a predetermined position on the lane L1. The same applies to the second front camera 31 and the second rear camera 32.

[0018] The identification device 10 collects a first front image F1 and a first rear image R1 from a first front camera 21 and a first rear camera 22 installed at each of a plurality of entrance toll gates EN via a communication line. The identification device 10 also collects a second front image F2 and a second rear image R2 from a second front camera 31 and a second rear camera 32 installed at each of a plurality of exit toll gates EX via a communication line. The identification device 10 identifies each vehicle that arrives at the entrance toll gate EN and the exit toll gate EX based on the license plate information read from the collected images.

[0019] (Functional configuration) FIG. 2 is a diagram illustrating a functional configuration of the identification device according to the first embodiment of the present disclosure. As shown in FIG. 2, the identification device 10 includes a CPU 11, a storage medium 12, an output device 13, an input device 14, and a communication interface (I / F) 15.

[0020] The CPU 11 is a processor that controls the overall operation of the identification device 10. The CPU 11 operates according to a predetermined program to thereby fulfill the functions of a first acquisition unit 110, a first image recognition unit 111, a second acquisition unit 112, a second image recognition unit 113, a selection unit 114, and a processing unit 115.

[0021] The first acquisition unit 110 acquires a first front image F1 including the front license plate NP1 of vehicle A that has arrived at the entrance toll gate EN, and a first rear image R1 including the rear license plate NP2 of vehicle A, from each of the first front camera 21 and the first rear camera 22 at the entrance toll gate EN.

[0022] The first image recognition unit 111 generates first license plate information (first NP information) D1 by integrating the character strings of the license plate read from each of the first pre-taken image F1 and the first post-taken image R1.

[0023] The second acquisition unit 112 acquires a second front image F2 including the front license plate NP1 of vehicle A that has arrived at the exit toll gate EX, and a second rear image R2 including the rear license plate NP2 of vehicle A, from each of the second front camera 31 and the second rear camera 32 at the exit toll gate EX.

[0024] The second image recognition section 113 generates second license plate information (second NP information) D2 by integrating the character strings of the license plate read from each of the second before-shot image F2 and the second after-shot image R2.

[0025] The selection unit 114 selects, from the plurality of pieces of first NP information D1, the first NP information D1 related to the same vehicle as the vehicle that has arrived at the exit toll gate EX, based on a comparison between the first NP information D1 and the second NP information D2.

[0026] The processing unit 115 executes processing for the vehicle A related to the NP information based on the second NP information D2 generated by the second image recognition unit 113 and the first NP information D1 selected by the selection unit 114. For example, the processing unit 115 specifies from which entrance toll gate EN the vehicle A entered the toll road and from which exit toll gate EX the vehicle A exited the toll road from the first NP information D1 and the second NP information D2. Then, the processing unit 115 transmits the identification information (toll gate ID) of the entrance toll gate EN and the first NP information D1, and the identification information (toll gate ID) of the exit toll gate EX and the second NP information D2, as information related to the vehicle A, to a higher-level device (not shown). Based on the information received from the identification device 10, the higher-level device can perform a charging process to charge the vehicle A a toll according to the travel distance of the vehicle A (the distance from the entrance toll gate EN to the exit toll gate EX).

[0027] The storage medium 12 is a so-called auxiliary storage device, and may be, for example, a hard disk drive (HDD) or a solid state drive (SSD). The storage medium 12 stores various data acquired and generated by each unit of the CPU 11. For example, the storage medium 12 stores a first pre-shot image F1, a first post-shot image R1, a second pre-shot image F2, and a second post-shot image R2 collected by the first acquisition unit 110 and the second acquisition unit 112. The storage medium 12 also stores first NP information D1 and second NP information D2 generated by the first image recognition unit 111 and the second image recognition unit 113.

[0028] The output device 13 is a display device that presents various information to the operator of the identification device 10. The output device 13 may also be a printer or the like.

[0029] The input device 14 is an input device such as a keyboard, a mouse, etc., that accepts operations from an operator of the identification device 10. The output device 13 and the input device 14 may be provided integrally as a touch panel.

[0030] The communication I / F 15 transmits and receives data to and from a higher-level device via a wide area communication network such as the Internet.

[0031] (Processing flow) FIG. 3 is a first flowchart illustrating an example of processing of the identification device according to the first embodiment of the present disclosure. Hereinafter, with reference to FIG. 3, a process flow in which the identification device 10 collects the first NP information D1 of a vehicle that has arrived at the entrance toll gate EN will be described.

[0032] When vehicle A arrives at the entrance toll gate EN, the first pre-photographing camera 21 captures an image of the front license plate NP1 of vehicle A. The first acquisition unit 110 of the identification device 10 acquires the first pre-photographed image F1 captured by the first pre-photographing camera 21 (step S100A).

[0033] Next, the first rear camera 22 captures an image of the rear side NP2 of the vehicle A. Then, the first acquisition unit 110 acquires the first rear image R1 captured by the first rear camera 22 (step S100B).

[0034] Next, the first image recognition unit 111 performs a predetermined image processing on the first pre-captured image F1 to read NP information (text information) on the front license plate NP1 of the vehicle A (step S101).

[0035] Moreover, the first image recognition unit 111 reads the NP information (text information) of the rear license plate NP2 of the vehicle A by performing a predetermined image processing on the first rear-view image R1 (step S102).

[0036] The first image recognition unit 111 generates first NP information D1 by integrating the NP information read from each of the first pre-shot image F1 and the first post-shot image R1 (step S103).

[0037] FIG. 4 is a diagram showing an example of the first license plate information according to the first embodiment of the present disclosure. As shown in FIG. 4, the NP information read from the first pre-photographed image F1 and the first post-photographed image R1 includes "place name (maximum 4 characters)", "classification number (maximum 3 characters)", "use code (maximum 1 character)", and "serial number (maximum 4 characters)". The first image recognition unit 111 generates first NP information D1 by combining the NP information of the first pre-photographed image F1 and the NP information of the first post-photographed image R1. In the example of FIG. 4, the first NP information D1 is information consisting of a character string of 24 characters. The first image recognition unit 111 also associates the image acquisition date and time, information (toll gate ID) that can identify the entrance toll gate EN that is the source of the image, and the first pre-photographed image F1 and the first post-photographed image R1 with the generated first NP information D1, and stores and accumulates them in the storage medium 12.

[0038] The classification device 10 executes the series of processes shown in FIG. 3 every time it acquires the first pre-image F1 and the first post-image R1.

[0039] FIG. 5 is a second flowchart showing an example of the process of the identification device according to the first embodiment of the present disclosure. Hereinafter, with reference to FIG. 5, a process flow in which the identification device 10 performs individual identification of a vehicle that has arrived at the exit toll gate EX will be described.

[0040] When vehicle A arrives at the exit toll gate EX, the second pre-photographing camera 31 captures an image of the front license plate NP1 of vehicle A. The second acquisition unit 112 of the identification device 10 acquires the second pre-photographed image F2 captured by the second pre-photographing camera 31 (step S110A).

[0041] Next, the second rear camera 32 captures an image of the rear license plate NP2 of the vehicle A. Then, the second acquisition unit 112 acquires the second rear image R2 captured by the second rear camera 32 (step S110B).

[0042] Next, the second image recognition unit 113 performs a predetermined image processing on the second pre-captured image F2 to read the NP information (text information) on the front license plate NP1 of the vehicle A (step S111).

[0043] Further, the second image recognition unit 113 performs a predetermined image processing on the second rear-view image R2 to read the NP information (text information) of the rear license plate NP2 of the vehicle A (step S112).

[0044] The second image recognition unit 113 generates second NP information D2 by integrating the NP information read from each of the second pre-shot image F2 and the second post-shot image R2 (step S113).

[0045] Next, the selection unit 114 compares the first NP information D1 and the second NP information D2 to identify, among the multiple vehicles that have passed through each entrance toll gate EN, a vehicle identical to vehicle A that has arrived at the exit toll gate EX. Specifically, the selection unit 114 first extracts, from the storage medium 12, candidates for the first NP information D1 that are related to vehicle A identical to the second NP information D2, and determines the degree of agreement between each candidate (step S114).

[0046] FIG. 6 is a diagram for explaining the function of the identification device according to the first embodiment of the present disclosure. In step S113, it is assumed that the second image recognition unit 113 generates the second NP information D2 as shown in Fig. 6. The selection unit 114 extracts the first NP information D1 having the same or similar serial number from the plurality of pieces of first NP information D1 stored in the storage medium 12, for example, using the "serial number" of the pre-shot image included in the second NP information D2 as a search key. The selection unit 114 may use other items such as the "serial number" of the post-shot image, "place name", or "classification number" as a search key, or may use a combination of a plurality of items as a search key. The method by which the selection unit 114 searches for the same or similar information using one or a plurality of search keys can be performed using existing technology, and therefore a description thereof will be omitted.

[0047] In the example of Fig. 6, it is assumed that the selection unit 114 extracts the first NP information D1_1 and the first NP information D1_2. The selection unit 114 obtains the degree of match between the extracted first NP information D1_1 and D1_2. The degree of match is expressed, for example, as the number or percentage of matching characters among a character string of 24 characters included in the first NP information D1 and the second NP information D2. In the example of Fig. 6, the degree of match (for example, the number of matching characters) of the first NP information D1_1 is "23 characters," and the degree of match of the first NP information D1_2 is "20 characters."

[0048] In the example of FIG. 6, it is assumed that the "classification number" of the pre-photographed image for the first NP information D1_1 is erroneously recognized. Therefore, the first NP information D1_1 does not completely match the second NP information D2. In conventional toll road billing systems, it is common to identify individual vehicles based only on the NP information acquired from the pre-photographed image. In that case, as in the example of FIG. 6, if different NP information is recorded at the entrance toll gate and the exit toll gate due to erroneous recognition, it is difficult to identify these as NP information related to the same vehicle.

[0049] Therefore, the selection unit 114 according to this embodiment compares the first NP information D1 and the second NP information D2, which are obtained by integrating the NP information of the pre-captured image and the NP information of the post-captured image into one character string, and selects the first NP information D1 that has the highest degree of match with the second NP information D2 overall. Specifically, the selection unit 114 determines whether the extracted multiple pieces of first NP information D1 can be narrowed down to one that has a degree of match with the second NP information D2 that is equal to or greater than a threshold and is the highest (step S115). The threshold is set arbitrarily depending on the performance of each of the cameras 21, 22, 31, and 32, the recognition accuracy of the first image recognition unit 111 and the second image recognition unit 113, and the like.

[0050] As described above, the first NP information D1 and the second NP information D2 are obtained by integrating the NP information read from the front image and the rear image, respectively, and therefore the amount of information is doubled (24 characters) compared to the conventional method of performing individual identification using only the NP information (12 characters) of one of the front image and the rear image. As a result, even if part of the NP information of the front image is misrecognized as in the example of Fig. 6, as long as the NP information of the rear image matches, the overall degree of match is high and it is less susceptible to the influence of misrecognition.

[0051] 6, the degree of match (e.g., the number of matching characters) of the first NP information D1_1 is "23 characters," and the degree of match of the first NP information D1_2 is "20 characters." In this case, the selection unit 114 can narrow down the information having the highest degree of match equal to or greater than the threshold to only the first NP information D1_1 (step S115: YES). In this case, the selection unit 114 selects this first NP information D1_1 as the first NP information related to the same vehicle as the second NP information D2 generated in step S113 (step S116).

[0052] On the other hand, if there is no first NP information D1 with a degree of matching equal to or greater than the threshold, or if there are multiple pieces of first NP information D1 with a degree of matching equal to or greater than the threshold and the maximum degree of matching (step S115: NO), the selection unit 114 accepts the selection of the first NP information D1 by the operator of the classification device 10 (step S117). For example, the selection unit 114 displays the extracted multiple pieces of first NP information D1, the first pre-photographed image F1 and the first post-photographed image R1 linked to each of the first NP information D1, the second NP information D2, and the second pre-photographed image F2 and the second post-photographed image R2 linked to the second NP information D2, on the output device 13 of the classification device 10. The operator compares each image and NP information, and performs an operation of selecting the first NP information D1 corresponding to the second NP information D2 via the input device 14.

[0053] Furthermore, the processing unit 115 transmits information including the generated second NP information D2 and the selected first NP information D1 to the higher-level device (step S118). The information transmitted to the higher-level device includes the acquisition date and time and the toll gate ID shown in FIG. 6. The higher-level device can charge a toll according to the distance traveled by the vehicle A based on the information received from the identification device 10. Furthermore, the processing unit 115 may delete the first NP information D1 from the storage medium 12 after transmitting the first NP information D1 to the higher-level device.

[0054] The classification device 10 executes the series of processes shown in Fig. 5 every time it acquires the second pre-photographed image F2 and the second post-photographed image R2. Note that in another embodiment, the classification device 10 may store the acquired second pre-photographed image F2 and second post-photographed image R2 in the storage medium 12, and execute the series of processes shown in Fig. 5 for multiple sets of images stored in the storage medium 12 at a predetermined timing (e.g., every hour).

[0055] (Action and effect) As described above, the identification device 10 of this embodiment includes a first acquisition unit 110 that acquires the first pre-captured image F1 and the first rear-captured image R1 from the first pre-captured camera 21 and the first rear-captured camera 22, respectively, at the entrance toll gate EN; a first image recognition unit 111 that generates first NP information D1 by integrating the NP information read from the first pre-captured image F1 and the first rear-captured image R1, respectively; a second acquisition unit 112 that acquires the second pre-captured image F2 and the second rear-captured image R2 from the second pre-captured camera 31 and the second rear-captured camera 32, respectively, at the exit toll gate EX; a second image recognition unit 113 that generates second NP information D2 by integrating the NP information read from the second pre-captured image F2 and the second rear-captured image R2, respectively; and a selection unit 114 that selects from the first NP information D1 the first NP information D1 that relates to the same vehicle as vehicle A that has arrived at the exit toll gate EX based on a comparison between the first NP information D1 and the second NP information.

[0056] As described above, in conventional toll road billing systems, it is common to identify individual vehicles based only on NP information acquired from a front image. In that case, if different NP information is recorded at the entrance toll gate and the exit toll gate due to erroneous recognition, it is difficult to identify these as NP information related to the same vehicle. In contrast, the identification device 10 according to the present embodiment integrates the NP information read from the front image and the rear image at each of the entrance toll gate EN and the exit toll gate EX into one character string and compares them. As a result, even if a part of the NP information of either the front image or the rear image is erroneously recognized, as long as the NP information of the other is consistent, the overall degree of agreement is high and the system is less susceptible to the influence of erroneous recognition. Therefore, the identification device 10 can suppress a decrease in the accuracy of individual vehicle identification due to erroneous recognition of NP information.

[0057] In addition, the selection unit 114 determines the degree of match (the number or percentage of matching characters) between the first NP information D1 and the second NP information D2, and selects the first NP information D1 with the highest degree of match from among the first NP information D1 whose degree of match is equal to or greater than a predetermined threshold.

[0058] By doing this, the identification device 10 can appropriately select, from among the multiple pieces of first NP information D1, the one that is closest to the second NP information D2, even if the first NP information D1 and the second NP information D2 contain misrecognized characters.

[0059] Moreover, the selection unit 114 uses at least one item of the second NP information D2 as a search key, extracts first NP information D1 that matches or is similar to the search key from the multiple pieces of first NP information, and finds the degree of match of the extracted first NP information D1.

[0060] In this way, the identification device 10 can reduce the time required for processing to find the degree of match by extracting in advance the first NP information D1 that is likely to have a high degree of match with the second NP information D2.

[0061] In addition, if there is no first NP information D1 with a degree of similarity equal to or greater than the threshold, or if there are multiple pieces of first NP information D1 with a degree of similarity equal to or greater than the threshold and the highest degree of similarity, the selection unit 114 may present the extracted first NP information D1 to the operator as a candidate.

[0062] This allows the operator to select appropriate first NP information D1 even when there are a large number of erroneously recognized characters.

[0063] <Second embodiment> Next, a description will be given of a recognition system 1 and a recognition device 10 according to a second embodiment of the present disclosure with reference to FIG. The same components as those in the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted. This embodiment is different from the first embodiment in that the selection unit 114 uses weights for each item when determining the degree of coincidence of the first NP information D1.

[0064] The classification device 10 according to the present embodiment is provided in advance with statistical information indicating the misrecognition rate for each item of NP information based on previously read NP information and its correct answer information (for example, NP information input by an operator based on a captured image). The selection unit 114 sets a weight according to the misrecognition rate for each item of NP information based on this statistical information.

[0065] For example, the selection unit 114 sets the weight for each item so that the degree of matching when characters in an item with a low misrecognition rate (less likely to be misrecognized) do not match is lower than the degree of matching when characters in an item with a high misrecognition rate (more likely to be misrecognized) do not match. Alternatively, the selection unit 114 sets the weight for each item so that the degree of matching when characters in an item with a low misrecognition rate match is higher than the degree of matching when characters in an item with a high misrecognition rate match.

[0066] FIG. 7 is a diagram for explaining the function of the identification device according to the second embodiment of the present disclosure. FIG. 7 shows an example of the second NP information D2 generated by the second image recognition section 113 and the first NP information D1 (D1_3, D1_4) extracted by the selection section 114 in steps S113 and S114 in FIG.

[0067] For example, it is assumed that statistical information indicates that the recognition error rate of the "classification number" is equal to or greater than a predetermined value, and that of the "application code" is less than a predetermined value. In this case, the selection unit 114 sets the weights so that the degree of match when the "application code" does not match between the first NP information D1 and the second NP information D2 is lower than the degree of match when the "classification number" does not match. In other words, the selection unit 114 sets the weights for the degree of match so that for items with a high recognition error rate, mismatched characters are permitted, but for items with a low recognition error rate, items with mismatched characters are not selected.

[0068] In the example of Fig. 7, the extracted first NP information D1_3 and D1_4 have the same number of characters as the second NP information D2, i.e., "22 characters." In this case, if the determination is based only on the number of characters that match, it may not be possible to automatically select one of the first NP information D1_3 and D1_4, and confirmation and operation by an operator may be required.

[0069] In contrast, the selection unit 114 according to the present embodiment can automatically select any one of the first NP information D1 by calculating the degree of match using weights for each item. In the example of FIG. 7, the selection unit 114 calculates the degree of match by giving weights to the "classification number" with a high misrecognition rate and the "use code" with a low misrecognition rate. As a result, the first NP information D1_4 with a mismatch in the "use code" has a lower degree of match overall than the first NP information D1_3 with a mismatch in the "classification number". Therefore, the selection unit 114 can select the first NP information D1_3 as the one having the highest degree of match that is equal to or greater than the threshold value.

[0070] (Action and effect) As described above, in the identification device 10 of this embodiment, the selection unit 114 determines the degree of similarity between the first NP information D1 and the second NP information D2 by weighting each of the multiple items contained in the first NP information D1 and the second NP information D2 according to the error recognition rate.

[0071] In this way, the classification device 10 can determine an appropriate degree of matching so that mismatched characters are tolerated for items with a high recognition error rate, but items with mismatched characters are not selected for items with a low recognition error rate. Even if there are multiple pieces of first NP information D1 that match the second NP information D2 in the same number of characters, the classification device 10 can vary the degree of matching by weighting the items, thereby improving the possibility of automatically selecting any one of the first NP information D1.

[0072] <Third embodiment> Next, a description will be given of a recognition system 1 and a recognition device 10 according to a third embodiment of the present disclosure with reference to FIG. The same components as those in the first and second embodiments are denoted by the same reference numerals and will not be described in detail. This embodiment is different from the first and second embodiments in that the selection unit 114 uses a score representing the recognition accuracy for each read character when determining the degree of match of the first NP information D1.

[0073] The first image recognition unit 111 according to this embodiment generates first NP information D1 in which characters on the license plate read from the first pre-photographed image F1 and the first post-photographed image R1 are associated with a score indicating the reliability of recognition of each character in steps S101 to S103 of Fig. 3. Similarly, the second image recognition unit 113 generates second NP information D2 in which characters on the license plate read from the second pre-photographed image F2 and the second post-photographed image R2 are associated with a score for each character in steps S111 to S113 of Fig. 5.

[0074] For example, when the first image recognition unit 111 and the second image recognition unit 113 recognize each character by template matching, they use the similarity to the template as a score. Also, when the first image recognition unit 111 and the second image recognition unit 113 recognize each character by machine learning (including deep learning), they use the reliability of the recognition result as a score.

[0075] If the first NP information D1 and the second NP information D2 are related to the same vehicle A, there is a high possibility that characters with high scores will match. On the other hand, there is a possibility that characters with low scores will not match due to erroneous recognition. For this reason, the selection unit 114 according to the present embodiment sets weights by score so that the degree of match when characters with high scores (low possibility of erroneous recognition) do not match is lower than the degree of match when characters with low scores (high possibility of erroneous recognition) do not match. Alternatively, the selection unit 114 sets weights by score so that the degree of match when characters with high scores match is higher than the degree of match when characters with low scores match. In other words, the selection unit 114 sets different weights for low-score characters and high-score characters so that it allows mismatch for low-score characters that are likely to be erroneously recognized and does not allow mismatch for high-score characters that are unlikely to be erroneously recognized.

[0076] FIG. 8 is a diagram for explaining the function of the identification device according to the third embodiment of the present disclosure. Fig. 8 shows an example of the second NP information D2 generated by the second image recognition unit 113 and the first NP information D1 (D1_5, D1_6) extracted by the selection unit 114 in steps S113 and S114 in Fig. 5. As shown in Fig. 8, a score is assigned to each of the first NP information D1 and the second NP information D2 for each recognized character. The score is, for example, a numerical value between 0 and 1, with 1 indicating the highest reliability and 0 indicating the lowest reliability.

[0077] For example, the selection unit 114 multiplies the scores of corresponding characters in the first NP information D1 and the second NP information D2 to obtain a weight. Specifically, when the recognition results match, the selection unit 114 obtains a positive weight by multiplying the score, and when the recognition results do not match, the selection unit 114 obtains a negative weight by multiplying the score. Hereinafter, a method of calculating the score weight will be described in detail with reference to the example in FIG. 8.

[0078] (1) If both scores are high and the recognition results match The recognition results ("a") of the usage codes CH53 and CH23 of the pre-shot images of the first NP information D1_5 and the second NP information D2 match, and both have a high score ("1"). In this case, the weight of the score of the usage code of the first NP information D1_5 is "+1", which is a positive value obtained by multiplying the score of CH53 ("1") and the score of CH23 ("1").

[0079] (2) If both scores are high and the recognition results do not match The recognition result of the usage code CH60 of the pre-shot image of the first NP information D1_6 ("O") does not match the recognition result of the usage code CH23 of the second NP information D2 ("A"). Furthermore, both of these scores are high values ​​("1"). In this case, the weight of the score of the usage code of the first NP information D1_6 is "-1", which is the negative value obtained by multiplying the score of CH63 ("1") and the score of CH23 ("1").

[0080] (3) If one score is lower and the recognition results match Regarding the classification numbers of the pre-shot images of the first NP information D1_6 and the second NP information D2, the recognition results ("3") of the first character CH60 and CH20 match. Furthermore, the score of the first NP information D1_6 (CH60) is "0.5", and the score of the second NP information D2 (CH20) is "1". In other words, one of the scores is higher and the other is lower. In this case, the weight of the score of the classification number (first character) of the first NP information D1_6 is "+0.5", which is a positive value obtained by multiplying the score of CH60 ("0.5") and the score of CH20 ("1").

[0081] (4) If one of the scores is lower and the recognition results do not match Regarding the classification number of the pre-shot image of the first NP information D1_5, the recognition result of the first character CH50 ("8") does not match the recognition result of the corresponding character CH20 of the first NP information NP ("3"). In addition, the score of the first NP information D1_5 (CH50) is "0.5", and the score of the second NP information D2 (CH20) is "1". In other words, one of the scores is higher and the other is lower. In this case, the weight of the score of the classification number (first character) of the first NP information D1_5 is "-0.5", which is a negative value obtained by multiplying the score of CH50 ("0.5") and the score of CH20 ("1").

[0082] (5) If both scores are low and the recognition results match Regarding the classification numbers of the pre-shot images of the first NP information D1_6 and the second NP information D2, the recognition results ("1") of the second characters CH61 and CH21 match. Furthermore, the scores of these characters CH61 and CH21 are both low values ​​("0.5"). In this case, the weight of the score of the classification number (second character) of the first NP information D1_6 is "+0.25", which is a positive value obtained by multiplying the score of CH61 ("0.5") and the score of CH21 ("0.5").

[0083] (6) If both scores are low and the recognition results do not match Regarding the classification number of the pre-shot image of the first NP information D1_5, the recognition result of the second character CH51 ("0") does not match the recognition result of the corresponding character CH21 of the first NP information NP ("1"). Furthermore, the scores of these characters CH51 and CH21 are both low values ​​("0.5"). In this case, the weight of the score of the classification number (second character) of the first NP information D1_5 is "-0.25", which is the negative value obtained by multiplying the score of CH51 ("0.5") and the score of CH21 ("0.5").

[0084] In this way, the selection unit 114 calculates the weight of each character based on whether the recognition results match for each corresponding character in the first NP information D1 and the second NP information D2 and the score of the recognition results, and obtains the sum of these as the degree of match. Considering the weight, the highest degree of match is "24" (all 12 characters of NP information in the front image + all 12 characters of NP information in the rear image match), and the lowest value is "-24" (all 12 characters of NP information in the front image + all 12 characters of NP information in the rear image do not match). As a result, the degree of match between characters with high recognition results scores is significantly reduced when the recognition results do not match. In addition, for characters with low recognition results scores, the effect on the degree of match when the recognition results do not match is small. In this way, it is possible to reduce the possibility that the NP information of the same vehicle will not be selected appropriately due to a decrease in recognition accuracy caused by dirt on the license plate, etc.

[0085] The selection unit 114 selects the first NP information D1 whose degree of coincidence is a predetermined threshold (for example, 20 or more) and is the largest. In the example of Fig. 8, the degree of coincidence of the second NP information D2 with the first NP information D1_5 is "21.25", and the degree of coincidence with the first NP information D1_6 is "18.75". Therefore, the selection unit 114 selects the first NP information D1_5.

[0086] In the present embodiment, an example in which the weight is calculated for each character included in each of the first NP information D1 and the second NP information D2 has been described, but the present invention is not limited to this. In other embodiments, the weight may be calculated for each character string instead of for each character. For example, since a place name, such as "Kobe" or "Himeji", is a place name in which a plurality of characters (character string) as a whole express one meaning (place name), it is more reasonable to compare the entire character string rather than comparing each character one by one. For this reason, the selection unit 114 treats the four characters included in the place name as one character string and calculates only one weight for the entire character string. At this time, the selection unit 114 sets the average value of the scores of the characters of the place name as the score of the entire character string. The selection unit 114 calculates the weight of the character string indicating the place name based on whether or not the character strings indicating the place name in the first NP information D1 and the second NP information D2 match, and on the score of the character string. In this case, the degree of match of the NP information is in the range of "-18" to "+18".

[0087] In still another embodiment, the selection unit 114 may determine the minimum or maximum value of the scores of the individual characters in the place name as the score of the entire character string indicating the place name.

[0088] (Action and effect) As described above, in the classification device 10 according to this embodiment, the selection unit 114 determines the degree of similarity between the first NP information D1 and the second NP information D2 by weighting according to the score of each of the characters contained in the first NP information D1 and the second NP information D2.

[0089] In this way, the classification device 10 can determine an appropriate degree of matching so that, for low-score characters that may be misrecognized, they do not match the corresponding characters in the second NP information D2, but for high-score characters that are less likely to be misrecognized, characters that do not match the corresponding characters in the second NP information D2 are not selected. Furthermore, even if there are multiple pieces of first NP information D1 that match the second NP information D2 in the same number of characters, the classification device 10 can vary the degree of matching by weighting according to the score, thereby improving the possibility of automatically selecting any one of the first NP information D1.

[0090] In addition, in the present embodiment, the first image recognition unit 111 and the second image recognition unit 113 have been described in a mode of splitting and recognizing each character of the license plate number and attaching a score to each character. However, the present invention is not limited to this. In other embodiments, the first image recognition unit 111 and the second image recognition unit 113 may recognize a specific item as a character string without splitting it character by character. For example, in the example of FIG. 8, the first image recognition unit 111 and the second image recognition unit 113 split and recognize the character string "Kobe" representing the place name into two characters, "神" and "戸", and attach scores to each of these two characters. However, in other embodiments, "Kobe" may be recognized as a single character string without splitting, and only one score may be attached to the entire character string.

[0091] In each of the above-described embodiments, the processes of various processes of the identification device 10 are stored in a computer-readable recording medium in the form of a program, and the above-described various processes are performed by the computer reading and executing this program. The computer-readable recording medium refers to a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Further, this computer program may be distributed to the computer via a communication line, and the computer that has received this distribution may execute the program.

[0092] The above program may be for realizing a part of the above-described functions. Further, it may be a so-called difference file (difference program) that can be realized in combination with a program already recorded in the computer system for the above-described functions.

[0093] As described above, several embodiments according to the present disclosure have been described, but all of these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and gist of the invention.

[0094] For example, in the classification system 1 according to each embodiment described above, the classification device 10 has the functions of the first acquisition unit 110, the first image recognition unit 111, the second acquisition unit 112, and the second image recognition unit 113, but this is not limiting. For example, as in the modified example described below, these functional units may be provided in a device other than the classification device 10.

[0095] <Modification> FIG. 9 is a diagram illustrating a functional configuration of a discrimination device according to a modified example of the present disclosure. As shown in Figure 9, the identification system 1 of this modified example further includes a first number plate recognition device (first NP recognition device) 20 installed at the entrance toll gate EN, and a second number plate recognition device (second NP recognition device) 30 installed at the exit toll gate EX.

[0096] The first NP recognition device 20 is a device for generating first NP information D1 of a vehicle A that has arrived at the entrance toll booth EN. The first NP recognition device 20 includes a first front camera 21 and a first rear camera 22, which are the same as those in the first embodiment, and a CPU 23.

[0097] The CPU 23 is a processor that controls the overall operation of the first NP recognition device 20. The CPU 23 operates according to a predetermined program to fulfill the functions of a first acquisition unit 230 and a first image recognition unit 231. The functions of the first acquisition unit 230 and the first image recognition unit 231 are similar to the functions of the first acquisition unit 110 and the first image recognition unit 111 according to the first to third embodiments, respectively.

[0098] The second NP recognition device 30 is a device for generating second NP information D2 of the vehicle A that has arrived at the exit toll gate EX. The second NP recognition device 30 includes the second front camera 31 and the second rear camera 32, which are the same as those in the first embodiment, and a CPU 33.

[0099] The CPU 33 is a processor that controls the overall operation of the second NP recognition device 30. The CPU 33 operates according to a predetermined program to fulfill the functions of a second acquisition unit 330 and a second image recognition unit 331. The functions of the second acquisition unit 330 and the second image recognition unit 331 are similar to the functions of the second acquisition unit 112 and the second image recognition unit 113 according to the first to third embodiments, respectively.

[0100] Moreover, the CPU 11 of the identification device 10 according to this embodiment further includes a license plate information acquisition unit (NP information acquisition unit) 116. The NP information acquisition unit 116 acquires the first NP information D1 from the first NP recognition device 20, and acquires the second NP information D2 from the second NP recognition device 30. The functions of the selection unit 114 and the processing unit 115 of the identification device 10 are similar to those of the above-mentioned embodiments.

[0101] Even if the identification system 1 has a device configuration like that of this modified example, it can obtain the same effects as those of the above-mentioned embodiments.

[0102] <Additional Notes> The identification device, the identification method, and the program described in the above-described embodiments can be understood, for example, as follows.

[0103] According to a first aspect of the present disclosure, an identification device (10) includes a first acquisition unit (110, 230) that acquires a first front image including a front license plate of a vehicle that has arrived at an entrance toll gate and a first rear image including a rear license plate of the vehicle from a first front camera (21) and a first rear camera (22) at the entrance toll gate, respectively, a first image recognition unit (111, 231) that generates first license plate information by integrating character strings of the license plate read from each of the first front image and the first rear image, and an exit toll gate from a second front camera (31) and a second rear camera (32) at the exit toll gate, respectively. the second acquisition unit (112, 330) that acquires a second front image including the front license plate of a vehicle that has arrived at the exit toll gate and a second rear image including the rear license plate of the vehicle, a second image recognition unit (113, 331) that generates second license plate information by integrating license plate character strings read from each of the second front image and the second rear image, and a selection unit (114) that selects, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

[0104] The identification device compares the NP information read from the front and rear images at the entrance toll gate EN and the exit toll gate EX as a single character string, so that even if a portion of the NP information in either the front or rear image is misrecognized, the overall degree of match is high and the device is less susceptible to the influence of misrecognition as long as the NP information in the other image matches. Therefore, the identification device can suppress a decrease in the accuracy of individual vehicle identification caused by misrecognition of NP information.

[0105] According to a second aspect of the present disclosure, in the identification device (10) relating to the first aspect, the selection unit (114) determines the degree of match between the first license plate information and the second license plate information, and selects the first license plate information with the highest degree of match from among the first license plate information whose degree of match is equal to or greater than a predetermined threshold.

[0106] In this way, the identification device can appropriately select, from among multiple pieces of first license plate information, the one that is closest to the second license plate information, even if the first license plate information and the second license plate information contain misrecognized characters.

[0107] According to a third aspect of the present disclosure, in the identification device (10) relating to the second aspect, the selection unit (114) determines the degree of match by weighting each of a plurality of items included in the first license plate information and the second license plate information according to the error recognition rate.

[0108] In this way, the identification device can determine an appropriate degree of match so that for items with a high rate of misrecognition, mismatched characters are tolerated, but for items with a low rate of misrecognition, items with mismatched characters are not selected. Furthermore, even if there are multiple pieces of first license plate information that have the same number of characters as the second license plate information, the identification device can differentiate the degree of match by weighting each item, thereby improving the possibility of automatically selecting any one of the first license plate information.

[0109] According to a fourth aspect of the present disclosure, in the identification device (10) relating to the first aspect, the first license plate information and the second license plate information further include a score representing the reliability of recognition of each character of the read license plate, and the selection unit (114) determines the degree of match by applying a weight according to the score.

[0110] In this way, the identification device can determine an appropriate degree of match so that low-score characters that may be misrecognized are acceptable even if they do not match the corresponding characters in the second license plate information, but high-score characters that are unlikely to be misrecognized are not selected that do not match the corresponding characters in the second license plate information. Furthermore, even if there are multiple first license plate information that have the same number of characters as the second license plate information, the identification device can differentiate the degrees of match by weighting according to the scores, thereby improving the possibility of automatically selecting any one of the first license plate information.

[0111] According to a fifth aspect of the present disclosure, the identification method includes the steps of acquiring a first front image including a front license plate of a vehicle arriving at the entrance toll gate and a first rear image including a rear license plate of the vehicle from a first front camera (21) and a first rear camera (22) at the entrance toll gate, respectively; generating first license plate information by integrating character strings of the license plate read from the first front image and the first rear image, respectively; acquiring the exit toll plate information from a second front camera (31) and a second rear camera (32) at the exit toll gate, respectively; The method includes the steps of: acquiring a second pre-captured image including the front license plate of a vehicle arriving at the exit toll gate; generating second license plate information by combining license plate character strings read from the second pre-captured image and the second rear-captured image; and selecting, from the first license plate information, first license plate information associated with the same vehicle as the vehicle that arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information.

[0112] According to a sixth aspect of the present disclosure, the program includes the steps of: acquiring, from a first front-view camera (21) and a first rear-view camera (22) at an entrance toll booth, a first front-view image including a front license plate of a vehicle that has arrived at the entrance toll booth, and a first rear-view image including a rear license plate of the vehicle; generating first license plate information by integrating character strings of the license plate read from each of the first front-view image and the first rear-view image; and acquiring, from a second front-view camera (31) and a second rear-view camera (32) at an exit toll booth, a first front-view image including a front license plate of a vehicle that has arrived at the exit toll booth. The method causes a computer of the identification device (10) to execute the steps of: acquiring a second front-view image including the front license plate of the vehicle and a second rear-view image including the rear license plate of the vehicle; generating second license plate information by combining the license plate character strings read from each of the second front-view image and the second rear-view image; and selecting, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information. [Explanation of symbols]

[0113] 1 Identification System 10 Identification device 11 CPU 110 First acquisition part 111 First Image Recognition Unit 112 Second Acquisition Department 113 Second Image Recognition Unit 114 Selection section 115 Processing section 116 License plate information acquisition unit (NP information acquisition unit) 12 Storage medium 13 Output Devices 14 Input Devices 20 First license plate recognition device (first NP recognition device) 21 First front camera 22 1st Rear View Camera 23 CPU 230 First acquisition part 231 First Image Recognition Unit 30 Second license plate recognition device (second NP recognition device) 31 Second front camera 32 2nd rear camera 33 CPU 330 Second Acquisition Department 331 Second Image Recognition Unit

Claims

1. a first acquisition unit that acquires a first front image including a front side license plate of a vehicle that has arrived at the entrance toll gate and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll gate, respectively; A first image recognition unit that generates first license plate information, which is a character string having twice the number of characters of the license plate by concatenating character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image, and a second acquisition unit that acquires, from a second front image including a front side license plate of a vehicle that has arrived at the exit toll gate and a second rear image including a rear side license plate of the vehicle, from a second front image camera and a second rear image camera of the exit toll gate, respectively; A second image recognition unit that generates second license plate information, which is a character string having twice the number of characters of the license plate by concatenating character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image, and a selection unit that selects, from the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate, based on a comparison between the first license plate information and the second license plate information; An identification device comprising:

2. The selection unit obtains a degree of match between the first license plate information and the second license plate information, and selects the first license plate information having the highest degree of match from among the first license plate information whose degree of match is equal to or greater than a predetermined threshold. The identification device according to claim 1 .

3. the selection unit determines the degree of match by weighting the items included in the first license plate information and the second license plate information according to a misrecognition rate of each of the items. The identification device according to claim 2 .

4. The first license plate information and the second license plate information further include a score representing a degree of confidence in the recognition of each character of the license plate read; The selection unit determines the degree of match by weighting the scores. The identification device according to claim 2 or 3.

5. acquiring a first front image including a front side license plate of a vehicle arriving at the entrance toll booth and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll booth, respectively; A step of generating first license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image; acquiring a second front image including a front side license plate of a vehicle arriving at the exit toll booth and a second rear image including a rear side license plate of the vehicle from a second front image camera and a second rear image camera at the exit toll booth, respectively; A step of generating second license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image; selecting, from among the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information; The identification method has the following features:

6. acquiring a first front image including a front side license plate of a vehicle arriving at the entrance toll booth and a first rear image including a rear side license plate of the vehicle from a first front image camera and a first rear image camera at the entrance toll booth, respectively; A step of generating first license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the first pre-photographed image and the first post-photographed image; acquiring a second front image including a front side license plate of a vehicle arriving at the exit toll booth and a second rear image including a rear side license plate of the vehicle from a second front image camera and a second rear image camera at the exit toll booth, respectively; A step of generating second license plate information which is a character string having twice the number of characters of the license plate by concatenating the character strings of the license plate read from each of the second pre-photographed image and the second post-photographed image; selecting, from among the first license plate information, first license plate information related to the same vehicle as the vehicle that has arrived at the exit toll gate based on a comparison between the first license plate information and the second license plate information; A program for causing a computer of an identification device to execute the above.

Images