Automobile name recognition device, automobile name recognition method, and automobile name recognition program

Abstract

To crop an image under uniform conditions from an image in which a target vehicle appears in a technology that recognizes the name of the target vehicle in the image using machine learning.SOLUTION: An automobile name recognition device 100 includes a pixel count detection unit 133 and an automobile name estimation unit 134. The pixel count detection unit 133 detects the number of horizontal direction pixels of a license plate detected from a target image and the number of vertical direction pixels of the license plate. The automobile name estimation unit 134 crops an image from the target image within a range that satisfies the cropping conditions which are the conditions indicating that the image is cropped so that each portion of the vehicle comprising a group of feature portions comprising at least one portion of the vehicle having a feature that is a clue to identifying the automobile name is included, and indicating a range determined according to an edge with a length that is a horizontal constant times the number of horizontal pixels along a horizontal direction and an edge with a length that is a vertical constant times the number of vertical pixels along a vertical direction.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a vehicle name recognition device, a vehicle name recognition method, and a vehicle name recognition program. [Background technology]

[0002] In the vehicle name recognition process for recognizing the name of a vehicle shown in an image, recognition processing using machine learning is performed. Specifically, in the vehicle name recognition process using machine learning, an image showing a part of the target vehicle is cut out, machine learning is performed using the cut-out image, and the name of the target vehicle is recognized using the results of the machine learning. Patent Document 1 discloses a technique for estimating the type of vehicle shown in an image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6900448 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 has a problem that images are not extracted under uniform conditions from images showing target vehicles. Here, if each image included in the training data used for machine learning is not extracted under uniform conditions, when an inference model is generated by learning using the training data, the noise included in each image is also learned, resulting in a so-called "over-learning (= over-fitting)" state. Therefore, in this case, variance (variation error) increases in predictions using the inference model, resulting in a problem of a decrease in the accuracy rate of car name recognition. The present disclosure aims to extract an image under uniform conditions from an image showing a target vehicle in a technology that uses machine learning to recognize the name of a target vehicle shown in an image. [Means for solving the problem]

[0005] The vehicle name recognition device according to the present disclosure comprises: a pixel number detection unit that detects a horizontal pixel number, which is the number of pixels in the horizontal direction of the license plate detected from a target image showing the license plate of a target vehicle, and a vertical pixel number, which is the number of pixels in the vertical direction of the detected license plate; a vehicle name estimation unit that cuts out an image within a range that satisfies a cutout condition, the cutout condition being a condition indicating a range based on the position of the detected license plate, the condition indicating that the image should be cut out so as to include each vehicle part constituting a characteristic part group consisting of at least one vehicle part having a feature that serves as a clue to identifying the name of the vehicle shown in the image, the cutout condition being a condition indicating a range determined according to a side having a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction; and A vehicle name recognition device comprising: Each of the horizontal direction and the vertical direction is a direction that is determined according to the detected license plate. Effect of the Invention

[0006] According to the present disclosure, the vehicle name estimation unit cuts out an image in a range that satisfies the cut-out condition from the target image. Here, the cut-out condition is a condition indicating a range based on the position of the detected license plate, and is a condition indicating that the image is cut out so as to include each vehicle part constituting a characteristic part group consisting of at least one vehicle part having a feature that serves as a clue to identifying the vehicle name of the vehicle shown in the image, and is a condition indicating a range determined according to a side having a length equal to a constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a constant multiple of the number of vertical pixels along the vertical direction. The number of horizontal pixels is the number of pixels in the horizontal direction of the detected license plate. The number of vertical pixels is the number of pixels in the vertical direction of the detected license plate. Therefore, according to the present disclosure, in a technology for recognizing the vehicle name of a target vehicle shown in an image using machine learning, an image can be cut out under uniform conditions from an image in which the target vehicle is shown. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of the configuration of a vehicle name recognition device 100 according to a first embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a characteristic portion group according to the first embodiment. [Diagram 3] FIG. 1 is a diagram showing an example of a vehicle name recognition device 100 according to a first embodiment. [Figure 4] 5A and 5B are diagrams illustrating a vehicle name estimation process according to the first embodiment, in which FIG. 5A illustrates a process for preparing dictionary subspace data, and FIG. 5B illustrates a process for estimating a vehicle name. [Diagram 5] FIG. 1 is a diagram for explaining a problem associated with the conventional technology. [Figure 6] FIG. 4 is a diagram for explaining extraction conditions according to the first embodiment. [Figure 7] FIG. 2 is a diagram showing an example of the hardware configuration of a recognition unit 130 according to the first embodiment. [Figure 8] 3 is a flowchart showing the operation of the vehicle name recognition device 100 according to the first embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a hardware configuration of a recognition unit 130 according to a modification of the first embodiment. [Figure 10] 11A and 11B are diagrams for explaining the cut-out range according to the second embodiment, where (a) is a diagram for explaining the problems associated with the conventional technology, (b) is a diagram showing the cut-out range corresponding to a normal passenger car, and (c) is a diagram showing the cut-out range corresponding to a large vehicle. [Figure 11] FIG. 11 is a diagram showing an example of the configuration of a vehicle name recognition device 100 according to a second embodiment. [Figure 12] FIG. [Figure 13] 11A and 11B are diagrams for explaining the cut-out conditions according to the second embodiment, where (a) is a diagram showing the cut-out range corresponding to a light vehicle, (b) is a diagram showing the cut-out range corresponding to a compact passenger car, and (c) is a diagram showing the cut-out range corresponding to a normal passenger car. [Figure 14] 10 is a flowchart showing the operation of the vehicle name recognition device 100 according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. Descriptions of elements given the same reference numerals are omitted or simplified as appropriate. Arrows in the drawings primarily indicate data flow or processing flow. In addition, "part" may be read as "circuit," "step," "procedure," "processing," or "circuitry" as appropriate.

[0009] Embodiment 1 Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0010] ***Configuration Description*** 1 shows an example of the configuration of a vehicle name recognition device 100 according to this embodiment. As shown in FIG. 1, the vehicle name recognition device 100 includes an illumination unit 110, a camera unit 120, and a recognition unit .

[0011] The illumination unit 110 is a device that emits light, and includes, as a specific example, a pulse drive circuit and an LED (Light Emitting Diode). When the illumination unit 110 includes an LED, the illumination unit 110 irradiates the LED light onto a traveling vehicle.

[0012] The camera unit 120 is a device having a function of capturing an image of a traveling vehicle, and specifically includes an optical filter, a light receiving lens, a light receiving element, and a camera signal processing circuit. The camera unit 120 may be a device that captures moving images.

[0013] The recognition unit 130 includes a timing control unit 131, a license plate detection unit 132, a pixel number detection unit 133, and a vehicle name estimation unit 134.

[0014] The timing control unit 131 controls the operation timing of each unit included in the vehicle name recognition device 100 .

[0015] The license plate detection unit 132 has a function of detecting the license plate of a vehicle from an image captured by the camera unit 120. This image is also called a target image. The license plate of the target vehicle is captured in the target image.

[0016] The pixel number detection unit 133 has a function of detecting the number of pixels of the license plate detected by the license plate detection unit 132 . Specifically, the pixel number detection unit 133 detects the number of horizontal pixels and the number of vertical pixels. The number of horizontal pixels is the number of pixels in the horizontal direction of the license plate detected from the target image. The number of vertical pixels is the number of pixels in the vertical direction of the license plate detected from the target image. Each of the vertical and horizontal directions is a direction in the image captured by the camera unit 120, and is a direction that is appropriately determined according to the license plate detected by the license plate detection unit 132. As a specific example, the horizontal direction is a direction along the bottom side of the detected license plate, and the vertical direction is a direction along the left side of the detected license plate. The vertical and horizontal directions do not have to be perpendicular to each other.

[0017] When the target image can be cut out according to the cut-out conditions, the vehicle name estimation unit 134 cuts out at least a part of the target image as a target cut-out image according to the cut-out conditions. The vehicle is typically a moving body having a license plate on the front, and specific examples are a light vehicle, a normal passenger car, a medium-sized car, or a large car. The front is a surface that exists in front of the moving body in the traveling direction. As a specific example, the front of the target vehicle is shown in the target image. The cut-out condition is a condition indicating a range based on the position of the license plate detected by the license plate detection unit 132, a condition indicating a range determined according to a side having a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction, and a condition indicating that an image is cut out so as to include each vehicle part constituting the characteristic part group. The cut-out condition may be a condition indicating that a range in which the license plate of the vehicle is shown is cut out. The cut-out condition may be a condition indicating that a range in which vehicle parts other than the vehicle parts constituting the characteristic part group are not included as much as possible. Note that the cut-out condition does not have to be a condition indicating that an image of a range including a range in which the license plate of the vehicle is shown is cut out. The side along the horizontal direction does not have to be a side that runs completely along the horizontal direction. The side along the vertical direction does not have to be a side that runs completely along the vertical direction. The group of characteristic parts is composed of at least one vehicle part having a feature that is a clue for identifying the name of the vehicle. Each part of the vehicle that constitutes the group of characteristic parts and is shown in the image is a part having a feature that is a clue for identifying the name of the vehicle shown in the image, and is a part that corresponds to a feature amount used by the inference model. As a specific example, each part is at least one of an emblem, a front grille, a corner molding, a headlight, a bonnet, and a front turn signal, as shown in FIG. 2. As a specific example, the vehicle name estimation unit 134 estimates the vehicle name using the feature amounts corresponding to these parts. The processing in the number plate detection unit 132 and the processing in the pixel number detection unit 133 correspond to pre-processing.

[0018] The vehicle name estimation unit 134 recognizes the name of a target vehicle shown in the image by using data representing the target cut-out image and an inference model. The inference model is a model trained using training data consisting of data indicating pairs of each image cut out according to the same cut-out condition as the cut-out condition followed by the vehicle name estimation unit 134 and the vehicle name of the vehicle shown in each cut-out image. That is, the inference model is a model that receives as input data indicating an image showing a range that satisfies the cut-out condition, and estimates the vehicle name of the vehicle shown in the image corresponding to the input. Here, the vehicle name corresponds to a class in machine learning. The vehicle name estimation unit 134 may train the inference model using training data consisting of data indicating each image cut out according to the cut-out condition and data indicating the vehicle name corresponding to each image.

[0019] 3 shows an embodiment of the vehicle name recognition device 100. In this embodiment, the vehicle name recognition device 100 is installed on or along a road. The lighting unit 110 irradiates light toward a vehicle traveling on the road, and the irradiated light is reflected by the vehicle, causing the reflected light to be irradiated toward the vehicle name recognition device 100. The irradiated reflected light is received by the camera unit 120.

[0020] Fig. 4 is a diagram for explaining a specific example of a vehicle name estimation process, in which Fig. 4(a) is a diagram for explaining a process for preparing dictionary subspace data, and Fig. 4(b) is a diagram for explaining a process for estimating a vehicle name.

[0021] 4A shows how dictionary subspace data is prepared in advance by the vehicle name estimation unit 134. The dictionary subspace data is data that represents a plurality of vehicle names. As a specific example, first, the vehicle name estimation unit 134 creates subspace data corresponding to each vehicle name by using the feature amount of the cut-out image corresponding to each vehicle name. Next, the vehicle name estimation unit 134 performs clustering on the created subspace data using the k-means method. Here, the related vehicle name group consists of the vehicle names of multiple vehicles that are similar in shape. As a specific example, the related vehicle name group consists of the vehicle names of a series of models that belong to a series defined by a certain automobile manufacturer. Next, the vehicle name estimation unit 134 calculates the center of gravity of each cluster as the representative point of each cluster. As a specific example, the dictionary subspace data includes subspace data corresponding to each feature amount, each cluster, and data indicating a representative point of each cluster. Note that the dictionary subspace data does not have to be prepared by the vehicle name estimation unit 134.

[0022] FIG. 4B shows how the vehicle name estimation unit 134 estimates a vehicle name using a machine learning pattern recognition method. The vehicle name estimation unit 134 picks up features for each image to generate a feature vector, identifies which class (vehicle name) the generated feature vector corresponds to, and classifies the feature vector for each identified class. In this case, the vehicle name estimation unit 134 captures fine features such as headlights, a front grille, front blinkers, and an emblem by extracting features related to the object shape in the image using brightness gradient information such as HOG (Histograms of Oriented Gradients), as a specific example. A specific example of the pattern recognition method is a method in which, like the linear subspace method, which is a representative pattern recognition method, dictionary subspace data expressing a plurality of predetermined vehicle names (classes) is prepared, a feature amount corresponding to a cutout image for vehicle name recognition in which a part of a traveling vehicle is shown is calculated, and a vehicle name corresponding to a subspace having a relatively large similarity to the calculated feature amount is output. The vehicle name estimation unit 134 is, as a specific example, a classifier that realizes the pattern recognition method, that is, a classifier that classifies input data based on the relative magnitude of the similarity. As a specific example, first, the vehicle name estimation unit 134 calculates HOG features from a cut-out image for vehicle name recognition. Next, the vehicle name estimation unit 134 calculates the similarity between the calculated HOG feature and the "subspace representing the center of gravity of each cluster." Here, the cluster with the largest similarity is determined as the estimated cluster. In the diagram shown in the center of FIG. 4(b), each number indicates the similarity. Next, the vehicle name estimation unit 134 calculates the similarity with the HOG feature amount for only the subspace belonging to the estimated cluster, and outputs the final vehicle name based on the similarity calculation result.

[0023] Fig. 5 is a diagram for explaining the conventional problem. As a specific example, when an image for vehicle name recognition is cut out with a certain number of pixels according to a nearby vehicle within the angle of view of the camera unit 120 as shown in Fig. 5, a vehicle existing far away within the angle of view of the camera unit 120 appears relatively small, so that objects such as road markings that are unrelated to the running vehicle appear in the image. Therefore, in this case, even noise in the training data is learned, and a problem occurs in that a learning model with large variance (variation error) is generated. On the other hand, when an image for vehicle name recognition is cut out with a certain number of pixels according to distant vehicles within the angle of view of the camera unit 120, vehicles existing close to the angle of view of the camera unit 120 are reflected relatively large, so it is not possible to extract headlights and front mask parts, etc., whose vehicle body shapes differ depending on the vehicle name. Therefore, in this case, the feature amount of the vehicle body cannot be appropriately extracted, and there is too little variation (too little difference in features) between classes (vehicle names), which causes a problem that the accuracy rate of vehicle name recognition using an inference model decreases. Here, the vehicle is photographed larger as the fixed zoom setting value of the camera unit 120 is set to zoom, and the vehicle is photographed smaller as the setting value is set to wide-angle. Therefore, the same problem may occur due to differences in the fixed zoom setting value of the camera unit 120.

[0024] In this embodiment, at least a part of an image of a traveling vehicle is cut out based on the position coordinates of the license plate. Fig. 6 is a diagram for explaining image cutout based on the position of the license plate detected by license plate detection unit 132. 6 shows the license plate detected by license plate detection unit 132, the number of pixels detected by pixel number detection unit 133, and the range cut out by vehicle name estimation unit 134. X indicates the number of pixels in the horizontal direction, i.e., the number of horizontal pixels. Y indicates the number of pixels in the vertical direction, i.e., the number of vertical pixels. Each of X and Y is the number of pixels detected by pixel number detection unit 133. a is a constant indicating the magnification in the horizontal direction. b is a constant indicating the magnification in the vertical direction. Each of a and b may be set arbitrarily and may be changed arbitrarily. In FIG. 6, the cutout condition is a condition indicating that an image is cut out within a range of Y×b pixels above the origin and X×a pixels to the right of the origin in a coordinate system with the lower left corner of the license plate as the origin. The vehicle name estimation unit 134 cuts out an image for vehicle name recognition based on the license plate size, which is a fixed length, so that the cutout image for vehicle name recognition can be cut out under uniform conditions. The side with a length of X×a pixels corresponds to a side with a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction. The side with a length of Y×b pixels corresponds to a side with a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction. Even if a license plate is detected in the target image, the vehicle name estimation unit 134 does not need to cut out the image from the target image if the entire range specified by the cut-out condition is not shown in the image. In addition, although a specific example has been described in which an image of a vehicle taken from the front of the vehicle is used, the vehicle name recognition device 100 may estimate the vehicle name of the vehicle using an image taken from the rear of the vehicle when a license plate is installed at the rear of the vehicle. The case where an image taken from the rear of the vehicle is used is similar to the case where an image taken from the front of the vehicle is used.

[0025] 7 shows an example of a hardware configuration of the recognition unit 130 according to this embodiment. The recognition unit 130 is made up of a computer. The recognition unit 130 may be made up of multiple computers.

[0026] As shown in the figure, the recognition unit 130 is a computer including hardware such as a processor 11, a memory 12, an auxiliary storage device 13, an input / output IF (Interface) 14, and a communication device 15. These pieces of hardware are appropriately connected via signal lines 19.

[0027] The processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), and a graphics processing unit (GPU). The recognition unit 130 may include a plurality of processors that replace the processor 11. The plurality of processors share the role of the processor 11.

[0028] The memory 12 is typically a volatile storage device, and a specific example is a RAM (Random Access Memory). The memory 12 is also called a primary storage device or a main memory. Data stored in the memory 12 is saved in the auxiliary storage device 13 as necessary.

[0029] The auxiliary storage device 13 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 13 is loaded into the memory 12 as necessary. The memory 12 and the auxiliary storage device 13 may be integrated into one unit.

[0030] The input / output IF 14 is a port to which an input device and an output device are connected. A specific example of the input / output IF 14 is a Universal Serial Bus (USB) terminal. Specific examples of the input device are a keyboard and a mouse. A specific example of the output device is a display.

[0031] The communication device 15 is a receiver and a transmitter, and is, for example, a communication chip or a network interface card (NIC).

[0032] Each part of the recognition unit 130 may use the input / output IF 14 and the communication device 15 as appropriate when communicating with the illumination unit 110, the camera unit 120, and the like.

[0033] The auxiliary storage device 13 stores a vehicle name recognition program. The vehicle name recognition program is a program that causes a computer to realize the functions of each unit included in the recognition unit 130. The vehicle name recognition program is loaded into the memory 12 and executed by the processor 11. The functions of each unit included in the recognition unit 130 are realized by software.

[0034] Data used when executing the vehicle name recognition program and data obtained by executing the vehicle name recognition program are appropriately stored in the storage device. Each part of the recognition unit 130 uses the storage device as appropriate. As a specific example, the storage device is composed of at least one of the memory 12, the auxiliary storage device 13, a register in the processor 11, and a cache memory in the processor 11. The storage device may be independent of the computer. The functions of the memory 12 and the auxiliary storage device 13 may be realized by other storage devices.

[0035] The vehicle name recognition program may be recorded in a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The vehicle name recognition program may be provided as a program product.

[0036] ***Explanation of Operation*** The operation procedure of the vehicle name recognition device 100 corresponds to a vehicle name recognition method. Also, the program that realizes the operation of the vehicle name recognition device 100 corresponds to a vehicle name recognition program.

[0037] 8 is a flowchart showing an example of the operation of the vehicle name recognition device 100. The operation of the vehicle name recognition device 100 will be described with reference to FIG.

[0038] (Step S101: Photographed image input process) The camera unit 120 inputs an image of a target vehicle as a target image to the recognition unit 130 in accordance with the timing controlled by the timing control unit 131 .

[0039] (Step S102: License plate detection process) The license plate detection unit 132 detects the license plate of the target vehicle from the target image.

[0040] (Step S103: Pixel number detection process) The pixel number detection unit 133 detects the number of pixels in the vertical and horizontal directions of the license plate detected in step S102.

[0041] (Step S104: Range calculation process) The vehicle name estimation unit 134 calculates the range for cutting out the image based on the number of pixels in the vertical and horizontal directions detected in step S103.

[0042] (Step S105: Cutting out process) The vehicle name estimation unit 134 uses the position of the license plate detected in step S102 as a reference and cuts out an image of the range calculated in step S104 from the target image as a cut-out image.

[0043] (Step S106: Contrast adjustment process) The vehicle name estimation unit 134 appropriately adjusts the contrast of the cut-out image cut out in step S105. The contrast adjustment process is a process that aims to increase the overall contrast of an image that does not have high contrast by applying histogram equalization. Since the vehicle name recognition device 100 continues to operate at night, the vehicle body in the image may become dark. The contrast adjustment process is a process to deal with such cases.

[0044] (Step S107: Feature extraction process) The vehicle name estimation unit 134 extracts feature amounts from the cut-out image whose contrast has been adjusted in step S106.

[0045] (Step S108: Pattern recognition processing) The vehicle name estimation unit 134 executes a pattern recognition process using the feature amount extracted in step S107 and an inference model, thereby recognizing the name of the vehicle shown in the cut-out image.

[0046] (Step S109: Result output process) The vehicle name estimation unit 134 outputs data indicating the result of recognizing the vehicle name.

[0047] ***Explanation of the Effects of the First Embodiment*** As described above, according to this embodiment, in the vehicle name recognition device 100 targeting a traveling vehicle, the license plate detection process and the pixel number detection process are performed as pre-processing of the vehicle name recognition process. Therefore, in the vehicle name recognition process, the number of pixels in the vertical direction of the image cutout range can be a value obtained by multiplying the number of pixels in the vertical direction of the license plate by a preset constant magnification. Also, the number of pixels in the horizontal direction of the image cutout range can be a value obtained by multiplying the number of pixels in the horizontal direction of the license plate by a preset constant magnification. Here, the coordinates of the detected license plate are set as the origin. In addition, the length of the license plate in the vertical and horizontal directions is determined for each type of license plate. Specifically, the dimensions of the license plate are 220 mm long x 440 mm wide for a large-sized license plate, and 165 mm long x 330 mm wide for a medium-sized license plate. Therefore, according to this embodiment, by cutting out the above-mentioned cutting range from the image, it is possible to cut out an image showing a traveling vehicle under uniform conditions. A large-sized license plate is also called a large plate. A medium-sized license plate is also called a medium plate. In other words, according to this embodiment, an image for vehicle name recognition is cut out based on the size of the license plate, which is a fixed length, so that the cut-out image for vehicle name recognition can be cut out under uniform conditions regardless of the shooting position of the traveling vehicle and the zoom setting value of the camera unit 120. Furthermore, when performing vehicle name recognition processing using a machine learning technique based on pattern recognition, manually designed features are acquired by learning. Here, by using images in the above-mentioned cropping range during learning, the variance (variation error) of the learning data can be suppressed. This eliminates the need for overlearning, and the learning model can be simplified. Therefore, according to this embodiment, it is possible to reduce the learning time and improve the performance of vehicle name recognition (improve the accuracy rate of vehicle name recognition).

[0048] ***Other configurations*** <Variation 1> FIG. 9 shows an example of the hardware configuration of the recognition unit 130 according to this modified example. The recognition unit 130 includes a processing circuit 18 in place of the processor 11 , the processor 11 and the memory 12 , the processor 11 and the auxiliary storage device 13 , or the processor 11 , the memory 12 and the auxiliary storage device 13 . The processing circuitry 18 is hardware that realizes at least a portion of each component of the recognition unit 130. The processing circuitry 18 may be dedicated hardware, or may be a processor that executes programs stored in the memory 12 .

[0049] When processing circuitry 18 is dedicated hardware, processing circuitry 18 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The recognition unit 130 may include a plurality of processing circuits that replace the processing circuit 18. The plurality of processing circuits share the role of the processing circuit 18.

[0050] In the recognition unit 130, some of the functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0051] Processing circuitry 18 is illustratively implemented in hardware, software, firmware, or a combination thereof. The processor 11, memory 12, auxiliary storage device 13, and processing circuit 18 are collectively referred to as "processing circuitry." In other words, the functions of the functional components of the recognition unit 130 are realized by the processing circuitry. The recognition unit 130 according to other embodiments may have a similar configuration to this modified example.

[0052] Embodiment 2 The following mainly describes the differences from the above-described embodiment with reference to the drawings.

[0053] ***Configuration Description*** The size of the vehicle body varies depending on the type of the target vehicle. Therefore, if an image for vehicle name recognition is cut out with a certain number of pixels, the same problem as that described with reference to FIG. 5 occurs. Vehicle types include large vehicles such as large trucks or large buses, normal passenger cars, small passenger cars, and light vehicles. Large vehicles include large freight vehicles and large passenger cars. As a specific example, the body of a large vehicle is larger than that of a normal passenger car, etc. Therefore, when the target vehicle is a large vehicle, when cutting out an image of the target vehicle based on the position coordinates of the license plate, it is necessary to make the size of the image cut out relatively large in order to include headlights and front mask parts, etc., whose shapes differ depending on the vehicle model, in the cut-out image. Here, when cutting out an image showing a large vehicle at the same size as cutting out an image showing a normal passenger car, it may not be possible to cut out the image so that the characteristic parts of each vehicle are included, as shown in (a) of Fig. 10. Therefore, in this case, there is a problem that the accuracy rate of vehicle name recognition using an inference model decreases due to too little variation (too little difference in characteristics) between classes (vehicle names). Note that Fig. 10 is a diagram explaining the cut-out range according to this embodiment. On the other hand, if an image showing a light vehicle is cut out at the same size as an image showing a large vehicle, many unnecessary parts that are unrelated to the characteristic parts of each vehicle will be included, resulting in a learning model with large variance (variation error). Therefore, in this case, the noise in the training data will also be learned, resulting in a so-called "overlearning" state, which creates the problem of a decrease in the accuracy rate of car name recognition using an inference model. Therefore, in this embodiment, the vehicle type is identified based on the detected license plate, and the range from which the image is cut out is switched depending on the identified vehicle type. As a specific example, in this embodiment, the magnification with respect to the reference length is variably set depending on each of large vehicles, normal passenger cars, compact passenger cars, and light vehicles, so that the vehicle name recognition image is cut out under more uniform conditions depending on the vehicle body dimensions. (b) and (c) of FIG. 10 are diagrams explaining the cut-out range according to this embodiment. (b) of FIG. 10 shows a specific example of the cut-out range corresponding to a normal passenger car. (c) of FIG. 10 shows a specific example of the cut-out range corresponding to a large vehicle.

[0054] 11 shows an example of the configuration of the vehicle name recognition device 100 according to this embodiment. The vehicle name recognition device 100 further includes a character recognition unit 135 as shown in FIG.

[0055] The character recognition unit 135 recognizes characters written on the license plate detected by the license plate detection unit 132. 12 shows an example of the structure of a license plate. A license plate is also called a vehicle registration plate. As a specific example, the character recognition unit 135 detects a vehicle model code from the license plate shown in the image. The vehicle model code indicates a classification number according to the type of vehicle.

[0056] The license plate detection unit 132 according to this embodiment determines whether the plate size of the detected license plate is a large plate size or a medium plate size.

[0057] The vehicle name estimation unit 134 according to this embodiment identifies the vehicle model based on the characters recognized by the character recognition unit 135, and changes the setting of the magnification in the first embodiment when cutting out an image for vehicle name recognition according to the identified vehicle model. At this time, the vehicle name estimation unit 134 considers the difference in the vehicle model code as a specific example. As a specific example, based on the character recognition result of the license plate, the vehicle name estimation unit 134 identifies the vehicle as a normal passenger car when the first digit of the vehicle model code is a number 3, identifies the vehicle as a compact passenger car (full width 1.7m or less and full height 2.0m or less) when the first digit of the vehicle model code is a number 5, and identifies the vehicle as a light vehicle (full width 1.48m or less and full height 2.0m or less) when the last two digits of the vehicle model code are 80 to 99. Note that an inference model may be prepared for each vehicle model. Therefore, in the case of a light vehicle or a compact passenger car, for example, the vehicle name estimation unit 134 can suppress the inclusion of images that are not directly related to the traveling vehicle by cutting out an image in a narrower range than that of a normal passenger car. Also, by performing learning using the image cut out by the vehicle name estimation unit 134, learning can be performed using training data with less variance (less noise). The vehicle name estimation unit 134 according to the present embodiment changes the range of cutting out an image depending on the vehicle type. Specifically, the vehicle name estimation unit 134 detects the plate size of the detected license plate, and if the detected license plate is a medium plate, identifies the vehicle type based on the characters of the detected license plate. Thereafter, the vehicle name estimation unit 134 uses a cutting condition corresponding to the identified vehicle type as the cutting condition. As a specific example, the vehicle name estimation unit 134 cuts out a relatively narrow range when the target vehicle is a light vehicle, and cuts out a relatively wide range when the target license plate is a large plate. Here, the medium plate and the large plate are each a classification of license plates in Japan.

[0058] In this embodiment, the cut-out condition exists for each vehicle type. As a specific example, when the vehicle type is any of a large vehicle, a normal passenger car, a compact passenger car, and a light vehicle, the cut-out conditions include a first cut-out condition, a second cut-out condition, a third cut-out condition, and a fourth cut-out condition. The vehicle name estimation unit 134 cuts out an image in a range that satisfies the first cut-out condition when the identified vehicle type is a large vehicle, cuts out an image in a range that satisfies the second cut-out condition when the identified vehicle type is a light vehicle, cuts out an image in a range that satisfies the third cut-out condition when the identified vehicle type is a compact passenger car, and cuts out an image in a range that satisfies the fourth cut-out condition when the identified vehicle type is a normal passenger car. The first cutout condition corresponds to a large vehicle, and indicates that a relatively wide range is to be cut out. Fig. 10(c) shows the cutout range corresponding to the first cutout condition, which is determined according to a side having a length of X x A pixels and a side having a length of Y x B pixels. The second cut-out condition is a cut-out condition corresponding to a light vehicle, and is a condition indicating that a range narrower than the range indicated by the cut-out condition corresponding to a compact passenger vehicle is to be cut out. (a) of Fig. 13 shows the cut-out range corresponding to the second cut-out condition, which is determined according to a side having a length of X x a1 pixels and a side having a length of Y x b1 pixels. The third cut-out condition is a cut-out condition corresponding to a compact passenger car, and is a condition indicating that a range narrower than the range indicated by the cut-out condition corresponding to a standard passenger car is to be cut out. (b) of Fig. 13 shows the cut-out range corresponding to the third cut-out condition, which is determined according to a side having a length of X x a2 pixels and a side having a length of Y x b2 pixels. The fourth cut-out condition is a cut-out condition corresponding to a standard passenger vehicle, and indicates that a range narrower than the range indicated by the cut-out condition corresponding to a large vehicle is to be cut out. (c) of Fig. 13 shows the cut-out range corresponding to the fourth cut-out condition, which is determined according to a side having a length of X x a3 pixels and a side having a length of Y x b3 pixels. Each of A, a1, a2, a3, B, b1, b2, and b3 is a constant and may be set arbitrarily or changed arbitrarily. The value obtained by multiplying the number of horizontal pixels by any of A, a1, a2, and a3 corresponds to a side having a length equal to a constant multiple of the number of horizontal pixels along the horizontal direction. The value obtained by multiplying the number of vertical pixels by any of B, b1, b2, and b3 corresponds to a side having a length equal to a constant multiple of the number of vertical pixels along the vertical direction. Typically, the value of a1 is smaller than the value of a2, the value of a2 is smaller than the value of a3, and the value of a3 is smaller than the value of A. Also, the value of b1 is smaller than the value of b2, the value of b2 is smaller than the value of b3, and the value of b3 is smaller than the value of B. Therefore, when the plate size of the license plate detected by the license plate detection unit 132 is the plate size of a large plate, the vehicle name estimation unit 134 cuts out an image in a range that satisfies the first cut-out condition from the target image as a target cut-out image. When the plate size of the detected license plate is the plate size of a medium plate, the vehicle name estimation unit 134 identifies whether the vehicle model of the target vehicle is a light vehicle, a compact passenger car, or a regular passenger car based on the characters recognized by the character recognition unit 135. Thereafter, when the identified vehicle model is a light vehicle, the vehicle name estimation unit 134 cuts out an image in a range that satisfies the second cut-out condition from the target image as a target cut-out image. When the identified vehicle model is a compact passenger car, the vehicle name estimation unit 134 cuts out an image in a range that satisfies the third cut-out condition from the target image as a target cut-out image. When the identified vehicle model is a regular passenger car, the vehicle name estimation unit 134 cuts out an image in a range that satisfies the fourth cut-out condition from the target image as a target cut-out image.

[0059] ***Explanation of Operation*** Fig. 14 is a flowchart showing an example of the operation of the vehicle name recognition device 100. The operation of the vehicle name recognition device 100 will be described with reference to Fig. 14. In this flowchart, it is assumed that the license plate of the target vehicle is either a medium plate or a large plate.

[0060] (Step S201: Plate size detection process) The license plate detection unit 132 detects the plate size of the license plate detected in step S102.

[0061] (Step S202: Character recognition processing) The character recognition unit 135 recognizes the characters on the license plate detected in step S102. Hereinafter, if the plate size of a large plate is detected in step S201, step S203 is executed, and if the plate size of a medium plate is detected in step S201, one of steps S204 to S206 is executed depending on the character recognition result.

[0062] (Step S203: Large vehicle range calculation process) The vehicle name estimation unit 134 calculates the range for cutting out the image based on the number of pixels detected in step S103 and the first cutting condition. The vehicle model codes for large vehicles are 1X, 1XX, 2X, or 2XX.

[0063] (Step S204: Light vehicle range calculation process) This step is executed when the vehicle model code of a light vehicle is recognized in step S202. The vehicle name estimation unit 134 calculates the range for cutting out the image based on the number of pixels detected in step S103 and the second cutting condition.

[0064] (Step S205: Small passenger car range calculation process) This step is executed when the vehicle model code of a compact passenger car is recognized in step S202. The vehicle name estimation unit 134 calculates the range for cutting out the image based on the number of pixels detected in step S103 and the third cutting condition.

[0065] (Step S206: Calculation of range for ordinary passenger cars) This step is executed when the vehicle model code of a passenger car is recognized in step S202. The vehicle name estimation unit 134 calculates the range for cutting out the image based on the number of pixels detected in step S103 and the fourth cutting condition.

[0066] (Step S207: Cutting out process) The vehicle name estimation unit 134 uses the position of the license plate detected in step S102 as a reference and cuts out an image within the range calculated in any one of steps S203 to S206 from the target image as a cut-out image.

[0067] (Step S106: Contrast adjustment process) The vehicle name estimation unit 134 appropriately adjusts the contrast of the cut-out image cut out in step S207.

[0068] ***Explanation of the effect of the second embodiment*** As described above, according to this embodiment, the vehicle name estimation unit 134 sets the cutout size of the image for machine learning according to the size of the license plate and the result of character recognition of the license plate. As a result, when a large vehicle with a large sign plate is shown in the image, the image is cut out in a wider range than when a normal passenger car is shown in the image. Therefore, according to this embodiment, it is possible to appropriately extract the headlights and front turn signal parts, etc., whose shapes differ depending on the vehicle name. On the other hand, when the size of the license plate is the size of a medium-sized plate, the vehicle type is either a normal passenger car, a compact passenger car, or a light vehicle. Here, for normal passenger cars, the first digit of the vehicle type code is a 3 number. For compact passenger cars, the first digit of the vehicle type code is a 5 number. For light vehicles, the last two digits of the vehicle type code are any of 80 to 99. The vehicle type code is indicated in the character recognition results of the license plate. Therefore, according to this embodiment, the vehicle dimensions can be determined based on the character recognition results of a medium-sized sign plate. In addition, when the vehicle type is determined to be a light vehicle or a small passenger vehicle based on the result of character recognition of the license plate, the image is cropped to a narrower range than when a standard passenger vehicle is captured in the image, thereby reducing the inclusion of images that are not directly related to the characteristics of the vehicle. Therefore, by adopting this embodiment, learning can be performed using training data with less variance (less noise).

[0069] ***Other embodiments*** The above-described embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. Moreover, the embodiments are not limited to those shown in the first and second embodiments, and various modifications are possible as necessary. The procedures explained using the flowcharts and the like may be modified as appropriate.

[0070] Various aspects of the present disclosure are summarized below as appendices.

[0071] (Appendix 1) a pixel number detection unit that detects a horizontal pixel number, which is the number of pixels in the horizontal direction of the license plate detected from a target image showing the license plate of a target vehicle, and a vertical pixel number, which is the number of pixels in the vertical direction of the detected license plate; a vehicle name estimation unit that cuts out an image within a range that satisfies a cutout condition, the cutout condition being a condition indicating a range based on the position of the detected license plate, the condition indicating that the image should be cut out so as to include each vehicle part constituting a characteristic part group consisting of at least one vehicle part having a feature that serves as a clue to identifying the name of the vehicle shown in the image, the cutout condition being a condition indicating a range determined according to a side having a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction; and A vehicle name recognition device comprising: A vehicle name recognition device, wherein each of the horizontal direction and the vertical direction is a direction determined based on a detected license plate.

[0072] (Appendix 2) The vehicle name estimation unit receives as input data representing the target cropped image and data representing an image showing a range that satisfies the cropping condition, and estimates the name of the target vehicle using an inference model that estimates the name of a vehicle shown in an image corresponding to the input.

[0073] (Appendix 3) 3. The vehicle name recognition device according to claim 1, wherein the target image shows a front of the target vehicle.

[0074] (Appendix 4) 4. The vehicle name recognition device according to claim 1, wherein the cut-out condition is a condition indicating that an area showing a license plate of a vehicle is to be cut out.

[0075] (Appendix 5) The license plate of the target vehicle is either a medium plate or a large plate, The vehicle name recognition device further includes: A license plate detection unit that determines whether the plate size of the detected license plate is the plate size of the large plate or the plate size of the medium plate; A character recognition unit that recognizes the characters on the detected license plate; Equipped with The vehicle name estimation unit, When the plate size of the detected license plate is the plate size of the large plate, an image of a range that satisfies a first cut-out condition, which is a cut-out condition corresponding to a large vehicle, is cut out from the target image as the target cut-out image; When the plate size of the detected license plate is the plate size of the intermediate plate, Identifying the type of the target vehicle as a light vehicle, a compact passenger car, or a standard passenger car based on the recognized characters; When the specified vehicle type is the light vehicle, an image in a range that satisfies a second cut-out condition, which is a cut-out condition corresponding to the light vehicle, is cut out from the target image as the target cut-out image; If the identified vehicle type is the compact passenger car, an image in a range that satisfies a third cut-out condition, which is a cut-out condition corresponding to the compact passenger car, is cut out from the target image as the target cut-out image; A vehicle name recognition device as described in any one of Appendices 1 to 4, wherein, when the identified vehicle type is the standard passenger car, an image within a range that satisfies a fourth cut-out condition, which is a cut-out condition corresponding to the standard passenger car, is cut out from the target image as the target cut-out image. [Explanation of symbols]

[0076] 11 processor, 12 memory, 13 auxiliary storage device, 14 input / output IF, 15 communication device, 18 processing circuit, 19 signal line, 100 vehicle name recognition device, 110 lighting unit, 120 camera unit, 130 recognition unit, 131 timing control unit, 132 license plate detection unit, 133 pixel number detection unit, 134 vehicle name estimation unit, 135 character recognition unit.

Claims

1. a pixel number detection unit that detects a horizontal pixel number, which is the number of pixels in the horizontal direction of the license plate detected from a target image showing the license plate of a target vehicle, and a vertical pixel number, which is the number of pixels in the vertical direction of the detected license plate; a vehicle name estimation unit that cuts out an image within a range that satisfies a cut-out condition, the cut-out condition being a condition indicating a range based on the position of the detected license plate, the condition indicating that the image should be cut out so as to include each vehicle part constituting a characteristic part group consisting of at least one vehicle part having a feature that serves as a clue to identifying the name of the vehicle shown in the image, the cut-out condition being a condition indicating a range determined according to a side having a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction; and A vehicle name recognition device comprising: A vehicle name recognition device, wherein each of the horizontal direction and the vertical direction is a direction determined based on a detected license plate.

2. The vehicle name estimation unit estimates the name of the target vehicle using an inference model that estimates the name of the vehicle shown in the image corresponding to the input, and receives data indicating the target cut-out image and data indicating an image showing the range that satisfies the cut-out condition.

3. 3. The vehicle name recognition device according to claim 1, wherein the target image shows a front face of the target vehicle.

4. 3. The vehicle name recognition device according to claim 1, wherein the cutout condition is a condition that specifies that an area showing a license plate of a vehicle is to be cut out.

5. The license plate of the target vehicle is either a medium plate or a large plate, The vehicle name recognition device further includes: A license plate detection unit that determines whether the plate size of the detected license plate is the plate size of the large plate or the plate size of the medium plate; A character recognition unit that recognizes the characters on the detected license plate; Equipped with The vehicle name estimation unit, When the plate size of the detected license plate is the plate size of the large plate, an image of a range that satisfies a first cut-out condition, which is a cut-out condition corresponding to a large vehicle, is cut out from the target image as the target cut-out image; When the plate size of the detected license plate is the plate size of the intermediate plate, Identifying the type of the target vehicle as a light vehicle, a compact passenger car, or a standard passenger car based on the recognized characters; When the specified vehicle type is the light vehicle, an image in a range that satisfies a second cut-out condition, which is a cut-out condition corresponding to the light vehicle, is cut out from the target image as the target cut-out image; When the specified vehicle type is the compact passenger car, an image in a range that satisfies a third cut-out condition, which is a cut-out condition corresponding to the compact passenger car, is cut out from the target image as the target cut-out image; A vehicle name recognition device as described in claim 1 or 2, wherein when the identified vehicle type is the standard passenger car, an image within a range that satisfies a fourth cut-out condition, which is a cut-out condition corresponding to the standard passenger car, is cut out from the target image as the target cut-out image.

6. The computer detects a horizontal pixel count, which is the number of pixels in the horizontal direction of the license plate detected from the target image showing the license plate of the target vehicle, and a vertical pixel count, which is the number of pixels in the vertical direction of the detected license plate; A vehicle name recognition method in which the computer cuts out an image from the target image as a target cut-out image, the image satisfying a cut-out condition being a condition indicating a range based on the position of a detected license plate, the condition indicating that the image should be cut out so as to include each vehicle part constituting a group of characteristic parts consisting of at least one vehicle part having a feature that serves as a clue to identifying the name of a vehicle shown in the image, the cut-out condition being a condition indicating a range determined according to a side having a length equal to a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side having a length equal to a vertical constant multiple of the number of vertical pixels along the vertical direction, A vehicle name recognition method, wherein each of the horizontal direction and the vertical direction is a direction determined according to a detected license plate.

7. a pixel number detection process for detecting a horizontal pixel number, which is the number of pixels in the horizontal direction of the license plate detected from a target image showing the license plate of a target vehicle, and a vertical pixel number, which is the number of pixels in the vertical direction of the detected license plate; a vehicle name estimation process for cutting out an image in a range that satisfies a cutting condition, the cutting condition being a condition indicating a range based on the position of the detected license plate, the condition indicating that the image should be cut out so as to include each vehicle part constituting a characteristic part group consisting of at least one vehicle part having a feature that serves as a clue to identifying the name of the vehicle shown in the image, the cutting condition being a condition indicating a range determined according to a side that is a horizontal constant multiple of the number of horizontal pixels along the horizontal direction and a side that is a vertical constant multiple of the number of vertical pixels along the vertical direction; and A vehicle name recognition program for causing a vehicle name recognition device that is a computer to execute the above. A vehicle name recognition program, wherein each of the horizontal direction and the vertical direction is a direction determined according to a detected license plate.

Images