Vehicle type identification device, vehicle type identification model generation device, and vehicle type identification model generation method

Abstract

To provide a technique capable of more accurately identifying a vehicle type of a vehicle.SOLUTION: A vehicle type identification device (2) includes a cut-out region size calculation unit (21) that calculates a cut-out region size, a cut-out processing unit (23) that cuts out an image for identifying the vehicle type of the vehicle from the captured image according to the cut-out region size, a first vehicle type identification unit (24) that identifies the vehicle type of the vehicle from the image cut out by the cut-out processing unit (23), a salesperson identification unit (25) that identifies the salesperson of the vehicle from the image cut out by the cut-out processing unit (23), and a second vehicle type identification unit (26) that identifies the vehicle type of the vehicle based on the vehicle type of the vehicle identified by the first vehicle type identification unit (24) and the seller of the vehicle identified by the seller identification unit (25).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a vehicle type identification device, a vehicle type identification model generation device, and a vehicle type identification model generation method. [Background technology]

[0002] For example, there is a vehicle number reader that uses an ITV (Industrial Television) camera to continuously capture images of moving vehicles that enter its field of view, and then uses computer processing to detect images of the vehicle's license plate based on the image information captured by the ITV camera, and identifies the vehicle by reading the text information in the license plate. A related technology is the invention disclosed in Patent Document 1 below.

[0003] Patent Document 1 discloses a focus evaluation unit that identifies the horizontal edge position of a symbol on a license plate from a binary image obtained by binarizing a grayscale image of the license plate obtained by an imaging camera, sets a gradient value measurement range and a reference value measurement range offset horizontally to the left and right by a predetermined number of pixels on the grayscale image centered on the horizontal edge position of the symbol, calculates the gradient value of the grayscale density within the gradient value measurement range as a measurement value, calculates the contrast value within the reference value measurement range as a reference value, and calculates the ratio of the measurement value to the reference value as a focus evaluation value. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-140160 A (Published June 25, 2009) Summary of the Invention [Problem to be solved by the invention]

[0005] By reading the text information in a vehicle's license plate, a vehicle license plate reader can manage entrance and exit to parking lots and premises, track and investigate suspicious vehicles on the road, etc. License plate information is sufficient for management purposes as it can uniquely identify a vehicle, but if vehicle model information is available in addition to license plate information, it will be easier for managers and supervisors to visually find the vehicle in question.

[0006] However, license plate information can only roughly distinguish between vehicle types (passenger cars, light cars, etc.), and cannot identify the detailed vehicle model (vehicle name). This problem cannot be solved even if the above-mentioned Patent Document 1 is used.

[0007] The present disclosure aims to provide a technology that can more accurately identify the vehicle model of an automobile. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the vehicle model identification device of the present disclosure includes a cut-out area size calculation unit that calculates a cut-out area size based on the license plate position coordinates in a captured image of a vehicle, a cut-out processing unit that cuts out an image for identifying the vehicle model from the captured image according to the cut-out area size calculated by the cut-out area size calculation unit, a first vehicle model identification unit that identifies the vehicle model from the image cut out by the cut-out processing unit using a vehicle model learning model that has learned the relationship between vehicle model learning images for identifying the vehicle model and vehicle model labels, a seller identification unit that identifies the seller of the vehicle from the image cut out by the cut-out processing unit using a seller learning model that has learned the relationship between seller learning images for identifying the vehicle model and seller labels, and a second vehicle model identification unit that identifies the vehicle model based on the vehicle model identified by the first vehicle model identification unit and the seller of the vehicle identified by the seller identification unit.

[0009] In order to solve the above-mentioned problems, the vehicle model identification model generation device according to the present disclosure includes an image processing unit that generates second training images by performing image processing on first training images for identifying vehicle models and sellers, and a learning unit that learns the relationship between the first training images and the second training images and labels to generate a training model.

[0010] In order to solve the above-mentioned problems, the vehicle model identification model generation method according to the present disclosure includes a step of generating second training images by performing image processing on first training images for identifying vehicle models and sellers, respectively, and a step of learning the relationship between the first training images and the second training images and labels to generate a training model. [Effects of the Invention]

[0011] According to one aspect of the present disclosure, it is possible to more accurately identify the vehicle model. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a schematic configuration of a vehicle model identification system according to an embodiment of the present disclosure. [Figure 2] FIG. 10 is a diagram for explaining the position coordinates of a license plate. [Figure 3] 10 is a table showing the relationship between the Y coordinate of the license plate position coordinates and the size of the cut-out region. [Figure 4] FIG. 10 is a diagram illustrating an example of a cutout region. [Figure 5] FIG. 10 is a diagram illustrating an example of an image cut out by a cutout processing unit. [Figure 6] 10 is a flowchart for explaining a processing procedure of a cutout region size calculation unit and a cutout processing unit of the vehicle model identification device according to an embodiment of the present disclosure. [Figure 7] 5 is a flowchart for explaining a processing procedure of a first identification unit of a vehicle model identification device according to an embodiment of the present disclosure. [Figure 8]4 is a flowchart for explaining a processing procedure of a seller identification unit 25 of the vehicle model identification device according to an embodiment of the present disclosure. [Figure 9] 9 is a flowchart for explaining details of the estimation of the position of the emblem (S22) shown in FIG. 8. [Figure 10] FIG. 1 is a block diagram illustrating a schematic configuration of a model generation device according to an embodiment of the present disclosure. [Figure 11] 10 is a flowchart illustrating a processing procedure of a vehicle model identification model generation device according to an embodiment of the present disclosure. [Figure 12] 12 is a flowchart for explaining details of the image processing (S42) shown in FIG. 11. [Figure 13] 13 is a flowchart for explaining details of the transformation matrix creation process (S55) shown in FIG. 12. [Figure 14] 10 is a flowchart illustrating a processing procedure of a seller identification model generation device according to an embodiment of the present disclosure. [Figure 15] 15 is a flowchart for explaining details of the image processing (S63) shown in FIG. 14. [Figure 16] 16 is a flowchart for explaining details of the transformation matrix creation process (S75) shown in FIG. 15. [Figure 17] FIG. 10 is a diagram showing an example of an OEM table referred to by a second vehicle model identification unit. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Configuration example of vehicle type identification system 100) FIG. 1 is a block diagram showing a schematic configuration of a vehicle model identification system 100 according to an embodiment of the present disclosure. The vehicle model identification system 100 includes an imaging device 1 and a vehicle model identification device 2. The vehicle model identification system 100 can capture images of vehicles traveling on roads and extract images for identifying the vehicle model. The vehicle model identification system 100 can also capture images of vehicles entering and exiting parking lots and extract images for identifying the vehicle model. The vehicle model refers not only to vehicle models such as trucks and light vehicles, but also to the vehicle's name (common name). Therefore, the vehicle model identification device 2 extracts images so that the vehicle name can be identified, and identifies the vehicle model from the extracted images using a pre-trained learning model. Furthermore, as will be described in detail later, the vehicle model identification device 2 can accurately identify even OEM vehicle models that have the same body shape and are prone to misrecognition.

[0014] The imaging device 1 includes an imaging unit 11 and a position coordinate detection unit 12. The imaging unit 11 is configured with a camera or the like and captures images of a traveling automobile. The imaging unit 11 is installed at a height of, for example, about 6 m and captures an image of the front (front part) of the traveling automobile from diagonally above. The imaging unit 11 may also capture an image of the rear of the traveling automobile.

[0015] The position coordinate detection unit 12 extracts the image portion of the license plate from the captured image of the front of the automobile and detects the position coordinates of the license plate. The detection of the position coordinates of the license plate will be explained with reference to FIG.

[0016] FIG. 2 is a diagram illustrating license plate position coordinates. As shown in FIG. 2, the license plate has a line indicating the area name "Kyoto" and the classification number "XXX," and a line indicating the use "A" and the serial number "XX-XXX." The position coordinate detection unit 12 detects the coordinates of point P, which is between the line indicating the area name and classification number and the line indicating the use and serial number and is in the center of the width, as the license plate position coordinates, and outputs this to the vehicle type identification device 2. The license plate position coordinates are not limited to these, and may be other coordinates based on the license plate.

[0017] As shown in Figure 2, the distance from point P to the left edge of the license plate is W L , the length from point P to the right edge of the license plate is W R Then, the position coordinate detection unit 12 calculates W L =W R Point P is determined so that: The coordinates of point P in the image captured by imaging unit 11 are the license plate position coordinates. Hereinafter, the license plate position coordinates, i.e., the coordinates of point P, are referred to as P(x, y). The license plate position coordinates P(x, y) are the position coordinates of the license plate when the captured image is expressed in XY coordinates.

[0018] An image processing device (not shown) can obtain the vehicle's license plate information by reading the text information in the vehicle's license plate with reference to the image captured by the imaging unit 11 and the license plate position coordinates. Therefore, the image captured by the imaging unit 11 can be used both to obtain the vehicle's license plate information and to cut out an image for identifying the vehicle model.

[0019] Returning to the explanation of Fig. 1, the vehicle model identification device 2 includes a cropping region size calculation unit 21, a setting file 22, a cropping processing unit 23, a first vehicle model identification unit 24, a seller identification unit 25, and a second vehicle model identification unit 26. The input and output of information between the imaging device 1 and the vehicle model identification device 2 may be wired or wireless communication, and is not particularly limited.

[0020] The cropping area size calculation unit 21 calculates the cropping area size when the Y coordinate of the license plate position coordinates P(x, y) output from the position coordinate detection unit 12 matches the Y coordinate of the reference license plate position coordinates stored in the setting file 22. The reference license plate position coordinates are predetermined position coordinates that serve as a reference when the cropping processing unit 23 crops an image.

[0021] As described above, the imaging unit 11 is installed at a height of about 6 m and captures the front of the vehicle from above, so if the vehicle is far away, the Y coordinate (coordinate in the height direction) of the license plate position coordinate P(x, y) will be located at the top in the captured image. As the vehicle approaches, the license plate position coordinate P(x, y) moves downward, and when the distance between the vehicle and the imaging unit 11 reaches a predetermined distance, the license plate position coordinate and the reference license plate position coordinate will match. The captured image (still image) at this time is the image to be cropped.

[0022] The setting file 22 defines information such as the size of the captured image (number of vertical and horizontal pixels), the coordinates of the reference license plate position, and parameters α, β, γ used when calculating the size of the cutout region.

[0023] α is the vertical size (number of pixels in the vertical direction) of the image to be processed (for example, it may be the entire captured image or a portion of the captured image). β is the size of the cropped area when the Y coordinate of the license plate position coordinates P(x,y) is maximum. γ is the size of the cropped area when the Y coordinate of the license plate position coordinates P(x,y) is minimum. The cropped area size will be described as being a square with the same vertical and horizontal sizes, but it may also be a rectangle with different vertical and horizontal sizes, and is not particularly limited. Below, we will describe the case where the image to be processed is the entire captured image.

[0024] For example, suppose the size of the image to be processed is 1024 x 768, the size of the cropped area when the Y coordinate of the license plate position coordinates P(x, y) is maximum is 300 x 300, and the size of the cropped area when the Y coordinate of the license plate position coordinates P(x, y) is minimum is 200 x 200. In this case, α = 768, β = 300, and γ = 200. These parameters are predefined in the setting file 22. If the upper left corner of the target processing image is taken as the origin, the minimum Y coordinate of the license plate position coordinates P(x, y) is "0" and the maximum is "767."

[0025] The cutout area size calculation unit 21 uses the parameters α, β, and γ to calculate the cutout area size L×L corresponding to the Y coordinate of the license plate position coordinates using the following equation (Equation 1), and defines it in the setting file 22 as a cutout area size table.

[0026] L=(β-γ)×y / α+γ (Formula 1) The origin of the image to be processed is the upper left corner, so the Y coordinate of the top of the image to be processed is "0" and the Y coordinate of the bottom of the image to be processed is "767".

[0027] Furthermore, substituting the values ​​of the parameters α, β, and γ into the above (Equation 1) results in the following equation (Equation 2): The clipping region size calculation unit 21 may calculate the clipping region size using the following equation (Equation 2).

[0028] L=0.13×Y+200 (Formula 2) Figure 3 is a table showing the relationship between the Y coordinate of the license plate position coordinates and the size of the cut-out area. As shown in Figure 3, when the Y coordinate of the license plate position coordinates is the minimum "0", the cut-out area size is 200 x 200, and the cut-out area size gradually increases as the Y coordinate increases. When the Y coordinate of the license plate position coordinates is the maximum "767", the cut-out area size is 300 x 300.

[0029] In this way, when the distance from the image capturing device 1 to the automobile is large, the crop area size is reduced. On the other hand, when the distance from the image capturing device 1 to the automobile is small, the crop area size is increased. As a result, the image is cropped so as to include the same portion of the front of the automobile. When the reference license plate position coordinates are fixed, the crop area size calculation unit 21 may calculate only the crop area size corresponding to the reference license plate position coordinates and define this value in the setting file 22.

[0030] The cropping area size calculation unit 21 outputs to the cropping processing unit 23 the captured image, the license plate position coordinates, and the cropping area size when the Y coordinate of the license plate position coordinates and the Y coordinate of the reference license plate position coordinates match.

[0031] The cut-out processing unit 23 uses the captured image, license plate position coordinates, and cut-out area size output from the cut-out area size calculation unit 21 to cut out an image of a cut-out area for identifying the vehicle type from the image to be processed, and outputs the image to the first vehicle type identification unit 24.

[0032] The first vehicle type identification unit 24 identifies the vehicle type from the extracted image by inputting the image extracted by the extraction processing unit 23 into a learning model trained by machine learning using a convolutional neural network (CNN) by a vehicle type identification model generation device described below. As described below, this learning model has learned the relationship between vehicle type learning images for identifying vehicle types and vehicle type labels.

[0033] Machine learning is not limited to CNN, but may be general NN, RNN, DNN, R-CNN, Fast R-CNN, Faster R-CNN, SegNet, U-Net, U-Net++, etc.

[0034] 4 is a diagram showing an example of a cutout area. As shown in FIG. 4, for example, with the license plate position coordinates P(x, y) as a reference, W L ':W R The cutout area is determined so that '=1:1. L ' is the size (number of pixels) from the X coordinate of the license plate position coordinate to the left edge, and W R ' is the size (number of pixels) from the X coordinate of the license plate position coordinate to the right edge. L ' and W R The ratio to ' may be other values.

[0035] In the vertical direction (y direction) H U :H LThe cutout area is determined so that H = 3:1. U is the size (number of pixels) from the Y coordinate of the license plate position coordinate to the top edge, and H L is the size (number of pixels) from the Y coordinate of the license plate position coordinate to the bottom edge. U and H L The ratio may be any other value.

[0036] If the cutout region exceeds the range of the captured image, the cutout processing unit 23 may pad the portion that exceeds the range of the captured image with black to maintain the square shape.

[0037] Fig. 5 is a diagram showing an example of an image cut out by the cut-out processing unit 23. As shown in Fig. 5, the front of the car is cut out to a size that does not include the background. The setting file 22 defines the reference license plate position coordinates α, β, and γ so that the background image of the car is not included.

[0038] (Processing flow of the cutout region size calculation unit 21 and the cutout processing unit 23) 6 is a flowchart for explaining the processing steps of the cropping region size calculation unit 21 and the cropping processing unit 23 of the vehicle model identification device 2 according to an embodiment of the present disclosure. First, a captured image and license plate position coordinates are input to the cropping region size calculation unit 21 (S1).

[0039] Next, the cropping area size calculation unit 21 determines whether the Y coordinate of the license plate position coordinates matches the Y coordinate of the reference license plate position coordinates defined in the setting file (S2). If the Y coordinate of the license plate position coordinates does not match the Y coordinate of the reference license plate position coordinates (S2, No), the process returns to step S1 and repeats the subsequent processes.

[0040] If the Y coordinate of the license plate position coordinates matches the Y coordinate of the reference license plate position coordinates (S2, Yes), the cut-out area size calculation unit 21 selects the captured image (still image) to be cut out using the method described above, and calculates the cut-out area size (S3).

[0041] Finally, the cut-out processing unit 23 uses the captured image, license plate position coordinates, and cut-out area size output from the cut-out area size calculation unit 21 to cut out an image of the cut-out area for identifying the vehicle type from the image to be processed, and outputs it to the first vehicle type identification unit 24 (S4).

[0042] (Processing flow of the first vehicle type identification unit 24) 7 is a flowchart for explaining the processing procedure of the first vehicle type identification unit 24 of the vehicle type identification device 2 according to an embodiment of the present disclosure. First, the first vehicle type identification unit 24 inputs the image cut out by the cut-out processing unit 23 (S11).

[0043] Next, the first vehicle type identification unit 24 inputs a vehicle type learning model generated by a vehicle type identification model generation device 3A (described later) (S12).The first vehicle type identification unit 24 then inputs the image cut out by the cutout processing unit 23 into the vehicle type learning model, thereby identifying the vehicle type from the image (S13), outputting the identification result (S14), and ending the process.

[0044] (Processing flow of seller identification unit 25) FIG. 8 is a flowchart for explaining the processing procedure of the seller identification unit 25 of the vehicle model identification device 2 according to an embodiment of the present disclosure.

[0045] First, the seller identifying unit 25 inputs the image cut out by the cut-out processing unit 23 (S21).

[0046] Next, the seller identifying unit 25 estimates the position of the car emblem included in the image cut out by the cut-out processing unit 23 through image processing (S22). Details of the processing in step S22 will be described later with reference to FIG.

[0047] If the emblem is located at the center of the image, the process proceeds to S24. In S24, the seller identification unit 25 inputs the upper center learning model and the middle center learning model generated by the seller identification model generation device 3B. Details of the upper center learning model and the middle center learning model will be described later with reference to FIG. 10 etc.

[0048] If the emblem is not located in the center of the image, the process proceeds to S25. In S25, it is determined whether the emblem is located on the right side of the image. If the emblem is located on the right side of the image, the process proceeds to S26.

[0049] In S26, the seller identification unit 25 inputs the upper right learning model and the middle right learning model generated by the seller identification model generation device 3B. Details of the upper right learning model and the middle right learning model will be described later with reference to FIG. 10 etc.

[0050] If the emblem is not located on the right side, the process proceeds to S27. In S27, the seller identification unit 25 inputs the upper left learning model and the middle left learning model generated by the seller identification model generation device 3B. Details of the upper left learning model and the middle left learning model will be described later with reference to FIG. 10 etc.

[0051] Next, seller identification unit 25 inputs the image cut out by cut-out processing unit 23 into a seller learning model, thereby identifying the seller of the car from the image (S28).

[0052] Then, the seller identification unit 25 outputs the identification result obtained in S28 to the second vehicle type identification unit 26 (S29).

[0053] Finally, based on the vehicle model identified by the first vehicle model identification unit 24 and the vehicle seller identified by the seller identification unit 25, the second vehicle model identification unit 26 identifies the vehicle model. As a result, the second vehicle model identification unit 26 can identify with higher accuracy even OEM vehicle models that have the same body shape and are prone to erroneous recognition. The operation of the second vehicle model identification unit 26 to identify the vehicle model will be described later with reference to FIG.

[0054] FIG. 9 is a flowchart for explaining details of the estimation of the position of the emblem (S22) shown in FIG.

[0055] First, the seller identification unit 25 divides the image cut out by the cutout processing unit 23 into a foreground and a background (S31). In this disclosure, the foreground of the image is assumed to be the body of a car. Furthermore, in this disclosure, the following two algorithms are adopted as algorithms for dividing the foreground and background, although they are not limited thereto.

[0056] The first algorithm is adopted when the position of the imaging unit 11 is fixed and only the background image is available in advance. More specifically, the seller identification unit 25 divides the image cut out by the cut-out processing unit 23 into the background and the foreground by obtaining the difference between the input image and the background image as the foreground.

[0057] The second algorithm is an algorithm other than the first algorithm. More specifically, the seller identification unit 25 divides the image cut out by the cutout processing unit 23 into a background and a foreground using U2-Net, a machine learning model that performs salient object detection (SOD) in a single shot.

[0058] Next, the seller identifying unit 25 calculates the ratio of the background divided in S31 (S32). For example, the seller identifying unit 25 calculates the ratio of the background image to the entire image based on the ratio of pixels.

[0059] Next, the seller identification unit 25 paints the background image with a single color of black and combines it with the foreground image (S33). Through this process, the seller identification unit 25 turns the image cut out by the cut-out processing unit 23 into an image with only a single color of black except for the foreground. If this does not reduce the accuracy of the seller learning model described below, the seller identification unit 25 may paint the background image with a single color or multiple colors other than black.

[0060] Next, the process proceeds to S34, where the seller identification unit 25 determines whether the background ratio calculated in S32 is equal to or less than a predetermined threshold value TH_0. If the answer is Yes in S32, the process proceeds to S35. The predetermined threshold value TH_0 is an appropriately determined value that indicates that the emblem is located in the center, that is, that the image cropped by the cropping processing unit 23 contains a small amount of background image.

[0061] In S35, the divided image input unit 35 divides the composite image obtained in S33 into nine parts in a grid pattern, and cuts out the upper center image and the middle center image.

[0062] If the background ratio calculated in S32 is not equal to or less than the predetermined threshold TH_0, the process proceeds to S36. In S36, the seller identification unit 25 determines whether the background ratio calculated in S32 is less than the predetermined threshold TH_1 and whether the ratio of the left half of the image is greater than the right half of the image. If Yes in S36, the process proceeds to S37. The predetermined threshold TH_1 is a value that suggests that the emblem is located on the right side and is determined as appropriate.

[0063] In S37, the divided image input unit 35 divides the composite image obtained in S33 into nine parts in a grid pattern, and cuts out the upper right image and the middle right image.

[0064] If the answer is No in S36, proceed to S38. In S38, the seller identification unit 25 determines whether the background ratio calculated in S32 is less than a predetermined threshold TH_1 and whether the ratio of the right half of the image is greater than the ratio of the left half of the image. If the answer is Yes in S38, proceed to S39.

[0065] In S39, the divided image input unit 35 divides the composite image obtained in S33 into nine parts in a grid pattern, and cuts out the top left image and the middle left image.

[0066] If the result of S38 is No, the process is terminated abnormally without further processing. This corresponds to the case where the image contains a large amount of background and is therefore deemed to be an unexpected image.

[0067] After the above processing, in S39-2, the divided image input unit 35 outputs an image of the estimated position (to S23 in FIG. 8).

[0068] (Effect of vehicle type identification device 2) As described above, according to the vehicle type identification device 2, the cutout processing unit 23 cuts out an image for identifying the vehicle type from the captured image in accordance with the cutout region size calculated by the cutout region size calculation unit 21. Then, the first vehicle type identification unit 24 identifies the vehicle type from the image cut out by the cutout processing unit 23 using a vehicle type learning model that has learned the relationship between the vehicle type learning image for identifying the vehicle type and the vehicle type label.

[0069] Furthermore, the seller identification unit 25 identifies the seller of the automobile from the image cut out by the cut-out processing unit 23 using a seller learning model that has learned the relationship between the seller learning image and the seller label for identifying the seller of the automobile.

[0070] Finally, the second vehicle type identification unit 26 finally identifies the vehicle type based on the vehicle type identified by the first vehicle type identification unit 24 and the vehicle seller identified by the seller identification unit 25.

[0071] In this way, the image captured by imaging unit 11 can be used both to obtain the vehicle's license plate information and to crop the image to identify the vehicle model and seller. Then, cropping processing unit 23 crops the image based on the license plate position coordinates so that the vertical and horizontal dimensions of the cropped area size calculated by cropped area size calculation unit 21 have a predetermined ratio. Therefore, cropping processing unit 23 can crop the image so as to include the same portion of the front of the vehicle, allowing second vehicle model identification unit 26 to more accurately identify the vehicle model.

[0072] Vehicle identification functions using machine learning can be used to identify vehicle types and are utilized in a variety of systems, such as parking lot entrance / exit management and road security measures. Conventional vehicle identification functions could make incorrect judgments in the case of products (OEM) manufactured under a customer's trademark, even though the appearance is almost identical, because the vehicle name is different.

[0073] In contrast, the vehicle model identification device 2 according to the present disclosure can identify the vehicle model with high accuracy even for OEM vehicles, since the second vehicle model identification unit 26 identifies the vehicle model based on the vehicle model and seller.

[0074] (Configuration example of model generation device 3) 10 is a block diagram showing a schematic configuration of a model generation device 3 according to an embodiment of the present disclosure. The model generation device 3 includes a vehicle model identification model generation device 3A and a seller identification model generation device 3B. The vehicle model identification model generation device 3A includes a training image input unit 31, an image processing unit 32, a training unit 33, and a training model output unit 34. The seller identification model generation device 3B includes a segmented image input unit 35, a segmented image processing unit 36, a segmented image training unit 37, and a segmented image training model output unit 38.

[0075] The learning image storage unit 4 stores a plurality of vehicle model learning images used to identify vehicle models in advance, and stores, for example, pairs of images of various vehicle models taken from the front and vehicle model labels corresponding to those images as vehicle model learning data.

[0076] Furthermore, the learning image storage unit 4 stores a plurality of seller learning images that are used in advance to identify the sellers of automobiles, and stores, for example, pairs of images of automobiles of various sellers taken from the front and seller labels corresponding to those images as seller learning data.

[0077] (Vehicle type identification model generation device 3A) The training image input unit 31 inputs a pair of one vehicle model training image and a vehicle model label corresponding to that image from the training image storage unit 4 as vehicle model training data, and outputs the pair of the vehicle model training image and the vehicle model label to the image processing unit 32.

[0078] Learning images are required when identifying vehicle types using machine learning or the like, but preparing learning images is difficult because the environment, such as the installation angle of the imaging device 1, differs depending on the installation location of the imaging device 1. Therefore, the image processing unit 32 performs image processing on the learning images taking into account the operational environment and outputs the results to the learning unit 33. Specifically, because the environment, such as the installation angle of the imaging device 1 and the ambient brightness, differs depending on the installation location of the imaging device 1, the image processing unit 32 performs image processing on the learning images, such as coordinate transformation using a homography transformation matrix, brightness correction, and color correction, to generate multiple learning images (second learning images) from one learning image (first learning image). The image processing unit 32 performs similar image processing on learning images of various vehicle types stored in the learning image storage unit 4.

[0079] The learning unit 33 generates a learning model by learning, through machine learning, the relationship between the training images (first training image, second training image) generated by the image processing unit 32 and the vehicle model labels corresponding to the training images. Note that the same vehicle model label as that assigned to the first training image is also assigned to the second training image generated by image processing.

[0080] The learning model output unit 34 outputs the learning model learned by the learning unit 33 to the learning model storage unit 5, and stores the learned learning model in the learning model storage unit 5. This learning model is a result of learning all of the relationships between the learning images (first learning images) of various types of automobiles stored in the learning image storage unit 4 and the learning images (second learning images) generated by image processing, and the corresponding vehicle model labels.

[0081] (Processing flow of the vehicle type identification model generation device 3A) 11 is a flowchart illustrating the processing steps of a vehicle model identification model generation device 3A according to an embodiment of the present disclosure. First, the training image input unit 31 inputs, as training data, pairs of vehicle model training images and vehicle model labels corresponding to the images from the training image storage unit 4 (S41).

[0082] Next, the image processing unit 32 performs image processing such as coordinate transformation using a homography transformation matrix, brightness correction, and color correction on the training image to generate multiple training images (second training images) from one training image (first training image) (S42). Details of the processing at step S42 will be described later.

[0083] Next, the learning unit 33 uses machine learning to learn the relationship between the first training image and the second training image generated by the image processing unit 32, and the vehicle model label corresponding to the first image, and generates a learning model (S43).

[0084] Finally, the learning model output unit 34 outputs the learned learning model generated by the learning unit 33 to the learning model storage unit 5 (S44), and the process ends. Note that the processes of steps 41 to S44 are performed on all of the learning images of various types of automobiles stored in the learning image storage unit 4.

[0085] Fig. 12 is a flowchart for explaining the details of the image processing (S42) shown in Fig. 11. First, the image processing unit 32 resizes the input learning image to generate an image of the same size as the image cut out by the cut-out processing unit 23 of the vehicle model identification device 2 (S51). For example, the image processing unit 32 resizes the learning image so that the number of pixels of the image after resizing the learning image is the same as the number of pixels of the image cut out by the cut-out processing unit 23.

[0086] Next, the image processing unit 32 duplicates the resized training image (S52). The image processing unit 32 inverts the color of one of the two duplicated training images (S53), and performs the processes in step S54 and subsequent steps on the other training image without color inverting it. For example, if the training image is a color image, the RGB values ​​of each pixel of the training image are set to R i,j , G i,j , B i,j (However, if the vertical and horizontal sizes of the training image are N and M, respectively, then i∈N, j∈M).

[0087] If the RGB values ​​are expressed as 0 to 255, the RGB values ​​of each pixel after color inversion (R' i,j ,G' i,j ,B' i,j ) can be expressed by the following equation (Equation 3):

[0088] R' i,j =255-R i,j G' i,j =255-G i,j B' i,j =255-B i,j ...(Formula 3) However, even if the learning image is color-inverted using this method, it cannot be converted to any color, so if you want to convert to any color, you can also use this method. Also, after converting a color image to a gray image, the gray image can be inverted.

[0089] Next, the image processing unit 32 performs brightness correction on both the color-inverted training image and the non-color-inverted training image (S54). This brightness correction can reduce the influence of the amount of light depending on the time of day or night when the vehicle is being imaged.

[0090] The RGB values ​​of each pixel in the training image are i,j =(R i,j , G i,j , B i,j ) then the RGB value of each pixel in the training image after brightness correction is img' i,j =(R' i,j , G' i,j , B' i,j ) is as shown in the following equation (Equation 4). Here, α' is a parameter called gain that adjusts the contrast of the image, and β' is a parameter called bias that adjusts the brightness of the image. The int(x) function converts the argument to an integer type, and the clip(dst, min, max) function changes the value to min if the dst value is smaller than min, and changes the value to max if it is larger than max.

[0091] img' i,j =α'×img i,j +β' int(clip(img' i,j ,0,255)) (Equation 4) The image processing unit 32 may also perform other image processing such as adding noise or correcting brightness using a different algorithm.

[0092] Next, the image processing unit 32 creates a transformation matrix for performing homography transformation on the training image (S55). Details of the processing in step S35 will be described later.

[0093] Finally, the image processing unit 32 performs homography transformation on the training image after brightness correction (S56), and the process ends.

[0094] Fig. 13 is a flowchart for explaining the details of the transformation matrix creation process (S55) shown in Fig. 12. First, the image processing unit 32 creates a first transformation matrix that randomly shifts the training image left and right (S61). The homography transformation matrix is ​​as shown in the following equation (Equation 5). (x, y) are the coordinates of each pixel in the training image before transformation, (x', y') are the coordinates of each pixel in the training image after transformation, and s is a proportionality constant.

[0095]

number

[0096] Next, the image processing unit 32 creates a second transformation matrix that randomly shifts the training image up and down within a predetermined range (S62), creates a third transformation matrix that randomly performs trapezoidal correction on the training image within a predetermined range (S63), and creates a fourth transformation matrix that randomly scales the training image within a predetermined range (S64). Note that in steps S61 to S64, various transformation matrices can be created by randomly changing the elements of the homography transformation matrix, and various training images can be created by using these matrices.

[0097] Next, the image processing unit 32 rearranges the four created transformation matrices (first to fourth transformation matrices) in a random order (S65). Finally, the image processing unit 32 calculates the matrix product of the four transformation matrices (first to fourth transformation matrices) (S66), and ends the process.

[0098] The eight elements a to h of the homography transformation matrix shown in (Equation 5) can be found as the solution to a simultaneous equation with eight unknowns by determining four pairs of coordinates in the training image before and after transformation. The image processing using the homography transformation matrix described above determines the transformation matrix by determining four pairs of coordinates in the image before and after transformation for each image processing operation to achieve the specified image processing. This homography transformation can reduce the effects of factors such as the installation location and installation angle of the image capture device 1 and the distance between the image capture device 1 and the vehicle being imaged.

[0099] (Seller identification model generation device 3B) Next, referring back to Fig. 10, the seller identification model generation device 3B will be described. The seller identification model generation device 3B includes a divided image input unit 35, a divided image processing unit 36, a divided image learning unit 37, and a divided image learning model output unit 38.

[0100] The divided image input unit 35 inputs a pair of one seller learning image and a seller label corresponding to that image as seller learning data from the learning image storage unit 4. Then, the divided image input unit 35 divides the image read from the learning image storage unit 4 into nine grid-like segments. Next, the divided image input unit 35 outputs the nine divided pairs of seller learning images and seller labels to the divided image processing unit 36.

[0101] Learning images are required when identifying sellers using machine learning or the like, but preparing learning images is difficult because the environment, such as the installation angle of the imaging device 1, differs depending on the installation location of the imaging device 1. Therefore, the segmented image processing unit 36 ​​performs image processing on the learning images taking into account the operational environment and outputs the results to the segmented image learning unit 37. Specifically, because the environment, such as the installation angle of the imaging device 1 and the surrounding brightness, differs depending on the installation location of the imaging device 1, the segmented image processing unit 36 ​​performs image processing on the learning images, such as coordinate transformation using a homography transformation matrix, brightness correction, and color correction, to generate multiple learning images (second learning images) from one learning image (first learning image). The segmented image processing unit 36 ​​performs similar image processing on learning images of vehicles of various sellers stored in the learning image storage unit 4.

[0102] The divided image learning unit 37 generates a learning model by learning, through machine learning, the relationship between the training images (first training image, second training image) generated by the divided image processing unit 36 ​​and the seller labels corresponding to the training images. The same seller label as that assigned to the first training image is also assigned to the second training image generated by image processing.

[0103] The divided image learning model output unit 38 outputs the learning model learned by the divided image learning unit 37 to the upper center learning model storage unit 6A, the upper right learning model storage unit 6B, the upper left learning model storage unit 7A, the middle center learning model storage unit 7B, the middle right learning model storage unit 8A, and the middle left learning model storage unit 8B, and stores them in the respective storage units. This learning model is a result of learning all of the relationships between the learning images (first learning images) of automobiles of various sellers stored in the learning image storage unit 4 and the learning images (second learning images) generated by image processing, and the corresponding seller labels.

[0104] The upper center learning model storage unit 6A stores a learning model generated from the image in the upper center when the composite image obtained in S33 of FIG. 9 is divided into nine grids. The upper right learning model storage unit 6B stores a learning model generated from the image in the upper right when the composite image obtained in S33 of FIG. 9 is divided into nine grids. The upper left learning model storage unit 7A stores a learning model generated from the image in the upper left when the composite image obtained in S33 of FIG. 9 is divided into nine grids. The middle center learning model storage unit 7B stores a learning model generated from the image in the middle center when the composite image obtained in S33 of FIG. 9 is divided into nine grids. The middle right learning model storage unit 8A stores a learning model generated from the image in the middle right when the composite image obtained in S33 of FIG. 9 is divided into nine grids. The middle left learning model storage unit 8B stores a learning model generated from the image in the middle left when the composite image obtained in S33 of FIG. 9 is divided into nine grids.

[0105] 9 may be divided into grids using any integer other than 9. In this case, an appropriate number of learning model storage units may be prepared based on the number of grids.

[0106] (Processing flow of seller identification model generation device 3B) 14 is a flowchart illustrating the processing steps of the seller identification model generation device 3B according to an embodiment of the present disclosure. First, the divided image input unit 35 inputs pairs of seller training images and seller labels corresponding to the images as training data from the training image storage unit 4 (S71).

[0107] Next, the divided image input unit 35 divides the seller learning image read from the learning image storage unit 4 into nine grids (S72).

[0108] Next, the divided image processing unit 36 ​​performs image processing such as coordinate transformation using a homography transformation matrix, brightness correction, and color correction on the seller training image to generate multiple training images (second training images) from one training image (first training image) (S73). Details of the processing in step S73 will be described later.

[0109] Next, the divided image learning unit 37 uses machine learning to learn the relationship between the first learning image and the second learning image generated by the divided image processing unit 36, and the seller label corresponding to the first image, to generate a learning model (S74).

[0110] Finally, the divided image learning model output unit 38 outputs the learned learning model generated by the divided image learning unit 37 to the upper center learning model storage unit 6A, the upper right learning model storage unit 6B, the upper left learning model storage unit 7A, and the middle center learning model storage unit 7B (S75), and the process ends. Note that the processes of steps 71 to S75 are performed for all of the learning images of automobiles of various sellers stored in the learning image storage unit 4.

[0111] Fig. 15 is a flowchart for explaining the details of the image processing (S73) shown in Fig. 14. First, the divided image processing unit 36 ​​resizes the input training image to generate an image of the same size as the image cut out by the cut-out processing unit 23 of the seller identification device 3 (S81). For example, the divided image processing unit 36 ​​resizes the training image so that the number of pixels of the image after resizing the training image is the same as the number of pixels of the image cut out by the cut-out processing unit 23.

[0112] Next, the divided image processing unit 36 ​​duplicates the resized training image (S82). The divided image processing unit 36 ​​inverts the color of one of the two duplicated training images (S83), and performs the processes in step S84 and subsequent steps on the other training image without color inverting it. For example, if the training image is a color image, the RGB values ​​of each pixel of the training image are set to R i,j , G i,j , B i,j (However, if the vertical and horizontal sizes of the training image are N and M, respectively, then i∈N, j∈M).

[0113] If the RGB values ​​are expressed as 0 to 255, the RGB values ​​of each pixel after color inversion (R' i,j ,G' i,j ,B' i,j ) can be expressed by the following equation (Equation 3):

[0114] R' i,j =255-R i,j G' i,j =255-G i,j B' i,j =255-B i,j ...(Formula 3) However, even if the learning image is color-inverted using this method, it cannot be converted to any color, so if you want to convert to any color, you can also use this method. Also, after converting a color image to a gray image, the gray image can be inverted.

[0115] Next, the divided image processing unit 36 ​​performs brightness correction on both the color-inverted and non-color-inverted training images (S84). This brightness correction reduces the influence of the amount of light depending on the time of day or night when the vehicle is being imaged.

[0116] The RGB values ​​of each pixel in the training image are i,j =(R i,j , G i,j , B i,j ) then the RGB value of each pixel in the training image after brightness correction is img' i,j =(R' i,j , G' i,j , B' i,j ) is as shown in the following equation (Equation 4). Here, α' is a parameter called gain that adjusts the contrast of the image, and β' is a parameter called bias that adjusts the brightness of the image. The int(x) function converts the argument to an integer type, and the clip(dst, min, max) function changes the value to min if the dst value is smaller than min, and changes the value to max if it is larger than max.

[0117] img' i,j =α'×img i,j +β' int(clip(img' i,j ,0,255)) (Equation 4) The divided image processing unit 36 ​​may also perform other image processing such as adding noise or correcting brightness using a different algorithm.

[0118] Next, the divided image processing unit 36 ​​creates a transformation matrix for performing homography transformation on the training image (S85). Details of the processing in step S85 will be described later.

[0119] Finally, the divided image processing unit 36 ​​performs homography transformation on the training image after brightness correction (S86), and the process ends.

[0120] Fig. 16 is a flowchart for explaining the details of the transformation matrix creation process (S85) shown in Fig. 15. First, the divided image processing unit 36 ​​creates a first transformation matrix that randomly shifts the training image left and right (S91). The homography transformation matrix is ​​as shown in the following equation (Equation 5). (x, y) are the coordinates of each pixel in the training image before transformation, (x', y') are the coordinates of each pixel in the training image after transformation, and s is a proportionality constant.

[0121]

number

[0122] Next, the divided image processing unit 36 ​​creates a second transformation matrix that randomly shifts the training image up and down within a predetermined range (S92), creates a third transformation matrix that randomly performs trapezoidal correction on the training image within a predetermined range (S93), and creates a fourth transformation matrix that randomly scales the training image within a predetermined range (S94). In steps S81 to S84, various transformation matrices can be created by randomly changing the elements of the homography transformation matrix, and these can be used to create various training images.

[0123] Next, the divided image processing unit 36 ​​rearranges the four created transformation matrices (first to fourth transformation matrices) in a random order (S95). Finally, the divided image processing unit 36 ​​calculates the matrix product of the four transformation matrices (first to fourth transformation matrices) (S96) and ends the process.

[0124] The eight elements a to h of the homography transformation matrix shown in (Equation 5) can be found as solutions to a simultaneous equation with eight unknowns by determining four pairs of coordinates in the training image before and after transformation.

[0125] In the image processing using the homography transformation matrix described above, four pairs of coordinates in the image before and after transformation are determined for each image processing to obtain the specified image processing. This homography transformation can reduce the effects of factors such as the installation location and installation angle of the image capture device 1, and the distance between the image capture target vehicle and the image capture device 1.

[0126] (Effect of Model Generator 3) As described above, according to the vehicle type identification model generation device 3A, the image processing unit 32 generates the second training image by performing image processing on the first training image. Then, the learning unit 33 learns the relationship between the first training image and the second training image and the corresponding vehicle type label to generate a vehicle type learning model. Therefore, the first vehicle type identification unit 24 of the vehicle type identification device 2 uses the vehicle type learning model learned by the vehicle type identification model generation device 3A to identify the vehicle type from the image clipped by the clipping processing unit 23, making it possible to accurately identify the vehicle type.

[0127] In addition, since the image processing unit 32 generates a second training image for the first training image using a homography transformation matrix, the training unit 33 can create a vehicle type training model that can reduce the influence of the installation location and installation angle of the imaging device 1, the distance between the vehicle to be imaged and the imaging device 1, etc.

[0128] In addition, the image processing unit 32 performs brightness correction on the first learning image to generate a second learning image, so the learning unit 33 can create a vehicle model learning model that can reduce the influence of brightness due to the time of day at the location where the imaging device 1 is installed, etc.

[0129] Furthermore, in the seller identification model generation device 3B according to this embodiment, the segmented image processing unit 36 ​​generates a second training image by performing image processing on the first training image. Then, the segmented image learning unit 37 learns the relationship between the first training image and the second training image and the corresponding seller label to generate a seller learning model. Therefore, the seller identification unit 25 of the vehicle model identification device 2 uses the seller learning model learned by the seller identification model generation device 3B to identify the seller of the vehicle from the image clipped by the clipping processing unit 23, thereby enabling accurate identification of the seller of the vehicle.

[0130] In addition, since the divided image processing unit 36 ​​generates a second training image for the first training image using a homography transformation matrix, the divided image training unit 37 can create a seller training model that can reduce the influence of the installation location and installation angle of the imaging device 1, the distance between the vehicle to be imaged and the imaging device 1, etc.

[0131] In addition, the divided image processing unit 36 ​​performs brightness correction on the first learning image to generate a second learning image, so the divided image learning unit 37 can create a seller learning model that can reduce the influence of brightness due to the time of day at the location where the imaging device 1 is installed, etc.

[0132] Then, the second vehicle type identification unit 26 identifies the vehicle type based on the vehicle type identified by the first vehicle type identification unit 24 and the vehicle seller identified by the seller identification unit 25. This enables the vehicle type identification device 2 to more accurately identify the vehicle seller.

[0133] The second vehicle model identification unit 26 identifies the vehicle model by referring to, but not limited to, an OEM table. The OEM table is a table that organizes all identifiable vehicle models, including whether an OEM exists for a certain vehicle model and, if so, which vehicle model is the OEM, and is created in advance.

[0134] Generally, an OEM vehicle is a vehicle that a seller receives from another company (manufacturer) a vehicle developed and manufactured by the seller, and then sells the vehicle with the seller's own emblem attached. Therefore, an OEM vehicle model is defined as multiple vehicles that have the same shape but are related by different sellers (emblems). The OEM table is a table that organizes such OEM vehicle models and is updated regularly or irregularly with the latest information.

[0135] An example of the OEM table referred to by the second vehicle model identification unit 26 will be described below with reference to Fig. 17. Fig. 17 is a diagram showing an example of the OEM table referred to by the second vehicle model identification unit 26.

[0136] As shown in FIG. 17, the OEM table is organized with "vehicle type," "manufacturer," "OEM vehicle type 1," "OEM seller 1," "OEM vehicle type 2," "OEM seller 2," etc., associated with each other.

[0137] For example, in the No. 1 column, vehicle model "ABC," manufacturer "Company A," OEM vehicle model 1 "DEF," and OEM seller 1 "Company B" are associated. In other words, Company A's vehicle model "ABC" and Company B's vehicle model "DEF" are both OEM vehicle models.

[0138] In column No. 4, vehicle model "PRQ," manufacturer "Company A," OEM vehicle model 1 "STU," OEM distributor 1 "Company B," OEM vehicle model 2 "VWX," and OEM distributor 2 "Company D" are associated. In other words, Company A's vehicle model "PRQ," Company B's vehicle model "STU," and Company D's vehicle model "VWX" are all OEM vehicle models.

[0139] Only the vehicle model "MNO" and manufacturer "Company A" are associated with the No. 3 column. In this way, the OEM table may include information about vehicle models that do not have an OEM vehicle model.

[0140] As described above, the second vehicle model identification unit 26 identifies the vehicle model by referring to the OEM table based on the vehicle model identified by the first vehicle model identification unit 24 and the vehicle seller identified by the seller identification unit 25. This allows the vehicle model identification device 2 to more accurately identify the vehicle seller.

[0141] [Software implementation example] The control blocks of the vehicle type identification device 2 and the model generation device 3 (in particular, the cut-out area size calculation unit 21, the cut-out processing unit 23, the first vehicle type identification unit 24, the image processing unit 32, the learning unit 33, the divided image processing unit 36, and the divided image learning unit 37) may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or by software.

[0142] In the latter case, the vehicle model identification device 2 and the model generation device 3 each include a computer that executes instructions from a program, which is software that realizes each function. This computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. The object of the present disclosure is achieved when the processor in the computer reads and executes the program from the recording medium. The processor may be, for example, a central processing unit (CPU). The recording medium may be a "non-transitory tangible medium," such as a read-only memory (ROM), a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer may also include a random access memory (RAM) for loading the program. The program may be supplied to the computer via any transmission medium (such as a communication network or broadcast waves) capable of transmitting the program. One aspect of the present disclosure may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0143] 〔summary〕 A vehicle model identification device according to aspect 1 of the present disclosure includes a cutout area size calculation unit that calculates a cutout area size based on license plate position coordinates in a captured image of a vehicle; a cutout processing unit that cuts out an image for identifying the vehicle model from the captured image according to the cutout area size calculated by the cutout area size calculation unit; a first vehicle model identification unit that identifies the vehicle model from the image cut out by the cutout processing unit using a vehicle model learning model that has learned the relationship between vehicle model learning images for identifying the vehicle model and vehicle model labels; a seller identification unit that identifies the seller of the vehicle from the image cut out by the cutout processing unit using a seller learning model that has learned the relationship between seller learning images for identifying the vehicle model and seller labels; and a second vehicle model identification unit that identifies the vehicle model based on the vehicle model identified by the first vehicle model identification unit and the seller of the vehicle identified by the seller identification unit.

[0144] In the vehicle model identification device of aspect 2 of the present disclosure, in aspect 1, the vehicle model learning image and the seller learning image each include a first learning image and a second learning image obtained by performing image processing on the first learning image.

[0145] A vehicle model identification device according to a third aspect of the present disclosure is in accordance with the second aspect, wherein the second training image is an image obtained by transforming the first training image using a homography transformation matrix.

[0146] A vehicle model identification device according to aspect 4 of the present disclosure is such that, in aspect 3, the second training image is an image transformed from the first training image using at least one of a first transformation matrix for shifting the image left and right, a second transformation matrix for shifting the image up and down, a third transformation matrix for trapezoidal correction of the image, and a fourth transformation matrix for scaling the image.

[0147] In a vehicle model identification device according to aspect 5 of the present disclosure, in aspect 4, the vehicle model learning image and the seller learning image each include a plurality of second learning images transformed from the first learning image using a matrix product in which the order of a plurality of transformation matrices among the first transformation matrix, the second transformation matrix, the third transformation matrix, and the fourth transformation matrix is ​​rearranged.

[0148] A vehicle model identification device according to a sixth aspect of the present disclosure is in any one of the second to fifth aspects, wherein the second learning image is an image obtained by performing brightness correction on the first learning image.

[0149] A vehicle model identification device according to a seventh aspect of the present disclosure is in any one of the second to sixth aspects, wherein the second training image is an image obtained by performing color inversion on the first training image.

[0150] In the vehicle model identification device according to aspect 8 of the present disclosure, in any of aspects 1 to 7, the cut-out area size calculation unit calculates the cut-out area size when the height coordinate of the license plate position coordinates matches the height coordinate of the reference license plate position coordinates that serve as the basis when the cut-out processing unit cuts out an image.

[0151] In the vehicle model identification device according to aspect 9 of the present disclosure, in any of aspects 1 to 8, when the height coordinate of the license plate position coordinate is defined as Y coordinate, the vertical size of the captured image is defined as α, the cut-out area size when the Y coordinate is maximum is defined as β, and the cut-out area size when the Y coordinate is minimum is defined as γ, the cut-out area size calculation unit calculates the cut-out area size L×L using the following formula (formula 1a).

[0152] L=(β-γ)×y / α+γ (Equation 1a) In a vehicle model identification device according to aspect 9 of the present disclosure, in any of aspects 1 to 9, the seller identification unit identifies the seller of the vehicle by cutting out an image including the vehicle's emblem from the image cut out by the cut-out processing unit.

[0153] A vehicle model identification model generation device according to aspect 11 of the present disclosure includes an image processing unit that generates second training images by performing image processing on first training images for identifying vehicle models and sellers, and a learning unit that learns the relationship between the first training images and the second training images and labels to generate a training model.

[0154] A vehicle model identification model generation method according to aspect 12 of the present disclosure includes the steps of generating second training images by performing image processing on first training images for identifying vehicle models and sellers, and generating a training model by learning the relationship between the first training images and the second training images and labels.

[0155] [Additional Notes] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present disclosure. [Explanation of symbols]

[0156] 1. Imaging device 2 Vehicle identification device 3. Model generation device 3A Vehicle identification model generator 3B Seller identification model generator 4. Training image storage 5 Learning model storage 6A Upper center learning model storage section 6B Upper right learning model storage section 7A Upper left learning model storage section 7B Middle center learning model storage section 8A Middle right learning model storage section 8B Middle left learning model storage section 11 Imaging unit 12 Position coordinate detection unit 21 Area size calculation unit 22 Configuration Files 23 Processing section 24 First Vehicle Identification Department 25 Seller Identification Section 26 2nd Vehicle Identification Department 31 Learning image input unit 32 Image processing section 33 Learning Department 34 Learning model output section 35 Divided image input unit 36 Segmented image processing section 37 Segmented image learning unit 38 Segmented image learning model output unit 100 Vehicle Identification System

Claims

1. a cutout region size calculation unit that calculates a cutout region size based on the position coordinates of the license plate in the captured image of the automobile; a cutout processing unit that cuts out an image for identifying the vehicle model from the captured image in accordance with the cutout region size calculated by the cutout region size calculation unit; a first vehicle model identification unit that identifies the vehicle model from the image cut out by the cutout processing unit using a vehicle model learning model that has learned the relationship between a vehicle model learning image for identifying the vehicle model and a vehicle model label; a seller identification unit that identifies the seller of the automobile from the image cut out by the cutout processing unit using a seller learning model that has learned the relationship between a seller learning image and a seller label for identifying the seller of the automobile; A vehicle model identification device comprising: a second vehicle model identification unit that identifies the vehicle model based on the vehicle model identified by the first vehicle model identification unit and the seller of the vehicle identified by the seller identification unit.

2. 2. The vehicle model identification device according to claim 1, wherein the vehicle model learning image and the seller learning image each include a first learning image and a second learning image obtained by performing image processing on the first learning image.

3. The vehicle type identification device according to claim 2 , wherein the second training image is an image obtained by transforming the first training image using a homography transformation matrix.

4. 4. The vehicle model identification device according to claim 3, wherein the second training image is an image transformed from the first training image using at least one of a first transformation matrix for shifting the image left or right, a second transformation matrix for shifting the image up or down, a third transformation matrix for performing trapezoidal correction on the image, and a fourth transformation matrix for enlarging or reducing the image.

5. 5. The vehicle model identification device according to claim 4, wherein the vehicle model learning images and the seller learning images each include a plurality of second learning images transformed from the first learning images using a matrix product in which the order of a plurality of transformation matrices selected from the first transformation matrix, the second transformation matrix, the third transformation matrix, and the fourth transformation matrix is ​​rearranged.

6. 6. The vehicle type identification device according to claim 2, wherein the second learning image is an image obtained by performing brightness correction on the first learning image.

7. 6. The vehicle type identification device according to claim 2, wherein the second learning image is an image obtained by performing color inversion on the first learning image.

8. The cut-out area size calculation unit calculates the cut-out area size when the height coordinate of the license plate position coordinates and the height coordinate of the reference license plate position coordinates that serve as the reference when the cut-out processing unit cuts out the image match. A vehicle model identification device as described in any one of claims 1 to 5.

9. 9. The vehicle model identification device according to claim 8, wherein the height coordinate of the license plate position coordinates is defined as Y coordinate, the vertical size of the captured image is defined as α, the cut-out region size when the Y coordinate is maximum is defined as β, and the cut-out region size when the Y coordinate is minimum is defined as γ. The cut-out region size calculation unit calculates the cut-out region size L × L using the following formula (Formula 1a). L=(β-γ)×y / α+γ...(Formula 1a)

10. 2. The vehicle model identification device according to claim 1, wherein the seller identification unit identifies the seller of the vehicle by cutting out an image including an emblem of the vehicle from the image cut out by the cutout processing unit.

11. an image processing unit that performs image processing on each of the first learning images for identifying the vehicle model and the seller of the vehicle to generate each of the second learning images; a learning unit that learns the relationship between the first training image and the second training image and a label to generate a learning model.

12. generating second learning images by performing image processing on the first learning images for identifying the vehicle type and the seller of the vehicle; and generating a learning model by learning the relationship between the first learning image and the second learning image and a label.

Images