Image processing device, image processing system, and image processing method

Abstract

To provide a technology capable of cutting out an image so as to discriminate an automobile type with high accuracy.SOLUTION: A cut-out area size calculation unit (21) calculates a cut-out area size on the basis of positional coordinates of a number plate in a captured image obtained by imaging a travelling automobile. A cut-out processing unit (23) cuts out an image for discriminating a type of the automobile from the captured image according to a cut-out area size calculated by the cut-out area size calculation unit (21).SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing apparatus, an image processing system, and an image processing method. [Background technology]

[0002] Various technologies for identifying the type of automobile have been developed over time. Related technologies include the inventions disclosed in Patent Documents 1 and 2 below.

[0003] Patent Document 1 relates to a vehicle type matching system. The vehicle type matching system comprises an in-vehicle device that outputs first vehicle type data, which is visual information; a vehicle type identification device that outputs second vehicle type data of a vehicle equipped with the in-vehicle device; a matching device that compares the first vehicle type data and the second vehicle type data; and a management center that notifies the vehicle owner of the matching result when the vehicle types indicated by the first vehicle type data and the second vehicle type data do not match.

[0004] Patent Document 2 relates to a technology for determining the type of vehicle passing by. The acquisition unit acquires a first image of the vehicle captured by one of the multiple imaging devices, and a second image captured by the imaging device at a different time than the first image. The detection unit detects a first image of the vehicle from the first image and a second image of the vehicle from the second image. The first discrimination unit determines the type of vehicle based on the first image and discrimination criteria. The second discrimination unit determines the type of vehicle based on the second image and discrimination criteria. The integrated discrimination unit integrates the discrimination result of the first discrimination unit and the discrimination result of the second discrimination unit to determine the type of vehicle. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2003-281687 (Published October 3, 2003) [Patent Document 2] Japanese Patent Publication No. 2018-190082 (Published November 29, 2018) [Overview of the project] [Problems that the invention aims to solve]

[0006] In the aforementioned Patent Document 1, the vehicle type is determined by comparing visually detected information (visual information) with information determined by a vehicle type determination device (determination information). However, there is no disclosure regarding how the captured images are processed.

[0007] Furthermore, in Patent Document 2, the integrated discrimination unit integrates the discrimination result of the first discrimination unit and the discrimination result of the second discrimination unit to determine the vehicle type. However, there is no disclosure regarding how to crop the image used for vehicle type determination.

[0008] One aspect of the present invention aims to provide a technology that enables image extraction for high-precision vehicle type identification. [Means for solving the problem]

[0009] To solve the above problems, an image processing apparatus according to one aspect of the present invention includes: a cropping area size calculation unit that calculates the size of the cropping area based on the license plate position coordinates in a captured image of a moving automobile; and a cropping processing unit that crops an image for determining the type of automobile from the captured image according to the cropping area size calculated by the cropping area size calculation unit.

[0010] Furthermore, in order to solve the above problems, an image processing system according to one aspect of the present invention is an image processing system comprising an imaging device and an image processing device, wherein the imaging device includes an imaging unit for photographing a moving automobile and a position coordinate detection unit for detecting the position coordinates of the license plate of the automobile photographed by the imaging unit, and the image processing device includes a cropping area size calculation unit for calculating the size of the cropping area based on the license plate position coordinates detected by the position coordinate detection unit and a cropping processing unit for cropping an image for determining the type of automobile from the captured image according to the cropping area size calculated by the cropping area size calculation unit.

[0011] Furthermore, in order to solve the above problems, an image processing method according to one aspect of the present invention includes the steps of: calculating the size of a cropping area based on the license plate position coordinates in a photographed image of a moving automobile; and extracting an image from the photographed image for determining the type of automobile according to the calculated cropping area size. [Effects of the Invention]

[0012] According to one aspect of the present invention, it is possible to extract an image in a way that enables high-precision vehicle type identification. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram showing the schematic configuration of an image processing system according to one embodiment of the present invention. [Figure 2] This is a diagram illustrating the coordinates of the license plate position. [Figure 3] This table shows the relationship between the Y-coordinate of the license plate position and the size of the cropping area. [Figure 4] This figure shows an example of a cutting area. [Figure 5] This figure shows an example of an image extracted by the extraction processing unit. [Figure 6] This is a flowchart illustrating the processing procedure of an image processing apparatus according to one embodiment of the present invention.

Embodiment for Carrying out the Invention

[0014] (Configuration Example of Image Processing System 100) FIG. 1 is a block diagram showing a schematic configuration of an image processing system 100 according to an embodiment of the present invention. The image processing system 100 includes an imaging device 1 and an image processing device 2. This image processing system 100 can photograph an automobile traveling on a road and cut out an image for discriminating the vehicle type. Also, the image processing system 100 may photograph an automobile entering or leaving a parking lot and cut out an image for discriminating the vehicle type. Note that the vehicle type not only means vehicle types such as trucks and light automobiles but also means the vehicle name of an automobile. Therefore, the image processing device 2 cuts out an image so as to be able to discriminate the vehicle name of an automobile.

[0015] The imaging device 1 includes an imaging unit 11 and a position coordinate detection unit 12. The imaging unit 11 is constituted by a camera or the like and photographs an image of an automobile traveling. The imaging unit 11 is installed, for example, at a height of about 6 m and photographs the front (front part) of an automobile traveling from obliquely above. Note that the imaging unit 11 may photograph the back of an automobile traveling.

[0016] The position coordinate detection unit 12 extracts an image portion of a license plate from an image of the front of the photographed automobile and detects license plate position coordinates. The detection of the license plate position coordinates will be described with reference to FIG. 2.

[0017] FIG. 2 is a diagram for explaining license plate position coordinates. As shown in FIG. 2, on the license plate, there are lines indicating the region name "Kyoto" and the classification number "xxx", and lines indicating the use "a" and the serial number "xx-xx". The position coordinate detection unit 12 detects the coordinates of a point P that is between the line indicating the region name and the classification number and the line indicating the use and the serial number and is at the center of the horizontal width as the license plate position coordinates and outputs them to the image processing device 2. Note that the license plate position coordinates are not limited to this and may be other coordinates based on the license plate.

[0018] As shown in Figure 2, the length from point P to the left edge of the license plate is W. L The length from point P to the right edge of the license plate is W. R Therefore, the position coordinate detection unit 12 is W L =W R Point P is determined such that the following occurs. The coordinates of point P in the image captured by the imaging unit 11 become the license plate position coordinates. Hereafter, the license plate position coordinates, i.e., the coordinates of point P, will be denoted as P(x,y). Note that the license plate position coordinates P(x,y) are the position coordinates of the license plate when the captured image is represented in XY coordinates.

[0019] Returning to the explanation of Figure 1, the image processing device 2 includes a cropping area size calculation unit 21, a setting file 22, and a cropping processing unit 23. The input and output of information between the imaging device 1 and the image processing device 2 may be via wired communication or wireless communication, and is not particularly limited.

[0020] The cropping area size calculation unit 21 calculates the cropping area size when the Y coordinate of the license plate position coordinate P(x,y) output from the position coordinate detection unit 12 matches the Y coordinate of the reference license plate position coordinate stored in the setting file 22. The reference license plate position coordinate is the position coordinate that the cropping processing unit 23 uses as a reference when cropping the image, and is predetermined.

[0021] As described above, the imaging unit 11 is installed at a height of approximately 6m and photographs the front of the car from above. Therefore, if the car is far away, the Y coordinate (height coordinate) of the license plate position coordinate P(x,y) will be located higher up in the captured image. As the car approaches, the license plate position coordinate P(x,y) moves downward, and when the distance between the car and the imaging unit 11 reaches a predetermined distance, the license plate position coordinate and the reference license plate position coordinate will coincide. The captured image (still image) at this time will be used as the image to be cropped.

[0022] The configuration file 22 defines information such as the size of the captured image (number of pixels in width and height), the reference license plate position coordinates, and parameters α, β, and γ used when calculating the cropping area size.

[0023] α is the vertical size (number of pixels in the vertical direction) of the image to be processed (for example, it may be the entire captured image or a part of the captured image). β is the size of the cropped area when the Y coordinate of the license plate position coordinate P(x,y) is at its maximum. γ is the size of the cropped area when the Y coordinate of the license plate position coordinate P(x,y) is at its minimum. Note that the cropped area size is described as a square with equal vertical and horizontal dimensions, but it may also be a rectangle with different vertical and horizontal dimensions, and is not particularly limited. The following description will focus on the case where the image to be processed is the entire captured image.

[0024] For example, suppose the size of the image to be processed is 1024×768, the size of the cropped area when the Y coordinate of the license plate position coordinate P(x,y) is at its maximum is 300×300, and the size of the cropped area when the Y coordinate of the license plate position coordinate P(x,y) is at its minimum is 200×200. In this case, α=768, β=300, and γ=200. These parameters are predefined in configuration file 22. Note that if the top left of the target image to be processed is taken as the origin, the minimum Y coordinate of the license plate position coordinate P(x,y) is "0" and the maximum is "767".

[0025] The cutout area size calculation unit 21 uses parameters α, β, and γ to calculate the cutout area size L × L corresponding to the Y coordinate of the license plate position coordinates using the following equation (Equation 1), and defines it as a cutout area size table in the setting file 22.

[0026] L=(β-γ)×y / α+γ (Formula 1) The image being processed has its origin at the top left. Therefore, the Y coordinate of the top of the image being processed is "0", and the Y coordinate of the bottom of the image being processed is "767".

[0027] Furthermore, substituting the values ​​of parameters α, β, and γ into (Equation 1) above yields the following equation (Equation 2). The excision region size calculation unit 21 may calculate the excision region size using the following equation (Equation 2).

[0028] L=0.13×Y+200 (Formula 2) Figure 3 is a table showing the relationship between the Y-coordinate of the license plate position and the size of the cropped area. As shown in Figure 3, when the Y-coordinate of the license plate position is at its minimum value of "0", the cropped area size is 200 x 200, and the size of the cropped area gradually increases as the Y-coordinate increases. When the Y-coordinate of the license plate position is at its maximum value of "767", the cropped area size becomes 300 x 300.

[0029] Thus, when the distance from the imaging device 1 to the vehicle is large, the cropping area size is reduced. Conversely, when the distance from the imaging device 1 to the vehicle is small, the cropping area size is increased. This ensures that the image is cropped to include the same portion of the front of the vehicle. If the reference license plate position coordinates are fixed, the cropping area size calculation unit 21 may calculate only the cropping area size corresponding to the reference license plate position coordinates and define this value in the setting file 22.

[0030] The cropping area size calculation unit 21 outputs the captured image, the license plate position coordinates, and the cropping area size to the cropping processing unit 23 when the Y coordinate of the license plate position coordinates matches the Y coordinate of the reference license plate position coordinates.

[0031] The cropping processing unit 23 uses the captured image, license plate position coordinates, and cropping area size output from the cropping area size calculation unit 21 to extract an image of the cropping area for determining the vehicle type from the image to be processed.

[0032] Figure 4 is a diagram showing an example of a cutout area. As shown in Figure 4, for example, based on the license plate position coordinates P(x, y), in the horizontal direction (x direction), W L :W R =1:1 to determine the cutout area. Note that W L is the size (number of pixels) from the X coordinate of the license plate position coordinates to the left end, and W R is the size (number of pixels) from the X coordinate of the license plate position coordinates to the right end. Note that the ratio of W L to W R may be other values.

[0033] Also, in the vertical direction (y direction), H U :H L =3:1 to determine the cutout area. Note that H U is the size (number of pixels) from the Y coordinate of the license plate position coordinates to the upper end, and H L is the size (number of pixels) from the Y coordinate of the license plate position coordinates to the lower end. Note that the ratio of H U to H L may be other values.

[0034] Note that when the cutout area exceeds the range of the captured image, the cutout processing unit 23 may black-mask the part that exceeds the range of the captured image and keep it square.

[0035] [[ID=三十九]]Figure 5 is a diagram showing an example of an image cut out by the cutout processing unit 23. As shown in Figure 5, the front of the automobile is cut out to a size that does not include the background. The setting file 22 defines the reference license plate position coordinates, α, β, and γ so as not to include the background image of the automobile. In the vehicle type discrimination process, parts such as the headlights, emblems, bonnets, and front grills of the front of the automobile are extracted from the cut-out image, and the vehicle type of the automobile is discriminated using AI technologies such as neural networks from these parts.

[0036] (Processing flow of image processing device 2) Figure 6 is a flowchart illustrating the processing procedure of an image processing apparatus 2 according to one embodiment of the present invention. First, the cropping area size calculation unit 21 receives the captured image and the license plate position coordinates as input (S1).

[0037] Next, the cutout area size calculation unit 21 determines whether the Y coordinate of the license plate position coordinate matches the Y coordinate of the reference license plate position coordinate defined in the configuration file (S2). If the Y coordinate of the license plate position coordinate does not match the Y coordinate of the reference license plate position coordinate (S2, No), the process returns to step S1 and the subsequent steps are repeated.

[0038] Furthermore, if the Y-coordinate of the license plate position coordinates matches the Y-coordinate of the reference license plate position coordinates (S2, Yes), the cropping area size calculation unit 21 selects the captured image (still image) to be cropped using the method described above and calculates the cropping area size (S3).

[0039] Finally, the cropping processing unit 23 uses the captured image, license plate position coordinates, and cropping area size output from the cropping area size calculation unit 21 to extract an image of the cropping area for determining the vehicle type from the image to be processed (S4).

[0040] (Effect of image processing device 2) As described above, according to the image processing apparatus 2 of this embodiment, the cropping area size calculation unit 21 calculates the cropping area size based on the license plate position coordinates in the captured image. Therefore, the cropping area size calculation unit 21 can easily calculate the cropping area size.

[0041] Furthermore, the cropping area size calculation unit 21 calculates the cropping area size when the Y coordinate of the license plate position coordinate output from the position coordinate detection unit 12 matches the Y coordinate of the reference license plate position coordinate stored in the setting file 22. Therefore, the cropping area size calculation unit 21 can calculate a cropping area size suitable for determining the type of vehicle. In addition, since only the coordinate of one point on the license plate is used, it is less affected by the position of the imaging device 1.

[0042] Furthermore, the cutout area size calculation unit 21 calculates the cutout area size L×L using (Equation 1). Therefore, the cutout area size calculation unit 21 can calculate the cutout area size L×L simply by referring to the various parameters defined in the configuration file 22.

[0043] Furthermore, the cropping processing unit 23 crops the image based on the license plate position coordinates, such that the vertical and horizontal dimensions of the cropping area calculated by the cropping area size calculation unit are in a predetermined ratio. Therefore, the cropping processing unit 23 can crop the image to include the same portion of the front of the vehicle.

[0044] Furthermore, the configuration file 22 now defines the reference license plate position coordinates, α, β, and γ in a way that does not include the background image of the car. As a result, it becomes possible to identify the type of car with high accuracy.

[0045] <Examples of implementation using software> The control block of the image processing device 2 (in particular, the cropping area size calculation unit 21 and the cropping processing unit 23) may be implemented by logic circuits (hardware) formed on an integrated circuit (IC chip) or the like, or by software.

[0046] In the latter case, the extraction area size calculation block and the extraction processing block are equipped with a computer that executes instructions for a program, which is software that implements each function. This computer is equipped with, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the program. The object of the present invention is achieved when the processor reads the program from the recording medium and executes it in the computer. For example, a CPU (Central Processing Unit) can be used as the processor. As the recording medium, a "tangible medium that is not temporary," such as ROM (Read Only Memory), can be used, as well as tape, disk, card, semiconductor memory, programmable logic circuit, etc. It may also be further equipped with RAM (Random Access Memory) for deploying the program. Furthermore, the program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast wave). In one aspect of the present invention, the program can also be realized in the form of a data signal embedded in a carrier wave, which is embodied by electronic transmission.

[0047] 〔summary〕 An image processing apparatus according to embodiment 1 of the present invention includes a cropping area size calculation unit that calculates the cropping area size based on the license plate position coordinates in a captured image of a moving automobile, The system includes a cropping processing unit that extracts an image from the captured image for determining the type of automobile, according to the cropping area size calculated by the cropping area size calculation unit.

[0048] An image processing apparatus according to aspect 2 of the present invention is the image processing apparatus described in aspect 1, wherein the cropping area size calculation unit calculates the cropping area size when the height coordinates of the license plate position coordinates and the height coordinates of the reference license plate position coordinates that serve as the reference when the cropping processing unit crops the image coincide.

[0049] The image processing apparatus according to aspect 3 of the present invention is the image processing apparatus according to aspect 1 or 2, wherein the height coordinate of the license plate position coordinate is the Y coordinate, the vertical size of the captured image is α, the size of the cropped area when the Y coordinate is at its maximum is β, and the size of the cropped area when the Y coordinate is at its minimum is γ, the cropped area size calculation unit calculates the cropped area size L × L by the following equation (Equation 1a).

[0050] L=(β-γ)×y / α+γ (Equation 1a) An image processing apparatus according to embodiment 4 of the present invention is an image processing apparatus according to embodiment 3, wherein the image processing apparatus further includes a configuration file that defines at least the reference license plate position coordinates, α, β, and γ, The cutout area size calculation unit calculates the cutout area size by referring to the setting file.

[0051] An image processing apparatus according to embodiment 5 of the present invention is an image processing apparatus according to embodiment 4, wherein the setting file defines the reference license plate position coordinates, α, β, and γ such that it does not include the background image of the automobile.

[0052] An image processing apparatus according to embodiment 6 of the present invention is an image processing apparatus according to embodiment 4 or 5, wherein the cropping area size calculation unit calculates the cropping area size L × L for each Y coordinate of the license plate position coordinate and defines it in the setting file as a table showing the relationship between the Y coordinate of the license plate position coordinate and the cropping area size L × L.

[0053] An image processing apparatus according to embodiment 7 of the present invention is an image processing apparatus according to any one of embodiments 1 to 6, wherein the cropping processing unit crops the image such that the vertical and horizontal sizes of the cropping area calculated by the cropping area size calculation unit, based on the license plate position coordinates, are in a predetermined ratio.

[0054] An image processing system according to aspect 8 of the present invention is an image processing system comprising an imaging device and an image processing device, The aforementioned imaging device includes an imaging unit that photographs a moving vehicle, It includes a position coordinate detection unit that detects the position coordinates of the vehicle's license plate from the captured image taken by the imaging unit, The image processing apparatus includes a cropping area size calculation unit that calculates the cropping area size based on the license plate position coordinates detected by the position coordinate detection unit, The system includes a cropping processing unit that extracts an image from the captured image for determining the type of vehicle, according to the cropping area size calculated by the cropping area size calculation unit.

[0055] The image processing method according to aspect 9 of the present invention includes the step of calculating the size of the cropping area based on the license plate position coordinates in a captured image of a moving automobile, The process includes extracting an image from the captured image for determining the type of vehicle, according to the calculated size of the extraction region.

[0056] [Additional Notes] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of Symbols]

[0057] 1. Imaging device 2 Image Processing Device 11 Imaging Unit 12 Position coordinate detection unit 21. Cutout area size calculation unit 22 Configuration files 23 Cutting Processing Unit 100 Image Processing Systems

Claims

1. A cropping area size calculation unit calculates the size of the cropping area based on the license plate position coordinates in the captured image of a moving vehicle, The system includes a cropping processing unit that extracts an image from the captured image for determining the type of automobile, according to the cropping area size calculated by the cropping area size calculation unit, The cropping area size calculation unit calculates the cropping area size when the height coordinates of the license plate position coordinates coincide with the height coordinates of the reference license plate position coordinates that serve as the reference when the cropping processing unit crops the image.

2. A cropping area size calculation unit calculates the size of the cropping area based on the license plate position coordinates in the captured image of a moving vehicle, The system includes a cropping processing unit that extracts an image from the captured image for determining the type of automobile, according to the cropping area size calculated by the cropping area size calculation unit, The image processing apparatus calculates the cropping area size L × L by the following equation (Equation 1a), where the Y-coordinate is the height coordinate of the license plate position, α is the vertical size of the captured image, β is the size of the cropping area when the Y-coordinate is at its maximum, and γ is the size of the cropping area when the Y-coordinate is at its minimum. L=(β-γ)×y / α+γ...(Formula 1a)

3. The image processing apparatus further includes a configuration file that defines at least the reference license plate position coordinates, α, β, and γ which serve as the reference when the cropping processing unit crops an image, The image processing apparatus according to claim 2, wherein the cropping area size calculation unit calculates the cropping area size by referring to the setting file.

4. The image processing apparatus according to claim 3, wherein the configuration file defines the reference license plate position coordinates, α, β, and γ such that the configuration file does not include the background image of the automobile.

5. The image processing apparatus according to claim 3 or 4, wherein the cropping area size calculation unit calculates the cropping area size L × L for each Y coordinate of the license plate position coordinate and defines it in the setting file as a table showing the relationship between the Y coordinate of the license plate position coordinate and the cropping area size L × L.

6. The image processing apparatus according to any one of claims 1 to 5, wherein the cropping processing unit crops the image such that the vertical and horizontal sizes of the cropping area calculated by the cropping area size calculation unit are in a predetermined ratio, based on the license plate position coordinates.

7. An image processing system comprising an imaging device and an image processing device, The aforementioned imaging device includes an imaging unit that photographs a moving vehicle, It includes a position coordinate detection unit that detects the position coordinates of the vehicle's license plate from the captured image taken by the imaging unit, The image processing apparatus includes a cropping area size calculation unit that calculates the cropping area size based on the license plate position coordinates detected by the position coordinate detection unit, Includes a cropping processing unit that crops an image from the captured image for determining the type of vehicle, according to the cropping area size calculated by the cropping area size calculation unit, The cropping area size calculation unit calculates the cropping area size when the height coordinates of the license plate position coordinates coincide with the height coordinates of the reference license plate position coordinates that serve as the basis for the cropping processing unit when cropping the image.

8. An image processing system comprising an imaging device and an image processing device, The aforementioned imaging device includes an imaging unit that photographs a moving vehicle, It includes a position coordinate detection unit that detects the position coordinates of the vehicle's license plate from the captured image taken by the imaging unit, The image processing apparatus includes a cropping area size calculation unit that calculates the cropping area size based on the license plate position coordinates detected by the position coordinate detection unit, Includes a cropping processing unit that crops an image from the captured image for determining the type of vehicle, according to the cropping area size calculated by the cropping area size calculation unit, The image processing system calculates the cropping area size L × L using the following formula (Formula 1a), where the Y-coordinate is the height coordinate of the license plate position, α is the vertical size of the captured image, β is the size of the cropping area when the Y-coordinate is at its maximum, and γ is the size of the cropping area when the Y-coordinate is at its minimum. L=(β-γ)×y / α+γ...(Formula 1a)

9. A process of calculating the size of the cropping area based on the license plate position coordinates in an image taken of a moving vehicle, The process includes the step of extracting an image from the captured image for determining the type of vehicle, according to the calculated size of the extraction region, An image processing method in which, in the step of calculating the size of the cropped area, the size of the cropped area is calculated when the height coordinates of the license plate position coordinates and the height coordinates of the reference license plate position coordinates that serve as the basis for cropping the image coincide.

10. A process of calculating the size of the cropping area based on the license plate position coordinates in an image taken of a moving vehicle, The process includes the step of extracting an image from the captured image for determining the type of vehicle, according to the calculated size of the extraction region, An image processing method in which, in the step of calculating the size of the cropped area, the size L × L is calculated by the following equation (Equation 1a), where α is the vertical size of the captured image, β is the size of the cropped area when the Y coordinate is at its maximum, and γ is the size of the cropped area when the Y coordinate is at its minimum. L=(β-γ)×y / α+γ...(Formula 1a)

Images