Image processing device, image processing method and program

Abstract

To improve the accuracy of detecting a traffic light through image recognition.SOLUTION: An image processing apparatus mounted on a vehicle includes a control unit. The control unit determines, on the basis of an aspect ratio of a rectangular region detected as a traffic light through image recognition on a camera image, whether the rectangular region is a detection region for the traffic light with three lights, at least.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The disclosed embodiments relate to an image processing device, an image processing method, and a program. [Background technology]

[0002] Conventionally, a technology has been proposed for determining whether a vehicle has run a red light, based on the light status (green, yellow, red, etc.) of a traffic light detected from information received by an optical beacon or an image captured by a camera, and the vehicle's operating status (e.g., braking) (see Patent Document 1, for example).

[0003] In these technologies, when determining the illumination state of a traffic light using an image captured by a camera, it is necessary to detect the traffic light itself in the image. In such cases, an image recognition method is known that uses a machine learning algorithm such as deep learning to extract the target traffic light as a rectangular area called a bounding box. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-069051 Summary of the Invention [Problem to be solved by the invention]

[0005] However, there is room for further improvement in the conventional technology in terms of improving the accuracy of detecting traffic lights using image recognition.

[0006] For example, when using the image recognition method described above, although the traffic light itself can be detected, there may be a discrepancy in the estimated position of the traffic light, i.e., the extracted position of the rectangular area. Also, there may be cases where objects other than traffic lights, such as signs, are mistakenly detected as traffic lights.

[0007] One aspect of the embodiment has been made in consideration of the above, and aims to provide an image processing device, an image processing method, and a program that can improve the accuracy of traffic light detection using image recognition. [Means for solving the problem]

[0008] According to an embodiment, an image processing device is mounted on a vehicle and includes a control unit, wherein the control unit determines whether a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light, based on an aspect ratio of the rectangular area. If the rectangular area is determined to be the detection area of ​​the traffic light and the feature amount of a specific shape extracted from the rectangular area does not satisfy a threshold, the rectangular area is expanded. . [Effects of the Invention]

[0009] According to one aspect of the embodiment, it is possible to improve the accuracy of detecting traffic signals by image recognition. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram (part 1) outlining an image processing method according to an embodiment. [Figure 2] FIG. 2 is a diagram (part 2) outlining the image processing method according to the embodiment. [Figure 3] FIG. 3 is a diagram (part 3) outlining the image processing method according to the embodiment. [Figure 4] FIG. 4 is a schematic explanatory diagram (part 4) of the image processing method according to the embodiment. [Figure 5] FIG. 5 is a schematic explanatory diagram (part 5) of the image processing method according to the embodiment. [Figure 6] FIG. 6 is a block diagram illustrating an example of the configuration of an image processing apparatus according to the embodiment. [Figure 7] FIG. 7 is an explanatory diagram of an image recognition AI according to the embodiment. [Figure 8] FIG. 8 is a supplementary explanatory diagram (part 1) of the correction process. [Figure 9] FIG. 9 is a supplementary explanatory diagram (part 2) of the correction process. [Figure 10] FIG. 10 is a flowchart showing a processing procedure executed by the image processing apparatus according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of an image processing device, an image processing method, and a program disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0012] In the following description, the image processing device 10 according to the embodiment is assumed to be an in-vehicle device mounted on a vehicle. The image processing device 10 is a device capable of detecting an object in an image based on an image captured by a camera 3 mounted on the vehicle, such as a red light violation determination device or a drive recorder. In the present embodiment, the object is assumed to be at least a traffic light 300.

[0013] First, an overview of an image processing method according to an embodiment will be described with reference to FIGS. 1 to 5. FIG. 1 is a diagram (part 1) outlining an image processing method according to an embodiment. FIG. 2 is a diagram (part 2) outlining an image processing method according to an embodiment. FIG. 3 is a diagram (part 3) outlining an image processing method according to an embodiment. FIG. 4 is a diagram (part 4) outlining an image processing method according to an embodiment. FIG. 5 is a diagram (part 5) outlining an image processing method according to an embodiment.

[0014] As already mentioned, when determining the light emission state of the traffic light 300 using the image captured by the camera 3, it is necessary to detect the traffic light 300 itself in the image. In such a case, for example, a machine learning algorithm such as deep learning is used to extract the traffic light 300 to be detected as a rectangular area called a bounding box.

[0015] However, when such an image recognition method is used, (1) although the traffic light 300 itself can be detected, there may be a deviation in the estimated position of the traffic light 300, i.e., the extracted position of the rectangular area BB, and (2) there may be a case where an object other than the traffic light 300, such as a sign, is erroneously detected as the traffic light 300.

[0016] The left diagram in Fig. 1 shows a case where there is no deviation in rectangular area BB1, which is the detection area of ​​traffic light 300, while the center and right diagrams in the same figure show cases where there are deviations in rectangular areas BB2 and BB3. Also, Fig. 2 shows a case where sign 500 is erroneously detected as rectangular area BB4 of traffic light 300. This phenomenon occurs because image recognition using a DNN (Deep Neural Network) model or the like trained using a machine learning algorithm detects an object in an unknown image based on the likelihood (accuracy) of what the object is.

[0017] Therefore, in the image processing method according to the embodiment, the control unit 12 (see FIG. 6) of the image processing device 10 determines whether the rectangular area BB detected as the traffic light 300 by image recognition of the camera image is at least the detection area of ​​the three-light traffic light 300, based on the aspect ratio of the rectangular area BB. The control unit 12 further expands the rectangular area BB by a predetermined ratio. The control unit 12 further corrects the position of or expands the rectangular area BB, based on the position of the light-emitting area in the rectangular area BB detected by the image recognition.

[0018] Specifically, as shown in Fig. 3, the control unit 12 determines whether the rectangular area BB extracted as the traffic light 300 is at least the detection area of ​​the three-light traffic light 300 based on the aspect ratio of the rectangular area BB. As shown in the figure, the control unit 12 determines that the rectangular area BB is the detection area of ​​the traffic light 300 if the "aspect ratio value ≠ 1.0." This is because when the traffic light 300 is a three-light traffic light, regardless of whether it is horizontal or vertical, there is always a difference in the lengths of the long and short sides, and therefore the aspect ratio value of the rectangular area BB is not 1.0. In other words, "aspect ratio value ≠ 1.0" means that the aspect ratio value is not an approximation of 1.0.

[0019] On the other hand, if the "aspect ratio value ≈ 1.0" of the rectangular area BB, the control unit 12 determines that the rectangular area BB is not the detection area of ​​the traffic light 300, and excludes the rectangular area BB from the image processing target for the traffic light 300, regarding the rectangular area BB as a false detection. This solves the problem (2) above. Note that the "aspect ratio value ≈ 1.0" can also be rephrased as meaning that the aspect ratio value is an approximation of 1.0.

[0020] Furthermore, the control unit 12 spatially corrects the positional deviation of the rectangular area BB. As shown in Fig. 4, the control unit 12 expands the rectangular area BB extracted as the traffic light 300 by a predetermined ratio, thereby spatially correcting the positional deviation of the rectangular area BB.

[0021] As shown in the left diagram of the same figure, the rectangular area BB of the traffic light 300 is not necessarily extracted to surround the entire traffic light 300. Therefore, as shown in the right diagram of the same figure, by expanding the rectangular area BB by a predetermined ratio, it is possible to correct the rectangular area BB so that it surrounds the entire traffic light 300, even if a positional deviation occurs. This solves the problem (1) above.

[0022] The control unit 12 may expand the rectangular area BB uniformly, or may expand the rectangular area BB when a predetermined condition is satisfied. For example, the predetermined condition may be satisfied when a feature quantity of a specific shape extracted from the rectangular area BB, such as three circles, does not meet a predetermined threshold. In such a case, it is estimated that a positional deviation has occurred in the rectangular area BB due to the lack of the feature quantity of the light-emitting element of the traffic light 300, i.e., three circles.

[0023] Furthermore, a learning model such as the above-described DNN model can detect the light emission state of the traffic light 300 by learning in advance images showing each light emission state, such as blue light, yellow light, and red light, of the traffic light 300. Then, the control unit 12 can spatially correct the positional deviation of the rectangular area BB based on the light emission state.

[0024] 5, when the control unit 12 detects the green light emitted from the traffic light 300 near the center of the rectangular area BB, it can infer that the rectangular area BB is shifted to the left, and therefore corrects the position of the rectangular area BB to the right (see rectangular area BB1 in the figure) or expands the rectangular area BB (see rectangular area BB2 in the figure). As a result, even if a positional shift occurs, the rectangular area BB can be corrected to surround the entire traffic light 300, as shown in FIG. 4, and the problem (1) above can be resolved.

[0025] As described above, the image processing method according to the embodiment is an image processing method executed by the image processing device 10 mounted on a vehicle, and includes determining whether a rectangular area BB detected as a traffic light 300 by image recognition of a camera image is at least a detection area of ​​a three-light traffic light 300 based on the aspect ratio of the rectangular area BB. The image processing method according to the embodiment also includes expanding the rectangular area BB by a predetermined ratio. The image processing method according to the embodiment also includes correcting the position of or expanding the rectangular area BB based on the position of the light-emitting area in the rectangular area BB detected by the image recognition.

[0026] Therefore, according to the image processing method of the embodiment, it is possible to improve the accuracy of detecting the traffic light 300 by image recognition.

[0027] An example of the configuration of the image processing device 10 to which the image processing method according to the above-described embodiment is applied will now be described in more detail.

[0028] Fig. 6 is a block diagram showing an example of the configuration of an image processing device 10 according to an embodiment. Note that Fig. 6 shows only components necessary for explaining the features of this embodiment, and omits descriptions of general components.

[0029] In other words, the components shown in Fig. 6 are conceptual functional components and do not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each block is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0030] In addition, in the description using FIG. 6, the description of components that have already been described may be simplified or omitted.

[0031] 6, the image processing device 10 includes a storage unit 11 and a control unit 12. The image processing device 10 is also connected to a camera 3 and an output unit 5.

[0032] The camera 3 is mounted on the vehicle and is provided so as to be able to capture an image of the area in front of the vehicle. Note that the camera 3 may be, for example, a 360-degree camera that is able to capture an image of the entire periphery of the vehicle.

[0033] The output unit 5 is an output device for outputting data. The output unit 5 is realized by, for example, a display, a speaker, etc. The output unit 5 may also be an external device that executes information processing based on the data output by the image processing device 10.

[0034] The camera 3 and the output unit 5 are connected to the image processing device 10 via an in-vehicle network such as a Controller Area Network (CAN). The camera 3 and the output unit 5 may also be connected to the image processing device 10 wirelessly via Bluetooth (registered trademark), Wi-Fi (registered trademark), UWB (Ultra Wide Band), or the like.

[0035] The image processing device 10 is a computer mounted on a vehicle, and as described above, is, for example, a red light running determination device. The image processing device 10 executes at least the image processing described with reference to FIGS.

[0036] The storage unit 11 is realized by a storage device such as a RAM (Random Access Memory) or a flash memory, and in the example of FIG. 6, stores an image recognition AI 11a and processing parameter information 11b.

[0037] The image recognition AI 11a is an AI (Artificial Intelligence) model for image recognition. Specifically, the image recognition AI 11a is a DNN model or the like trained using a machine learning algorithm. After being read into the image recognition unit 12a (described later) as a DNN model, the image recognition AI 11a is configured to be able to detect various objects contained in an image captured by the camera 3 when the image is input to the image recognition unit 12a. Furthermore, the image recognition AI 11a is configured to be able to detect the light emission state of a traffic light 300 when a traffic light 300 is detected as an object.

[0038] 7 is an explanatory diagram of the image recognition AI 11a according to the embodiment. As shown in FIG. 7, the image recognition AI 11a is trained in advance in a learning device 700 including a GPU (Graphics Processing Unit) and an AI platform (not shown) to be able to detect various traffic lights 300 using images of these traffic lights 300 as learning data, such as a normal three-light traffic light, various arrow-type traffic lights, and warning traffic lights.

[0039] Then, after the image recognition AI 11a is read as a DNN model into the aforementioned image recognition unit 12a in the image processing device 10, when an image captured by the camera 3 is input to the image recognition unit 12a, the image recognition AI 11a detects the traffic light 300 and its light emission state in the captured image.

[0040] Returning to the explanation of Fig. 6, the processing parameter information 11b is information on various parameters used in the processes executed by an exclusion unit 12b, a correction unit 12c, and a determination unit 12d, which will be described later, and includes various thresholds, the ratio for expanding the rectangular area BB, and the like.

[0041] The control unit 12 is a controller, and is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like, executing a program according to the embodiment stored in the storage unit 11 using RAM as a work area. The control unit 12 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0042] The control unit 12 has an image recognition unit 12a, an exclusion unit 12b, a correction unit 12c, and a determination unit 12d, and realizes or executes the functions and actions of information processing described below.

[0043] The image recognition unit 12a acquires a captured image captured by the camera 3 and inputs the captured image to the image recognition AI 11a that has read the image. Furthermore, the image recognition unit 12a acquires an image recognition result output from the image recognition AI 11a as a result of inputting the captured image, and outputs the acquired image recognition result to the exclusion unit 12b. The image recognition result includes a rectangular area BB extracted as the traffic light 300. The camera 3 captures a moving image. In response to this, the image recognition unit 12a extracts a rectangular area BB from each frame of the moving image.

[0044] The exclusion unit 12b executes the exclusion process described with reference to Fig. 3. That is, the exclusion unit 12b determines whether or not the rectangular area BB extracted as the traffic light 300 is the detection area of ​​the traffic light 300 based on the aspect ratio of the rectangular area BB. If the aspect ratio value is not equal to 1.0, the exclusion unit 12b determines that the rectangular area BB is the detection area of ​​the traffic light 300. Furthermore, if the aspect ratio value is approximately equal to 1.0, the exclusion unit 12b determines that the rectangular area BB is not the detection area of ​​the traffic light 300, and excludes the rectangular area BB from the image processing target for the traffic light 300, assuming that the rectangular area BB is an erroneous detection.

[0045] The correction unit 12c executes the correction process described with reference to Fig. 4 and Fig. 5. That is, the correction unit 12c spatially corrects the positional deviation of the rectangular area BB. At this time, the correction unit 12c spatially corrects the positional deviation of the rectangular area BB by expanding the rectangular area BB extracted as the traffic light 300 by a predetermined ratio.

[0046] Furthermore, the correction unit 12c spatially corrects the positional deviation of the rectangular area BB based on the light emission state of the traffic light 300 detected by the image recognition unit 12a. In this case, the correction unit 12c corrects the position of or expands the rectangular area BB based on the position of the light emission area of ​​the traffic light 300 in the rectangular area BB.

[0047] Here, Fig. 8 is a supplementary explanatory diagram (part 1) of the correction process. Also, Fig. 9 is a supplementary explanatory diagram (part 2) of the correction process. The ratio by which the rectangular area BB is expanded may be determined individually for each type of traffic light 300, such as a normal three-light traffic light, an arrow traffic light, or a warning traffic light.

[0048] As a result, for arrow-type traffic lights, the rectangular area BB can be corrected to surround the entire traffic light 300, including the arrow portion, as shown in Figure 8. For warning traffic lights, the rectangular area BB can be corrected to surround the entire traffic light 300, including the portion that reads "warning signal," as shown in Figure 9.

[0049] Returning to the description of Fig. 6, the determination unit 12d determines whether or not the vehicle has run a red light based on the image recognition result of the image recognition unit 12a, the exclusion result of the exclusion unit 12b, and the correction result of the correction unit 12c.

[0050] The determination unit 12d estimates the behavior of the vehicle based on, for example, the trajectory of each object that has been image-recognized. The determination unit 12d also estimates the trajectory of the traffic light 300 relative to the vehicle based on the traffic light 300 that has been image-recognized and from which the rectangular area BB has been removed and corrected.

[0051] The determination unit 12d also acquires the light emission state of the traffic light 300 detected by the image recognition unit 12a. The determination unit 12d also determines whether the vehicle has run a red light based on the estimated vehicle behavior, the trajectory of the traffic light 300, and the light emission state of the traffic light 300.

[0052] For example, when a traffic light 300 indicating whether or not a vehicle has priority in the direction of travel of the vehicle is red, and the vehicle continues to pass through the red light for a predetermined time or longer at a predetermined speed or faster, the determination unit 12d determines that the vehicle has ignored the traffic light. The determination unit 12d outputs the determination result to the output unit 5.

[0053] When the image processing device 10 constitutes part of the function of a drive recorder, the determination unit 12d may notify the drive recorder of the determination result that the vehicle has run a red light as an event, and cause the drive recorder to record the event.

[0054] Next, a processing procedure executed by the image processing device 10 according to the embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing the processing procedure executed by the image processing device 10 according to the embodiment. The processing procedure shown in Fig. 10 is repeated, for example, while the vehicle is running.

[0055] 10, the control unit 12 of the image processing device 10 acquires a camera image from the camera 3 (step S101). Then, the control unit 12 performs image recognition on the acquired camera image (step S102).

[0056] Then, the control unit 12 determines whether or not there is a rectangular area BB detected as the traffic light 300 as a result of the image recognition (step S103). If there is no corresponding rectangular area BB (step S103, No), the process ends.

[0057] If a corresponding rectangular area BB is found (step S103, Yes), the control unit 12 determines whether the aspect ratio of the corresponding rectangular area BB is ≠ 1.0 (step S104). If the aspect ratio is ≠ 1.0 (step S104, Yes), the control unit 12 expands the corresponding rectangular area BB by a predetermined ratio (step S105).

[0058] Then, the control unit 12 corrects the position of or expands the rectangular area BB based on the position of the light-emitting area in the rectangular area BB (step S106).The control unit 12 then determines whether there are any more rectangular areas BB detected as traffic lights 300 (step S107).

[0059] If there is another rectangular area BB (step S107, Yes), the control unit 12 repeats the process from step S104. If there is no more rectangular area BB (step S107, No), the process ends.

[0060] If the aspect ratio value of the rectangular area BB is not 1.0 in step S104 (step S104, No), the control unit 12 excludes the rectangular area BB (step S108) and proceeds to step S107.

[0061] As described above, the image processing device 10 according to the embodiment is an image processing device mounted on a vehicle, and includes the control unit 12. Based on the aspect ratio of the rectangular area BB detected as a traffic light 300 by image recognition of a camera image, the control unit 12 determines whether or not the rectangular area BB is a detection area of ​​at least a three-light traffic light 300.

[0062] Therefore, the image processing device 10 according to the embodiment can improve the accuracy of detecting the traffic light 300 by image recognition.

[0063] Furthermore, the control unit 12 determines that the rectangular area BB is the detection area of ​​the traffic light 300 when the aspect ratio of the rectangular area BB is within a predetermined range.

[0064] Therefore, according to the image processing device 10 of the embodiment, if the aspect ratio value of the rectangular area BB detected as the traffic light 300 is within a predetermined range, it is determined to be the detection area of ​​the traffic light 300, thereby improving the detection accuracy of the traffic light 300 by image recognition.

[0065] Furthermore, the control unit 12 determines that the rectangular area BB is the detection area of ​​the traffic light 300 when the aspect ratio of the rectangular area BB is not an approximation of 1.0.

[0066] Therefore, according to the image processing device 10 of the embodiment, the detection area of ​​at least a normal three-light traffic light 300 can be accurately determined to be the detection area, thereby improving the detection accuracy of the traffic light 300 by image recognition.

[0067] Furthermore, the control unit 12 determines that the rectangular area BB is not within the detection area of ​​the traffic light 300 when the aspect ratio of the rectangular area BB is a value close to 1.0.

[0068] Therefore, the image processing device 10 according to the embodiment can prevent the sign 500 or the like from being erroneously determined to be the traffic light 300.

[0069] Furthermore, the control unit 12 expands the rectangular area BB at a predetermined ratio.

[0070] Therefore, the image processing device 10 according to the embodiment can correct the positional deviation of the rectangular area BB.

[0071] Furthermore, the control unit 12 expands the rectangular area BB when the feature amount of a specific shape extracted from the rectangular area BB does not satisfy a threshold value.

[0072] Therefore, according to the image processing device 10 of the embodiment, when it is estimated that a positional shift has occurred in the rectangular area BB due to a lack of features of the light-emitting part of the traffic light 300, for example, three circles, the rectangular area BB can be expanded.

[0073] Furthermore, the control unit 12 corrects the position of or expands the rectangular area BB based on the position of the light emitting area in the rectangular area BB detected by the image recognition.

[0074] Therefore, according to the image processing device 10 of the embodiment, even if a positional misalignment occurs, the rectangular area BB can be corrected so as to surround the entire traffic light 300 based on the position of the light-emitting area in the rectangular area BB.

[0075] Furthermore, the control unit 12 expands the rectangular area BB in accordance with the type of traffic light 300 detected by the image recognition, at the ratio determined individually for each type of traffic light 300.

[0076] Therefore, according to the image processing device 10 of the embodiment, for example, for an arrow-type traffic light, the rectangular area BB can be corrected so as to surround the entire traffic light 300, including the arrow portion, as shown in Fig. 8. Furthermore, for a warning traffic light, the rectangular area BB can be corrected so as to surround the entire traffic light 300, including the portion that reads "warning signal," as shown in Fig. 9.

[0077] In addition, the image processing method of the embodiment is an image processing method executed by an image processing device 10 mounted on a vehicle, and includes determining whether or not a rectangular area BB detected as a traffic light 300 by image recognition of a camera image is a detection area of ​​at least a three-light traffic light 300 based on the aspect ratio of the rectangular area BB.

[0078] Therefore, according to the image processing method of the embodiment, it is possible to improve the accuracy of detecting the traffic light 300 by image recognition.

[0079] In addition, the program of the embodiment causes the image processing device 10 (corresponding to an example of a "computer") to determine whether or not the rectangular area BB detected as a traffic light 300 by image recognition of the camera image is a detection area of ​​at least a three-light traffic light 300, based on the aspect ratio of the rectangular area BB.

[0080] Therefore, according to the program according to the embodiment, it is possible to improve the accuracy of detecting the traffic light 300 by image recognition.

[0081] In the above-described embodiment, the light emission state of the traffic light 300 and whether or not a traffic light has been ignored are determined based on image recognition, but it goes without saying that sensor data from various sensors mounted on the vehicle may be appropriately combined. For example, the behavior of the vehicle may be estimated using sensor values ​​from a steering sensor or an acceleration sensor, and the vehicle speed may be obtained using sensor values ​​from a speed sensor.

[0082] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0083] 3 Camera 5 Output section 10 Image processing device 11 Storage section 11a Image recognition AI 11b Processing parameter information 12 Control Unit 12a Image recognition unit 12b Exclusion part 12c correction section 12d Judgment section BB detection area

Claims

1. An image processing device mounted on a vehicle, the image processing device having a control unit, The control unit determining whether or not a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light; When the rectangular area is determined to be the detection area of ​​the traffic light and the feature amount of the specific shape extracted from the rectangular area does not satisfy a threshold, the rectangular area is expanded. Image processing device.

2. An image processing device mounted on a vehicle, comprising: a control unit; The control unit determining whether or not a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light; When the rectangular area is determined to be the detection area of ​​the traffic light, the position of the rectangular area is corrected or expanded based on the position of the light-emitting area in the rectangular area detected by the image recognition. Image processing device.

3. The control unit Expanding the rectangular area by a predetermined ratio; 3. The image processing device according to claim 1.

4. An image processing device mounted on a vehicle, comprising: a control unit; The control unit determining whether or not a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light; When it is determined that the rectangular area is the detection area of ​​the traffic light, the rectangular area is expanded in accordance with the type of the traffic light detected by the image recognition at a ratio individually determined for each type of the traffic light. Image processing device.

5. The control unit If the aspect ratio of the rectangular area is within a predetermined range, the rectangular area is determined to be the detection area of ​​the traffic light.

5. The image processing device according to claim 1.

6. The control unit If the aspect ratio of the rectangular area is not an approximate value of 1.0, the rectangular area is determined to be the detection area of ​​the traffic light. The image processing device according to claim 5 .

7. The control unit If the aspect ratio of the rectangular area is an approximate value of 1.0, the rectangular area is determined to be outside the detection area of ​​the traffic light.

7. The image processing device according to claim 5 or 6.

8. An image processing method executed by an image processing device mounted on a vehicle, comprising: determining whether or not a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light; When the rectangular area is determined to be the detection area of ​​the traffic light and a feature amount of a specific shape extracted from the rectangular area does not satisfy a threshold, the rectangular area is expanded. An image processing method comprising:

9. determining whether or not a rectangular area detected as a traffic light by image recognition of a camera image is a detection area of ​​at least a three-light traffic light; When the rectangular area is determined to be the detection area of ​​the traffic light and a feature amount of a specific shape extracted from the rectangular area does not satisfy a threshold, the rectangular area is expanded. A program that causes a computer to execute the following.

Images