Object recognition control device and object recognition control method

Abstract

To properly recognize objects.SOLUTION: A control unit 20 that is an object recognition control apparatus, comprises: a video data acquisition unit 21 for acquiring video data captured by a camera 11 that images a range including the front of a vehicle; a light distribution information acquisition unit 22 for acquiring light distribution information indicating a light distribution state of headlights of the vehicle; a recognition processing unit 23 for recognizing the video data acquired by the video data acquisition unit 21 as a specific object if a score indicating the likelihood of the specific object is equal to or greater than a threshold; and a presentation processing unit 24 for presenting information of the specific object recognized by the recognition processing unit 23 to a driver of the vehicle. The light distribution information acquisition unit 22 acquires variable light distribution information of the headlights of the vehicle. The recognition processing unit 23 changes the threshold in a partial light-out range in the lighting of the vehicle so as to be lower than a threshold in other lighting range.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an object recognition control device and an object recognition control method. [Background Art]

[0002] For human recognition in a captured image, in recognition processing using a human recognition dictionary, it is determined that a target is a human if a score indicating human-likeness is equal to or higher than a predetermined threshold. A technology is known that determines that a target object is a pedestrian based on a high score indicating pedestrian-likeness obtained by using a recognition dictionary for a captured image (see, for example, Patent Document 1). [Prior Art Literature] [Patent Literature]

[0003] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2016-015029 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] A recognition system using a far-infrared camera recognizes persons, animals, and the like located in places that are difficult for humans to visually observe, and notifies a vehicle driver of the recognition result. For this reason, in the range irradiated by the vehicle's headlamps, the driver can visually respond, so accuracy of recognition is required. In contrast, in a range not irradiated by the vehicle's headlamps, if a recognition omission occurs by the recognition system, there is a possibility that it may lead to danger.

[0005] The present invention has been made in view of the above, and an object of the present invention is to appropriately perform object recognition. [Means for Solving the Problem]

[0006] To solve the above-mentioned problems and achieve the objective, the object recognition control device according to the present invention comprises: an image data acquisition unit that acquires image data captured by an image capture unit that captures an area including the front of a vehicle; a light distribution information acquisition unit that acquires light distribution information indicating the light distribution state of the vehicle's headlights; a recognition processing unit that recognizes an image data acquired by the image data acquisition unit as a specific object if the score indicating the likelihood of it being a specific object is above a threshold; and a presentation processing unit that presents information about the specific object recognized by the recognition processing unit to the driver of the vehicle, wherein the recognition processing unit changes the threshold based on the light distribution information of the vehicle's headlights acquired by the light distribution information acquisition unit.

[0007] The object recognition control method performed by the object recognition control device according to the present invention includes the steps of: acquiring video data captured by a camera that captures an area including the front of a vehicle; acquiring light distribution information indicating the light distribution state of the vehicle's headlights; changing a threshold that determines a score indicating the likelihood of an object being a specific object based on the light distribution information of the vehicle's headlights with respect to the acquired video data, and recognizing the object as a specific object if the score is equal to or greater than the threshold; and presenting information of the recognized specific object to the driver of the vehicle. [Effects of the Invention]

[0008] The present invention provides the effect of enabling proper object recognition. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing an example configuration of an object recognition device having an object recognition control device according to the first embodiment. [Figure 2] Figure 2 is a flowchart showing the processing flow in the object recognition control device according to the first embodiment. [Figure 3] Figure 3 is a schematic diagram illustrating the irradiation range of the low beam. [Figure 4]Figure 4 is a schematic diagram illustrating the illumination range of the high beam. [Figure 5] Figure 5 is a flowchart showing the processing flow in the object recognition control device according to the second embodiment. [Figure 6] Figure 6 is a schematic diagram illustrating an example of a variable light distribution irradiation range. [Figure 7] Figure 7 is a schematic diagram illustrating another example of a variable beam pattern illumination range. [Figure 8] Figure 8 is a flowchart showing the processing flow in the object recognition control device according to the third embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the object recognition control device and object recognition control method according to the present invention will be described in detail below with reference to the attached drawings. However, the present invention is not limited to the following embodiments.

[0011] [First Embodiment] <Object Recognition Device> Figure 1 is a block diagram showing an example configuration of an object recognition device 10 having a control unit 20, which is an object recognition control device according to the first embodiment. The object recognition device 10 performs recognition processing on video data using an object recognition dictionary to recognize a specific object. The object recognition device 10 notifies the vehicle driver of information about the recognized specific object. When the vehicle driver's visibility is good, the object recognition device 10 reduces the sensitivity of object recognition compared to when visibility is poor.

[0012] The object recognition device 10 includes a camera (shooting unit) 11, a recognition dictionary storage unit 12, a display unit 13, a CAN (Controller Area Network) interface unit (hereinafter referred to as "IF unit") 14, and a control unit (object recognition control device) 20. The object recognition device 10 may be implemented, for example, as a function of a vehicle-installed safety driving support device, navigation device, drive recorder, etc.

[0013] Camera 11 is a camera that captures an area including the front of the vehicle. Camera 11 is mounted at the front of the vehicle. The video data acquisition unit 21 captures an area that includes at least the illumination range of the vehicle's headlights. Camera 11 is a far-infrared camera, but it may be a visible light camera, or a combination of a far-infrared camera and a visible light camera, as long as it can detect objects in an area not illuminated by the vehicle's headlights. Camera 11 is positioned, for example, to capture the area in front of the vehicle, which is the direction of travel. Camera 11 continuously captures images from the time the engine is started until it is stopped, that is, while the vehicle is in operation. Camera 11 outputs the captured video data to the video data acquisition unit 21 of the control unit 20. The video data is a moving image consisting of, for example, 30 frames per second. In this embodiment, camera 11 is positioned at the front of the vehicle.

[0014] The recognition dictionary storage unit 12 stores dictionary data for recognizing various objects from video data. For example, the recognition dictionary storage unit 12 stores an object recognition dictionary that can verify whether an object included in the video data is a specific object by using machine learning on various videos in which a specific object is captured. The recognition dictionary storage unit 12 is a storage device such as a semiconductor memory element such as ROM (Read Only Memory) or flash memory, or an external storage device via a network. For example, the recognition dictionary storage unit 12 stores a dictionary for recognizing specific objects such as people or animals.

[0015] The display unit 13 is a device that displays various types of information. For example, it may be a display device unique to the object recognition device 10, or a display device shared with other systems including a drive recorder and a navigation system. The display unit 13 is, for example, a display including a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display. The display unit 13 displays video based on a video signal output from the presentation processing unit 24 of the control unit 20. In the present embodiment, the display unit 13 is arranged at a position visible to the driver of the vehicle while driving.

[0016] The IF unit 14 is an interface for acquiring light distribution information of the vehicle's headlamps via CAN.

[0017] The light distribution information is information indicating a light distribution state of the vehicle's headlamps. The light distribution information includes information indicating whether the vehicle's headlamps are in an on state or an off state. The light distribution information includes information indicating whether the vehicle's headlamps are illuminated in a high beam or a low beam. Furthermore, the light distribution information may include information indicating on / off of the vehicle's headlamps.

[0018] <Object recognition control device> The control unit 20 is an arithmetic processing device (control device) configured by, for example, a CPU (Central Processing Unit) or the like. The control unit 20 loads a stored program into a memory and executes instructions included in the program. The control unit 20 includes an internal memory (not shown), and the internal memory is used for temporary storage of data in the control unit 20 and the like. Accordingly, the control unit 20 executes the object recognition method performed by the object recognition device 10. The control unit 20 is also a computer that runs the program according to the present invention. The control unit 20 includes, connected to a bus 20X, a video data acquisition unit 21, a light distribution information acquisition unit 22, a recognition processing unit 23, and a presentation processing unit 24.

[0019] The video data acquisition unit 21 acquires video data captured by the camera 11. More specifically, the video data acquisition unit 21 acquires video data output by the camera 11 that captures the area in front of the vehicle, which is the direction of travel. The video data includes at least the illumination range of the vehicle's headlights. If the camera 11 is a far-infrared camera, the video data acquisition unit 21 acquires video data showing the heat distribution captured by the far-infrared camera.

[0020] The light distribution information acquisition unit 22 acquires light distribution information indicating the light distribution state of the vehicle's headlights. More specifically, the light distribution information acquisition unit 22 acquires light distribution information indicating whether the vehicle's headlights are on high beam or low beam. The light distribution information acquisition unit 22 acquires the light distribution information of the vehicle's headlights via the IF unit 14. If the light distribution of the vehicle's headlights changes in response to right or left turning operations, the light distribution information acquisition unit 22 acquires light distribution information corresponding to the change in the light distribution direction. In addition, the position in the video data that the recognition processing unit 23 performs recognition processing on and the illumination range corresponding to the light distribution state of the headlights are pre-associated.

[0021] The recognition processing unit 23 recognizes a specific object from the video data acquired by the video data acquisition unit 21. If the camera 11 is a far-infrared camera, the recognition processing unit 23 recognizes a person or an animal as the specific object. The recognition processing unit 23 performs object recognition processing to recognize the specific object by performing pattern matching on the video data using the object recognition dictionary stored in the recognition dictionary storage unit 12. Known methods can be used in the object recognition processing. The recognition processing unit 23 may perform processing continuously while the object recognition device 10 is running.

[0022] A specific object includes people, two-wheeled vehicles including bicycles ridden by people, other vehicles, and moving objects such as animals. If camera 11 is a far-infrared camera, the specific object is people and animals.

[0023] The recognition processing unit 23 recognizes, with respect to the video data acquired by the video data acquisition unit 21, the specific object as a specific object if a score indicating the likelihood of being the specific object is equal to or greater than a predetermined threshold. Let the threshold of the score in normal times be A (a prescribed value). The score threshold is changed in accordance with the light distribution information acquired by the light distribution information acquisition unit 22. Let the changed score threshold be B (B < A). Increasing the score threshold makes it harder to recognize the specific object, reducing erroneous recognitions, but causes a possibility that recognition omissions occur. Lowering the score threshold makes it easier to recognize the specific object, reducing the occurrence of recognition omissions, but increases the possibility that erroneous recognitions occur.

[0024] The recognition processing unit 23 recognizes the specific object by changing the score threshold based on the light distribution information acquired by the light distribution information acquisition unit 22. In the present embodiment, the recognition processing unit 23 changes the score threshold based on the light distribution information of the vehicle headlamps acquired by the light distribution information acquisition unit 22. The recognition processing unit 23 recognizes the specific object by changing the score threshold based on the light distribution information acquired by the light distribution information acquisition unit 22.

[0025] The recognition processing unit 23 sets the score threshold when the vehicle headlamps are on low beam to a lower value than the score threshold when the vehicle headlamps are on high beam. In other words, when the light distribution information indicates that the vehicle headlamps are on low beam, the recognition processing unit 23 recognizes an object as the specific object using a threshold B that is lower than the normal-time score threshold A. When the vehicle headlamps are on high beam, the driver has a good field of view and can see far ahead, so the recognition of the specific object by the recognition processing unit 23 prioritizes accuracy. When the light distribution information indicates that the vehicle headlamps are on high beam, the specific object is recognized using the normal score threshold A.

[0026] For example, the maximum score indicating the likelihood of an object being a specific object is 1.0. As an example, let's set threshold A to 0.9 and threshold B to 0.7. For example, the recognition processing unit 23 determines that a detected object is a specific object if the score is 0.7 or higher when the vehicle's headlights are on low beam. For example, the recognition processing unit 23 determines that a detected object is a specific object if the score is 0.9 or higher when the vehicle's headlights are on high beam. For example, when the recognition processing unit 23 is performing object recognition based on the condition that the score is 0.9 or higher, the recognition processing is changed to use a lower score threshold when the vehicle's headlights switch to low beam.

[0027] The score threshold may be set for each specific object, or in other words, for each object recognition dictionary.

[0028] The score threshold can be changed gradually or linearly.

[0029] The presentation processing unit 24 presents information about a specific object recognized by the recognition processing unit 23 to the vehicle driver. By presenting information about the specific object recognized by the presentation processing unit 24, it prompts the vehicle driver to pay attention to the specific object. The presentation processing unit 24 presents the information about the recognized specific object to the driver using a display by the display unit 13 or sound output from an audio output unit (not shown). Therefore, when the presentation processing unit 24 presents information about the specific object to the driver using a display by the display unit 13, it functions as a display control unit. More specifically, the presentation processing unit 24 generates video to present information to the vehicle driver. For example, the presentation processing unit 24 generates presentation video data that overlays a frame around the recognized specific object onto the video data. The presentation processing unit 24 may also generate presentation video data that displays text or an icon to prompt attention to the specific object. The presentation processing unit 24 may also generate sound such as a warning sound to be output along with the video. The display processing unit 24 outputs a video signal to the display unit 13 to display the generated video. The display processing unit 24 may also output an audio signal to a speaker (not shown) to output the sound generated along with the video.

[0030] <Processing in the control unit> Next, the processing flow in the control unit 20 will be explained using Figure 2. Figure 2 is a flowchart showing the processing flow in the control unit 20, which is an object recognition control device according to the first embodiment. When the object recognition device 10 is activated, the processing shown in the flowchart in Figure 2 is disclosed. The processing in Figure 2 can be started under arbitrary conditions. For example, when the vehicle becomes available, such as when the engine of the vehicle equipped with the object recognition device 10 is started, or when the operation of the object recognition device 10 is started by a user operation. The processing in step S101 is executed on the condition that the vehicle's headlights are on. The processing in step S101 may also be executed on the condition that it is dark around the vehicle and the vehicle's headlights are on. As a result, if the headlights are on during the daytime when it is bright around the vehicle, in other words, if daytime running lights are being used, the processing in step S101 will not be executed. The processing in step S101 may also be executed on the condition that the vehicle's headlights have started to turn on due to the auto-light function. Furthermore, the processing in step S101 may also be executed on the condition that the vehicle is moving.

[0031] The darkness around the vehicle may be determined based on sensor data acquired by an illuminance sensor (not shown) placed on the vehicle. The darkness around the vehicle may also be determined based on brightness or lightness after performing image processing on video data. The darkness around the vehicle may also be determined by comparing sunset information acquired from an external device (not shown) with current time information that can be acquired by the object recognition device 10. The methods for determining whether the area around the vehicle is dark are not limited to these, and known methods can be used.

[0032] As processing in Figure 2 begins, the control unit 20 starts the image capture and object recognition process (step S101). More specifically, the control unit 20 starts image capture using the camera 11. The control unit 20 acquires the video data output by the camera 11 using the video data acquisition unit 21. The control unit 20 recognizes specific objects around the vehicle from the video data acquired by the video data acquisition unit 21 using the recognition processing unit 23. The threshold score for the object recognition process in step S101 is the normal score threshold A. For example, the control unit 20 determines that the detected object is a specific object if the score is 0.9 or higher using the recognition processing unit 23. The control unit 20 then proceeds to step S102.

[0033] As the processing in step S101 begins, the control unit 20 determines whether the headlights are on high beam or not (step S102). More specifically, the control unit 20 acquires light distribution information via the IF unit 14 using the light distribution information acquisition unit 22. If the acquired light distribution information indicates that the headlights are on high beam (Yes in step S102), the control unit 20 proceeds to step S103. If the acquired light distribution information indicates that the headlights are not on high beam (No in step S102), the control unit 20 proceeds to step S104.

[0034] If the headlights are not on high beam (No in step S102), the control unit 20 uses the recognition processing unit 23 to perform specific object recognition processing using threshold B, which is lower than the normal score threshold A (step S104). More specifically, the recognition processing unit 23 performs pattern matching on the video data, and if the score indicating the likelihood of it being a specific object is 0.7 or higher, it determines that the detected object is a specific object. The control unit 20 then proceeds to step S105.

[0035] If the headlights are on high beam (Yes in step S102), the control unit 20 uses the recognition processing unit 23 to perform specific object recognition processing using the normal score threshold A (step S103). More specifically, the recognition processing unit 23 performs pattern matching on the video data, and if the score indicating the likelihood of it being a specific object is 0.9 or higher, it determines that the detected object is a specific object. As a result, when the headlights are on low beam, the detected object is more likely to be recognized as a specific object compared to when they are on normal or high beam. The control unit 20 proceeds to step S105.

[0036] The control unit 20 determines whether or not it has recognized a specific object based on the object recognition result of the recognition processing unit 23 (step S105). If the control unit 20 determines that it has recognized a specific object (Yes in step S105), it proceeds to step S106. If the control unit 20 does not determine that it has recognized a specific object (No in step S105), it proceeds to step S107.

[0037] The control unit 20 displays a border around the recognized specific object, as determined by the presentation processing unit 24 (step S106). More specifically, the control unit 20, as determined by the presentation processing unit 24, overlays a border around the specific object surrounding the vehicle onto the video data and displays it on the display unit 13. The control unit 20 then proceeds to step S107.

[0038] The control unit 20 determines whether or not to terminate the shooting and object recognition process (step S107). The control unit 20 determines to terminate the shooting and object recognition process, for example, if it detects an operation to terminate the object recognition process, or if the vehicle stops and the engine is turned off. If the control unit 20 determines to terminate the shooting and object recognition process (Yes in step S107), it terminates the process. If the control unit 20 does not determine to terminate the shooting and object recognition process (No in step S107), it executes the process in step S102 again.

[0039] <Effects> As described above, in this embodiment, the score threshold in the object recognition process is changed based on the vehicle's light distribution information. According to this embodiment, specific objects can be appropriately recognized based on the vehicle's light distribution information.

[0040] In this embodiment, when the vehicle's headlights are on low beam, the score threshold in the object recognition process is lowered compared to when they are on high beam. According to this embodiment, when the vehicle's headlights are on high beam, the headlights illuminate a long distance, making it easier for the driver to visually confirm people and other objects from afar. Therefore, this embodiment can reduce misrecognition of specific objects and present accurate recognition results for specific objects.

[0041] In this embodiment, when the vehicle's headlights are on low beam, the score threshold in the object recognition process is lowered compared to when they are on high beam. According to this embodiment, when the vehicle's headlights are on low beam, it may be more difficult for the driver to visually confirm people and other objects compared to when they are on high beam. In this embodiment, when the headlights are on low beam, which do not illuminate far distances, the occurrence of missed recognition of specific objects is reduced, and the recognition results of specific objects can be presented more appropriately.

[0042] In this embodiment, a far-infrared camera is used as camera 11. According to this embodiment, even when the area around the vehicle is dark, it is possible to appropriately recognize people, animals, and other specific objects.

[0043] This embodiment presents information about a specific object that has been recognized to the driver. This embodiment can appropriately present the recognition result of a specific object to the driver even when the vehicle is illuminated by low beams, which have many areas that are difficult for the driver to see.

[0044] [Second Embodiment] The object recognition device 10 according to this embodiment will be described with reference to Figures 3 to 5. Figure 3 is a schematic diagram illustrating the illumination range of the low beam. Figure 4 is a schematic diagram illustrating the illumination range of the high beam. Figure 5 is a flowchart showing the processing flow in the control unit 20, which is an object recognition control device according to the second embodiment. In this embodiment, the object recognition device 10 reduces the sensitivity of object recognition in the illumination range of the headlights compared to the non-illuminated range. The basic configuration of the object recognition device 10 is the same as that of the object recognition device 10 in the first embodiment. In the following description, components similar to those of the object recognition device 10 are denoted by the same reference numerals or corresponding reference numerals, and their detailed descriptions are omitted.

[0045] The light distribution information acquired from the vehicle via the IF unit 14 includes information indicating the illumination range of the headlights. The light distribution information may also include information indicating the illumination range of the high beams and information indicating the illumination range of the low beams.

[0046] The recognition processing unit 23 recognizes a specific object using a score threshold value lower than the score threshold value in the illuminated area of ​​the vehicle's headlights. In other words, the recognition processing unit 23 recognizes the area not illuminated by the headlights as a specific object using a threshold B that is lower than the normal score threshold A.

[0047] The recognition processing unit 23 may recognize the specific object by changing the score threshold between the non-irradiation range of high beam and the non-irradiation range of low beam. More specifically, the recognition processing unit 23 may recognize the specific object by setting the score threshold in the non-irradiation range of low beam to a value lower than the score threshold in the non-irradiation range of high beam. For example, the recognition processing unit 23 may recognize an object as the specific object in the non-irradiation range of high beam by using a threshold BH that is lower than the normal score threshold A. For example, the recognition processing unit 23 may recognize an object as the specific object in the non-irradiation range of low beam by using a threshold BL (BL<BH) that is lower than the normal score threshold A. For example, BH is set to 0.8 and BL is set to 0.7. Note that the irradiation range of high beam and the irradiation range of low beam may be recognized as the specific object using the normal threshold A.

[0048] As shown in FIG. 3, in the case of low beam, the video data 100 includes a low beam irradiation range 101 and a non-irradiation range 102 in the video data 100. In FIG. 3, a boundary line L between the irradiation range 101 and the non-irradiation range 102 is illustrated as a solid line. The irradiation range 101 is located below the boundary line L relative to the non-irradiation range 102.

[0049] As shown in FIG. 4, in the case of high beam, the video data 100 includes a high beam irradiation range 103 and a non-irradiation range 104 in the video data 100. In FIG. 4, a boundary line H between the irradiation range 103 and the non-irradiation range 104 is illustrated as a solid line. The irradiation range 103 is located below the boundary line H relative to the non-irradiation range 104. The high beam irradiation range 103 is wider forward than the low beam irradiation range 101. Compared with the low beam irradiation range 101, the high beam irradiation range 103 includes a range further away from the vehicle in the center in the left-right direction of the diagram of the video data 100.

[0050] In the video data 100 shown in Figure 3, the irradiated area 101 is recognized as a specific object using threshold A, and the unirradiated area 102 is recognized as a specific object using threshold BL. In the video data 100 shown in Figure 4, the irradiated area 103 is recognized as a specific object using threshold A, and the unirradiated area 104 is recognized as a specific object using threshold BH.

[0051] If the system has a function that changes the beam pattern of the high beams to avoid illuminating preceding or oncoming vehicles when the high beams are on, a score threshold will be set according to the changed beam pattern.

[0052] The recognition processing unit 23 may perform recognition processing only in the non-illuminated area of ​​the vehicle's headlights. In other words, the recognition processing unit 23 may not perform recognition processing in the illumination area of ​​the vehicle's headlights. In this case, recognition processing is performed in the non-illuminated area 102 shown in Figure 3 and the non-illuminated area 104 shown in Figure 4. Recognition processing is not performed in the illuminated area 101 shown in Figure 3 and the illuminated area 103 shown in Figure 4. In this case, the processing load can be reduced, or processing resources can be further utilized to improve the accuracy of the recognition processing.

[0053] <Processing in the control unit> Next, the processing flow in the control unit 20 will be explained using Figure 5. The processing in steps S111, S112, and S115 to S117 is the same as the processing in steps S101, S102, and S105 to S107 of the flowchart shown in Figure 3.

[0054] If the headlights are not on high beam (No in step S112), the control unit 20 has the recognition processing unit 23 perform specific object recognition processing using different threshold scores inside and outside the low beam illumination range (step S113). More specifically, the recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is BL or higher in the low beam non-illuminated range. The recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is A or higher in the low beam illumination range. This reduces the occurrence of missed recognition of specific objects in the low beam non-illuminated range compared to the illuminated range. The control unit 20 proceeds to step S115.

[0055] If the headlights are on high beam (Yes in step S112), the control unit 20 has the recognition processing unit 23 perform specific object recognition processing using different threshold scores inside and outside the high beam illumination range (step S114). More specifically, the recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is BH or higher in the high beam non-illuminated range. The recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is A or higher in the high beam illumination range. As a result, specific objects are detected more accurately in the high beam non-illuminated range than in the low beam non-illuminated range. The control unit 20 proceeds to step S115.

[0056] As described above, in this embodiment, the score threshold in the area not illuminated by the vehicle's headlights is set to a lower value than the score threshold in the illuminated area. The area illuminated by the vehicle's headlights is where the driver can easily visually confirm people and other objects. Therefore, in this embodiment, even in the area not illuminated by the vehicle's headlights, the score threshold in the area not illuminated is changed according to the light distribution of the headlights, making it possible to present the recognition result of a specific object more appropriately.

[0057] Because the area not illuminated by the high beams in the direction of vehicle travel is located far from the vehicle, the size of the detected specific object in the image is small. Therefore, in this embodiment, misrecognition in the area not illuminated by the high beams can be reduced, and more accurate information about the specific object can be presented to the vehicle driver. Furthermore, even if a specific object is present in the area not illuminated by the high beams in the direction of vehicle travel, there is sufficient time to avoid danger, so it is useful to reduce misrecognition in the area not illuminated and present more accurate information about the specific object to the user.

[0058] Furthermore, since the area not illuminated by the low beam in the direction of vehicle travel is located closer to the vehicle than the area not illuminated by the high beam, the size of the detected specific object in the image is larger than that of a specific object at a distance. For this reason, in this embodiment, the occurrence of recognition failures in the area not illuminated by the low beam can be reduced, and information that makes it easier to notice the presence of a specific object near the vehicle can be presented to the vehicle driver. In addition, if a specific object is present in the area not illuminated by the low beam in the direction of vehicle travel, there is less time to avoid danger, so it is useful to reduce the occurrence of recognition failures in the area not illuminated and to present more appropriate information about the specific object to the user.

[0059] [Third Embodiment] Referring to Figures 6 to 8, the object recognition device 10 according to this embodiment Let me explain. Figure 6 is a schematic diagram illustrating an example of the illumination range of a variable light distribution. Figure 7 is a schematic diagram illustrating another example of the illumination range of a variable light distribution. Figure 8 is a flowchart showing the processing flow in the object recognition control device according to the third embodiment. The object recognition device 10 increases the sensitivity of object recognition in the partially off range of the headlight compared to other illumination ranges. The basic configuration of the object recognition device 10 is the same as that of the object recognition device 10 of the first embodiment.

[0060] In this embodiment, the vehicle's headlights perform variable light distribution control. More specifically, variable light distribution control is a function that detects oncoming or preceding vehicles using a camera or radar, for example, and partially switches off the high beams or switches to low beams, thereby illuminating the high beam range while setting the partially off range as needed. When the headlights are performing variable light distribution control, the partially off range is not illuminated by the headlights. Therefore, if pedestrians or other objects are present in the partially off range, they may not be properly detected. Also, if pedestrians or other objects are present between the vehicle and an oncoming vehicle, visual confirmation may be difficult due to glare.

[0061] The light distribution information acquisition unit 22 acquires variable light distribution information of the vehicle's headlights. More specifically, the light distribution information acquisition unit 22 acquires variable light distribution information that indicates the light distribution state when the vehicle's headlights are under variable light distribution control.

[0062] The variable light distribution information includes information on the partial light-off range of the headlights. Furthermore, the position in the video data that the recognition processing unit 23 performs recognition processing on and the illumination range and partial light-off range when the headlights perform variable light distribution control are pre-associated.

[0063] The recognition processing unit 23 sets the threshold value for the partially off area of ​​the vehicle's headlights to a lower value than the threshold value for other illuminated areas. In other words, the recognition processing unit 23 recognizes the partially off area of ​​the vehicle's headlights as a specific object using a threshold B that is lower than the threshold A for the normal score.

[0064] The recognition processing unit 23 may set a threshold value for the partial light extinction range of the vehicle's headlamp to a value lower than the threshold value for other non-irradiation ranges.

[0065] FIG. 6 is a diagram showing an example of the irradiation range of variable light distribution when an oncoming vehicle is detected. As shown in FIG. 6, in the case of variable light distribution when an oncoming vehicle is detected, a partial light extinction range 107 is generated so that the oncoming vehicle is not irradiated. The image data 100 includes an irradiation range 105 of variable light distribution, a non-irradiation range 106, and a partial light extinction range 107 in the image data 100. In FIG. 6, a boundary line KL between the irradiation range 105, the non-irradiation range 106, and the partial light extinction range 107 is illustrated by a solid line. The partial light extinction range 107 is located on the upper right side of the irradiation range 105.

[0066] FIG. 7 is a diagram showing an example of the irradiation range of variable light distribution when a preceding vehicle is detected. As shown in FIG. 7, in the case of variable light distribution when a preceding vehicle is detected, a partial light extinction range 110 is generated so that the preceding vehicle is not irradiated. The image data 100 includes an irradiation range 108 of variable light distribution, a non-irradiation range 109, and a partial light extinction range 110 in the image data 100. In FIG. 7, a boundary line KH between the irradiation range 108, the non-irradiation range 109, and the partial light extinction range 110 is illustrated by a solid line. The partial light extinction range 110 is located on the upper center side of the irradiation range 108.

[0067] In the image data 100 shown in FIG. 6, the irradiation range 105 uses a threshold A, and the partial light extinction range 107 uses a threshold BL to be recognized as a specific object. The non-irradiation range 106 may also be recognized as a specific object using the threshold BL. In the image data 100 shown in FIG. 7, the irradiation range 108 uses the threshold A, and the partial light extinction range 110 uses a threshold BH (BL<BH) to be recognized as a specific object. The non-irradiation range 109 may also be recognized as a specific object using the threshold BH.

[0068] <Processing in Control Unit> Next, the processing flow in the control unit 20 will be explained using Figure 8. The processing in steps S121, S123, and S125 to S127 in Figure 8 is the same as the processing in steps S101, S103, and S105 to S107 in the flowchart shown in Figure 3.

[0069] The control unit 20 determines whether the headlights are performing variable light distribution control (step S122). More specifically, if the light distribution information acquired by the light distribution information acquisition unit 22 indicates that the headlights are not performing variable light distribution control (No in step S122), the control unit 20 proceeds to step S123. If the acquired light distribution information indicates that the headlights are performing variable light distribution control (Yes in step S122), the control unit 20 proceeds to step S124.

[0070] If the headlights are performing variable light distribution control (Yes in step S122), the control unit 20, using the recognition processing unit 23, performs specific object recognition processing using a threshold B lower than the normal score threshold A for the partially off area, and using the score threshold A for the other illuminated areas (step S124). More specifically, in the partially off area, the recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is 0.7 or higher. In the illuminated area other than the partially off area, the recognition processing unit 23 performs pattern matching on the video data and determines that the detected object is a specific object if the score indicating the likelihood of it being a specific object is 0.9 or higher. The control unit 20 proceeds to step S125.

[0071] As described above, this embodiment reduces the occurrence of recognition omissions in the partially darkened area of ​​variable light distribution control and makes it possible to present information to the vehicle driver that makes it easier to notice the presence of a specific object near the vehicle. Furthermore, if a specific object is present in the partially darkened area, there is little time to avoid danger. According to this embodiment, the occurrence of recognition omissions in the partially darkened area can be reduced, and more appropriate information about the specific object can be presented to the user.

[0072] In this embodiment, the threshold value in the partially darkened area can be set lower than the threshold value in other unlit areas. According to this embodiment, for example, if a pedestrian or the like is present in the partially darkened area, the occurrence of detection failures can be reduced. According to this embodiment, for example, even if a pedestrian or the like is present between the vehicle and an oncoming vehicle and visual confirmation is difficult due to glare, the occurrence of detection failures can be reduced.

[0073] In this embodiment, the partial light-off range is such that the partial light-off range 107 shown in Figure 6 is more prone to glare from the headlights of oncoming vehicles than the partial light-off range 110 shown in Figure 7. For this reason, the threshold value for the partial light-off range 107 shown in Figure 6 may be lower than the threshold value for the partial light-off range 110 shown in Figure 7 to recognize a specific object. The same applies when the partial light-off range 107 shown in Figure 6 and the partial light-off range 110 shown in Figure 7 occur simultaneously.

[0074] <Variation> In the above description, the light distribution information was obtained from the vehicle via the IF unit 14, but this is not limited to this. The light distribution information may also be detected based on brightness or lightness by performing image processing on the video data. If the size and shape of the bright area is as shown in Figure 4, it may be detected as the high beams being on, and if the size and shape of the bright area is as shown in Figure 3, it may be detected as the low beams being on.

[0075] In the above description, the illumination range of the headlights was assumed to be included in the light distribution information acquired from the vehicle via the IF unit 14, but this is not limited to this. The illumination range of the headlights may also be detected based on brightness or lightness by performing image processing on the video data. More specifically, for example, image processing is performed on the video data to recognize bright areas and dark areas. Bright areas are detected as the illumination range, and dark areas are detected as the non-illuminated area.

[0076] In the above description, the recognition processing unit 23 was described as changing the threshold value based on the light distribution information of the vehicle's headlights acquired by the light distribution information acquisition unit 22. The recognition processing unit 23 may further change the threshold value based on vehicle information, including wiper operation information, raindrop detection information, fog lamp information, and system information of the on-board camera. More specifically, if the vehicle information indicates adverse weather conditions such as rain, snow, or fog, the score threshold may be lower than under normal conditions. Vehicle information is acquired from the vehicle via the IF unit 14. This makes it easier for the driver to recognize specific objects that are difficult to recognize visually when the driver's visibility is poor due to adverse weather conditions.

[0077] Furthermore, the recognition processing unit 23 may gradually change the score threshold depending on the amount of rain, snowfall, poor visibility due to fog, and the brightness around the vehicle. [Explanation of Symbols]

[0078] 10 Object Recognition Device 11. Camera (Photography Department) 12 Recognition Dictionary Memory Unit 13 Display section 20 Control Unit (Object Recognition Control Unit) 21 Video data acquisition unit 22 Light distribution information acquisition unit 23 Recognition Processing Unit 24 Presentation Processing Unit 100 video data 101 Irradiation range 102 Non-irradiated area 103 Irradiation range 104 Non-irradiated area

Claims

1. A video data acquisition unit acquires video data captured by a shooting unit that captures an area including the front of the vehicle, A light distribution information acquisition unit acquires light distribution information indicating the light distribution state of the headlights of the aforementioned vehicle, The video data acquisition unit performs recognition processing on the video data acquired, and if the score indicating the likelihood of a specific object is above a threshold, the recognition processing unit recognizes it as the specific object. A presentation processing unit presents information about the specific object recognized by the recognition processing unit to the driver of the vehicle, Equipped with, The light distribution information acquisition unit acquires variable light distribution information of the vehicle's headlights, The recognition processing unit sets the threshold value for the partial light-off range of the vehicle's headlights based on an oncoming vehicle to a value lower than the threshold value for the partial light-off range based on a preceding vehicle. Object recognition control device.

2. An object recognition control method executed by an object recognition control device, A video data acquisition step involves acquiring video data captured by a camera unit that captures an area including the front of the vehicle, and A light distribution information acquisition step, which acquires light distribution information indicating the light distribution state of the headlights of the aforementioned vehicle, A recognition processing step in which recognition processing is performed on the acquired video data, and if the score indicating the likelihood of it being a specific object is above a threshold, it is recognized as the specific object, A presentation processing step that presents information about the recognized specific object to the driver of the vehicle, Includes, The light distribution information acquisition step acquires variable light distribution information of the vehicle's headlights, The recognition processing step involves setting the threshold value in the partial light-off range based on an oncoming vehicle to a value lower than the threshold value in the partial light-off range based on a preceding vehicle, with respect to the illumination range of the vehicle's headlights. Object recognition control method.

Images