Information processing device, information processing method, imaging device, lighting device and moving object

Abstract

The present technology relates to an information processing device, an information processing method, an imaging device, a lighting device, and a mobile body that can improve the accuracy of image recognition processing. The information processing device is provided with: a recognition unit for performing image recognition processing on an image captured by an imaging unit and calculating the reliability of recognition results of each recognition area, which is an area based on the recognition result of the object; and an illumination control unit for controlling the illumination pattern of visible light in the imaging direction of the imaging unit so as to increase the reliability for at least some of the recognition areas. The present technology can be applied to, for example, a vehicle control device.

Description

Technical Field

[0001] The present technology relates to an information processing device, an information processing method, a photographing device, a lighting device and a mobile object, and in particular to an information processing device, an information processing method, a photographing device, a lighting device and a mobile object that are suitable for performing image recognition processing on visible images. Background Art

[0002] Conventionally, it has been proposed to control the light distribution pattern of a headlight device based on a detection result of a road user ahead of a vehicle in order to prevent the road user from being dazzled (for example, refer to Patent Document 1).

[0003] Citation List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Translation of PCT Application No. 2013-511418 Summary of the invention

[0006] Technical issues

[0007] In a visible image obtained by photographing the surroundings of a vehicle by a visible light camera in a dark environment such as at night, noise or blur increases in an area where the illumination of the illumination light from the vehicle is low, or saturation of pixels occurs in an area where the illumination of the illumination light is high. As a result, the accuracy of image recognition processing for the visible image decreases, and, for example, the detection accuracy of objects around the vehicle decreases.

[0008] In this regard, Patent Document 1 does not consider controlling the light distribution pattern of the headlight device in order to improve the accuracy of the image recognition process.

[0009] The present technology has been devised in consideration of the above circumstances, and an object of the present invention is to improve the accuracy of image recognition processing.

[0010] Solution to the problem

[0011] According to the first aspect of the present technology, an information processing device includes: a recognition unit, which is configured to perform image recognition processing on an image captured by a shooting unit, and calculate the reliability of the recognition result for each recognition area as an area of ​​the object-based recognition result; and an illumination control unit, which is configured to control the illumination pattern of visible light in the shooting direction of the shooting unit so that the reliability for at least a part of the recognition area is increased.

[0012] According to the second aspect of the present technology, the information processing method includes: performing image recognition processing on an image captured by a capturing unit, and calculating the reliability of the recognition result for each recognition area as an area of ​​the object-based recognition result; and controlling the irradiation pattern of visible light in the capturing direction of the capturing unit so that the reliability for at least a part of the recognition area is increased.

[0013] According to the third aspect of the present technology, a shooting device includes: an imaging element; a recognition unit, which is configured to perform image recognition processing on an image captured by the imaging element, and calculate the reliability of the recognition result for each recognition area as an area of ​​the object-based recognition result; and an illumination control unit, which is configured to control the illumination pattern of visible light in the shooting direction of the imaging element so that the reliability for at least a part of the recognition area is increased.

[0014] According to the fourth aspect of the present technology, the lighting device includes: a light source unit; a recognition unit, which is configured to perform image recognition processing on an image captured by a shooting unit, and calculate the reliability of the recognition result for each recognition area as an area based on the object recognition result; and an illumination control unit, which is configured to control an illumination pattern of visible light based on the light source unit in the shooting direction of the shooting unit so that the reliability for at least a part of the recognition area is increased.

[0015] According to the fifth aspect of the present technology, a mobile body includes: a shooting unit; a lighting unit, which is configured to irradiate visible light in a shooting direction of the shooting unit; a recognition unit, which is configured to perform image recognition processing on an image captured by the shooting unit, and calculate the reliability of the recognition result for each recognition area as an area of ​​the object-based recognition result; and an illumination control unit, which is configured to control the illumination pattern of the visible light of the illumination unit so that the reliability for at least a part of the recognition area is increased.

[0016] In the first to fifth aspects of the present technology, image recognition processing is performed, the reliability of the recognition result is calculated for each recognition area as an area of ​​the object-based recognition result, and the irradiation pattern of visible light in the shooting direction is controlled so that the reliability for at least a part of the recognition area is increased.

[0017] Beneficial effects of the present invention

[0018] According to the first to fifth aspects of the present technology, the accuracy of image recognition processing can be improved.

[0019] It should be noted that the above-mentioned beneficial effects are not necessarily limitative, and any beneficial effects described in the present disclosure may be applied. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a block diagram showing a configuration example of a vehicle to which the present technology is applied.

[0021] Figure 2 is a block diagram showing a configuration example of a lighting section.

[0022] Figure 3 is a block diagram showing a configuration example of a photographing section.

[0023] Figure 4 is a block diagram showing a configuration example of an information processing section.

[0024] Figure 5 This is a flowchart for explaining the lighting control process.

[0025] Figure 6 It is a diagram for explaining the control method of the irradiation pattern.

[0026] Figure 7 It is a diagram for explaining the control method of the irradiation pattern.

[0027] Figure 8 It is a diagram for explaining the control method of the irradiation pattern. DETAILED DESCRIPTION

[0028] Hereinafter, a mode for implementing the present technology will be described. The description will be given in the following order.

[0029] 1. Example

[0030] 2. Variant

[0031] 3. Others

[0032] <<1. Example>>

[0033] <Configuration Example of Vehicle 101>

[0034] Figure 1 : is a block diagram showing a schematic configuration example of a vehicle 101 to which the present technology is applied.

[0035] Vehicle 101 includes a vehicle control system 111 .

[0036] The vehicle control system 111 includes a plurality of control units connected via a communication network 121. Figure 1In the example shown in , the vehicle control system 111 includes a drive system control unit 122, a body system control unit 124, a battery control unit 126, an external vehicle information detection unit 127, an internal vehicle information detection unit 130, and an integrated control unit 133. The communication network 121 connecting the plurality of control units may be an in-vehicle communication network compatible with any standard such as CAN (Controller Area Network), LIN (Local Interconnect Network), LAN (Local Area Network), or FlexRay (registered trademark).

[0037] Each control unit includes: a microcomputer that performs arithmetic processing according to various programs; a storage unit that stores programs to be executed by the microcomputer, parameters to be used in various calculations, etc.; and a drive circuit that drives various devices that are control targets. Each control unit includes a network I / F for communicating with other control units via the communication network 121 and a communication I / F for communicating with devices, sensors, etc. inside and outside the vehicle via wired communication or wireless communication. Figure 1 As functional components of the integrated control unit 133, a microcomputer 151, a general communication I / F 152, a dedicated communication I / F 153, a positioning section 154, a beacon receiving section 155, an in-vehicle device I / F 156, an audio / video output section 157, an in-vehicle network I / F 158, and a storage section 159 are shown. Other control units similarly include a microcomputer, a communication I / F, a storage section, and the like.

[0038] The drive system control unit 122 controls the operation of devices related to the drive system of the vehicle 101 according to various programs. For example, the drive system control unit 122 functions as a control device for a drive force generating device (such as an internal combustion engine or a drive motor) for generating a drive force of the vehicle 101, a control device for a drive force transmitting mechanism for transmitting the drive force to the wheels, a control device for a steering mechanism for adjusting a steering angle of the vehicle 101, a control device for a braking device that generates a braking force of the vehicle 101, etc. The drive system control unit 122 may have a function as a control device for an ABS (anti-lock braking system), a control device for an ESC (electronic stability control), etc.

[0039] The vehicle state detection unit 123 is connected to the drive system control unit 122. For example, the vehicle state detection unit 123 includes at least one of a gyro sensor that detects the angular velocity of the rotational motion of the axis of the vehicle body, an acceleration sensor that detects the acceleration of the vehicle 101, and a sensor for detecting the operation amount of the accelerator pedal, the operation amount of the brake pedal, the steering angle of the steering wheel, the number of revolutions of the engine, the rotation speed of the wheels, etc. The drive system control unit 122 performs arithmetic processing using a signal input from the vehicle state detection unit 123, and controls the internal combustion engine, the drive motor, the electric steering device, the brake device, etc.

[0040] The body system control unit 124 controls the operation of various devices installed to the vehicle body according to various programs. For example, the body system control unit 124 is used as a control device for a keyless entry system, a smart key system, a power window device, or various lights (such as headlights, taillights, brake lights, turn indicators, and fog lights). In this case, radio waves or signals of various switches sent from a portable device instead of a key may be input to the body system control unit 124. The body system control unit 124 accepts the input of the radio waves or signals, and controls the door lock device, power window device, lights, etc. of the vehicle 101.

[0041] It should be noted that Figure 1 Only the lighting portion 125 is shown as a control target of the vehicle body system control unit 124. The lighting portion 125 includes at least a part of the various lamps described above.

[0042] The battery control unit 126 controls a secondary battery (not shown) as a power source for driving the motor according to various programs. For example, information about battery temperature, battery output voltage, battery remaining capacity, etc. is input from a battery device including the secondary battery to the battery control unit 126. The battery control unit 126 uses these signals to perform arithmetic processing to control temperature regulation of the secondary battery or control a cooling device included in the battery device, etc.

[0043] The external vehicle information detection unit 127 detects information about the outside of the vehicle 101. For example, at least one of the photographing unit 128 and the external vehicle information detection unit 129 is connected to the external vehicle information detection unit 127. The photographing unit 128 includes at least one of a ToF (time of flight) camera, a stereo camera, a monocular camera, an infrared camera, and other cameras. For example, the external vehicle information detection unit 129 includes at least one of an environmental sensor for detecting current weather or meteorological phenomena and an environmental information detection sensor for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle 101.

[0044] For example, the environmental sensor may be at least one of a raindrop sensor for detecting rainy weather, a fog sensor for detecting fog, a sunshine sensor for detecting sunshine degree, and a snow sensor for detecting snowfall. The environmental information detection sensor may be at least one of an ultrasonic sensor, a radar device, and a LIDAR (light detection and ranging, laser imaging detection and ranging) device. The photographing unit 128 and the external vehicle information detection unit 129 may be included as independent sensors or independent devices, respectively, or may be included as a device integrating multiple sensors or devices.

[0045] The external vehicle information detection unit 127 enables the shooting unit 128 to shoot an image of the outside of the vehicle and receive the shot image data. In addition, the external vehicle information detection unit 127 receives detection information from the external vehicle information detection unit 129 connected thereto. When the external vehicle information detection unit 129 is an ultrasonic sensor, a radar device, or a LIDAR device, the external vehicle information detection unit 127 enables the external vehicle information detection unit 129 to send ultrasonic waves, electromagnetic waves, etc. and receive information about the received reflected waves (hereinafter referred to as reflected wave information). Based on the reflected wave information, the external vehicle information detection unit 127 can perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, characters on the road surface, etc. Based on the reflected wave information, the external vehicle information detection unit 127 can perform environment recognition processing for identifying rainfall, fog, road conditions, etc. Based on the reflected wave information, the external vehicle information detection unit 127 can calculate the distance to the object outside the vehicle.

[0046] In addition, based on the received image data, the external vehicle information detection unit 127 can perform image recognition processing or distance detection processing for identifying people, vehicles, obstacles, signs, characters on the road surface, etc. The external vehicle information detection unit 127 can perform processing such as distortion correction or positioning on the received image data, and synthesize the image data captured by the different shooting units 128 to generate a bird's-eye view image or a panoramic image. The external vehicle information detection unit 127 can use the image data captured by the different shooting units 128 to perform viewpoint conversion processing.

[0047] The internal vehicle information detection unit 130 detects information about the interior of the vehicle. For example, a driver state detection unit 131 that detects the state of the driver is connected to the internal vehicle information detection unit 130. The driver state detection unit 131 may include a camera that photographs the driver, a biometric sensor that detects the driver's biometric information, a microphone that collects sounds inside the vehicle, and the like. For example, a biometric sensor is provided on a seat surface, a steering wheel, and the like, and the biometric information of a passenger sitting on the seat or a driver holding the steering wheel is detected. Based on the detection information input from the driver state detection unit 131, the internal vehicle information detection unit 130 can calculate the driver's fatigue level or concentration level, or can determine whether the driver has fallen asleep. The internal vehicle information detection unit 130 can perform processing such as noise reduction processing on the collected sound signal.

[0048] The integrated control unit 133 controls the overall operation in the vehicle control system 111 according to various programs. The input unit 132 is connected to the integrated control unit 133. The input unit 132 is implemented by a device (such as a touch panel, a button, a microphone, a switch or a joystick) on which a passenger can perform an input operation. Data obtained by subjecting the sound input from the microphone to voice recognition can be input to the integrated control unit 133. For example, the input unit 132 may be a remote control device using infrared light or other radio waves or an external connection device (such as a mobile phone or a PDA (personal digital assistant)) adapted to the operation of the vehicle control system 111. For example, the input unit 132 may be a camera, in which case the passenger can input information by gesturing to the camera. Alternatively, data obtained by detecting the movement of a wearable device worn by a passenger may be input. In addition, for example, the above-mentioned input unit 132 may include an input control circuit, etc., which generates an input signal based on information input by a passenger, etc. using the input unit 132, and outputs the generated input signal to the integrated control unit 133. By operating the input section 132 , a passenger or the like inputs various types of data and issues instructions to perform processing operations for the vehicle control system 111 .

[0049] The storage section 159 may include a ROM (Read Only Memory) storing various programs to be executed by the microcomputer 151, and a RAM (Random Access Memory) storing various parameters, calculation results, sensor values, etc. In addition, the storage section 159 may be implemented by a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, a magneto-optical storage device, etc., and may provide various programs to be executed by the microcomputer 151.

[0050] The general communication I / F 152 is a general communication I / F that mediates communication with various devices present in the external environment 102. The general communication I / F 152 can implement a cellular communication protocol (such as GSM (registered trademark) (Global System for Mobile Communications), WiMAX, LTE (Long Term Evolution), or LTE-A (LTE-Advanced)) or other wireless communication protocols (such as wireless LAN (also known as Wi-Fi (registered) trademark)) or Bluetooth (registered trademark)). For example, the general communication I / F 152 can be connected to a device (such as an application server or a control server) present on an external network (such as the Internet, a cloud network, or a company-specific network) via a base station or an access point. In addition, for example, the general communication I / F 152 can be connected to a terminal (for example, a terminal belonging to a driver or a pedestrian, a terminal of a store, or an MTC (Machine Type Communication) terminal) present near the vehicle 101 using P2P (peer-to-peer) technology.

[0051] For example, various programs to be executed by the microcomputer 151 may be provided from the external environment 102 .

[0052] The dedicated communication I / F 153 is a communication I / F that supports a communication protocol designed for use in the vehicle 101. For example, the dedicated communication I / F 153 may implement a standard protocol such as WAVE (Wireless Access in Vehicular Environment) which is a combination of IEEE 802.11p constituting a lower layer and IEEE 1609 constituting a higher layer, DSRC (Dedicated Short Range Communication), or a cellular communication protocol. Typically, the dedicated communication I / F 153 performs V2X communication, which is a concept including one or more of communication between vehicles (vehicle-to-vehicle communication), communication between roads and vehicles (vehicle-to-infrastructure communication), communication between the vehicle 101 and a home (vehicle-to-home communication), and communication between pedestrians and vehicles (vehicle-to-pedestrian communication).

[0053] For example, the positioning section 154 receives a GNSS signal from a GNSS (Global Navigation Satellite System) satellite (e.g., a GPS (Global Positioning System) signal from a GPS satellite) and performs positioning, and generates position information including the latitude, longitude, and altitude of the vehicle 101. Alternatively, the positioning section 154 may specify the current position by exchanging signals with a wireless access point, or acquire position information from a terminal such as a mobile phone, PHS, or smartphone having a positioning function.

[0054] For example, the beacon receiving unit 155 receives radio waves or electromagnetic waves emitted from a radio station installed on the road, etc., and acquires information such as the current position, congestion, closure, and required time. Alternatively, the function of the beacon receiving unit 155 may be included in the above-mentioned dedicated communication I / F 153.

[0055] The vehicle-mounted device I / F 156 is a communication interface that mediates the connection between the microcomputer 151 and various vehicle-mounted devices 134 present inside the vehicle. The vehicle-mounted device I / F 156 can establish a wireless connection using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), NFC (near field communication), or WUSB (wireless USB). In addition, the vehicle-mounted device I / F 156 can establish a wired connection such as USB (Universal Serial Bus), HDMI (registered trademark) (High Definition Multimedia Interface), or MHL (Mobile High Definition Link) via a connection terminal (not shown) (and a cable when necessary). For example, the vehicle-mounted device 134 may include at least one of a mobile device or wearable device held or worn by a passenger, and an information device carried on or attached to the vehicle 101. In addition, the vehicle-mounted device 134 may include a navigation device that searches for a route to an arbitrary destination. The vehicle-mounted device I / F 156 exchanges control signals and data signals with the vehicle-mounted device 134.

[0056] The in-vehicle network I / F 158 is an interface that mediates communication between the microcomputer 151 and the communication network 121. The in-vehicle network I / F 158 transmits and receives signals and the like according to a prescribed protocol supported by the communication network 121.

[0057] The microcomputer 151 of the integrated control unit 133 controls the vehicle control system 111 according to various programs based on information acquired via at least one of the general communication I / F 152, the dedicated communication I / F 153, the positioning section 154, the beacon receiving section 155, the vehicle-mounted device I / F 156, and the vehicle-mounted network I / F 158. For example, based on the acquired information about the exterior and interior of the vehicle, the microcomputer 151 can calculate a control target value of a driving force generating device, a steering mechanism, or a braking device, and output a control command to the drive system control unit 122. For example, the microcomputer 151 can perform cooperative control for the purpose of realizing an ADAS (Advanced Driver Assistance System) function, which includes collision avoidance or collision mitigation of the vehicle 101, vehicle-to-vehicle distance control based on the distance between vehicles, cruise control, collision warning of the vehicle 101, and lane departure warning of the vehicle 101. In addition, by controlling the driving force generating device, steering mechanism, braking device, etc. based on the information obtained about the surroundings of the vehicle 101, the microcomputer 151 can perform cooperative control for the purpose of automatic driving, etc., which enables the vehicle to drive autonomously without relying on the driver's operation.

[0058] The microcomputer 151 may generate three-dimensional distance information between the vehicle 101 and surrounding objects (e.g., structures or people), and create local map information including peripheral information of the current position of the vehicle 101 based on information acquired via at least one of the general communication I / F 152, the dedicated communication I / F 153, the positioning unit 154, the beacon receiving unit 155, the vehicle-mounted device I / F 156, and the vehicle-mounted network I / F 158. In addition, based on the acquired information, the microcomputer 151 may predict danger (such as a collision involving the vehicle 101, the approach of pedestrians, etc., or entry into a closed road), and generate a warning signal. For example, the warning signal may be a signal for generating a warning sound or turning on a warning light.

[0059] The audio / video output section 157 transmits an output signal of at least one of sound and image to an output device capable of auditorily or visually notifying information to passengers of the vehicle 101 or the outside of the vehicle 101. Figure 1 In the example shown in , an audio speaker 135, a display unit 136, and an instrument panel 137 are illustrated as output devices. For example, the display unit 136 may include at least one of an on-board display and a head-up display. The display unit 136 may have an AR (augmented reality) display function. The output device may be a device other than those described above, such as headphones, wearable devices (such as glasses-type displays worn by passengers), projectors, or lights. When the output device is a display device, the display device displays the results obtained by performing various types of processing by the microcomputer 151 and information received from other control units in various formats such as text, images, tables, and graphics. In addition, when the output device is an audio output device, the audio output device converts an audio signal consisting of reproduced voice data, acoustic data, etc. into an analog signal, and outputs the converted analog signal auditorily.

[0060] exist Figure 1 In the example shown in , at least two control units connected via the communication network 121 can be integrated into a single control unit. Alternatively, each control unit can be composed of multiple control units. In addition, the vehicle control system 111 may include other control units not shown. In addition, some or all of the functions undertaken by any control unit in the description provided above can be undertaken by another control unit. In other words, when information is to be sent and received via the communication network 121, the prescribed operation processing can be performed by any control unit. In a similar manner, a sensor or device connected to any control unit can be connected to another control unit, and at the same time, multiple control units can send and receive detection information to each other via the communication network 121.

[0061] Hereinafter, the description of the communication network 121 will be omitted when the control unit performs communication via the communication network 121. For example, when the drive system control unit 122 and the integrated control unit 133 perform communication via the communication network 121, a simple description of "the drive system control unit 122 and the integrated control unit 133 perform communication" will be used.

[0062] In addition, hereinafter, descriptions of the in-vehicle network I / F 158 and the communication network 121 when the microcomputer 151 of the integrated control unit 133 performs communication via the in-vehicle network I / F 158 and the communication network 121 will be omitted. For example, when the microcomputer 151 performs communication with the drive system control unit 122 via the in-vehicle network I / F 158 and the communication network 121, a simple description of "the microcomputer 151 performs communication with the drive system control unit 122" will be used.

[0063] <Configuration Example of Illumination Section 125>

[0064] Figure 2 It is shown Figure 1 0 is a block diagram of a part of a configuration example of the illumination section 125 shown in FIG.

[0065] The lighting unit 125 includes a variable irradiation lamp 201 and an infrared lamp 202 .

[0066] The variable illumination lamp 201 includes a visible light source portion 211 that emits visible light, and is installed at a position where the front of the vehicle 101 can be illuminated with visible light. In addition, the variable illumination lamp 201 illuminates the front of the vehicle 101 with visible light under the control of the body system control unit 124.

[0067] In addition, the variable irradiation lamp 201 is capable of controlling the light intensity of each unit area created by dividing the irradiation area of ​​the visible light into a plurality of units, and controlling the irradiation pattern by controlling the light intensity of each unit area. Therefore, the irradiation pattern indicates the light intensity distribution of the visible light emitted from the variable irradiation lamp 201 in the spatial direction, or more specifically, indicates the light intensity distribution of each unit area of ​​the visible light.

[0068] For example, the visible light source section 211 has a configuration in which a plurality of light sources (e.g., LEDs) are arranged in an array. In addition, the irradiation pattern of the visible light is controlled by individually controlling the light intensity of each light source of the visible light source section 211. Controlling the light intensity of each light source includes setting the light intensity to 0, or in other words, turning off the light source.

[0069] Alternatively, for example, the visible light source section 211 includes a light source, a DMD (Digital Mirror Device), a lens, etc. In addition, the irradiation pattern of visible light is controlled by controlling the on / off state of each mirror of the DMD of the visible light source section 211 .

[0070] It should be noted that all unit regions do not necessarily have the same shape and the same size, but may include unit regions having different shapes or different sizes.

[0071] The infrared lamp 202 includes an infrared light source section 221 that emits infrared light, and is installed at a position where the front of the vehicle 101 can be irradiated with infrared light. In addition, the infrared lamp 202 irradiates the front of the vehicle 101 with infrared light under the control of the body system control unit 124.

[0072] <Configuration Example of Image Capture Unit 128>

[0073] Figure 3 It is shown Figure 1 2 is a block diagram of a configuration example of the imaging section 128 shown in FIG.

[0074] The imaging unit 128 includes a visible light camera 251 and an infrared camera 252 .

[0075] The visible light camera 251 includes a visible light imaging element 261 having sensitivity to visible light, and is installed at a position where the front of the vehicle 101 can be photographed. For example, the visible light camera 251 is installed inside the headlight of the vehicle 101, on the top of the dashboard, above the rearview mirror inside the cabin, on the roof, etc. In addition, the visible light camera 251 photographs the front of the vehicle 101 under the control of the external vehicle information detection unit 127, and supplies the data of the color image (hereinafter, referred to as the visible image) obtained as a result of the photographing to the external vehicle information detection unit 127.

[0076] The variable irradiation lamp 201 irradiates visible light in the shooting direction of the visible light camera 251 (visible light imaging element 261 ), and can irradiate visible light substantially uniformly at least within the viewing angle of the visible light camera 251 .

[0077] The infrared camera 252 includes an infrared light imaging element 271 having sensitivity to infrared rays (infrared light), and is installed at a position that can shoot the front of the vehicle 101. For example, the infrared camera 252 is installed at a position similar to that of the visible light camera 251. In addition, the infrared camera 252 shoots the front of the vehicle 101 under the control of the external vehicle information detection unit 127, and supplies the data of a black and white image (hereinafter, referred to as an infrared image) obtained as a result of the shooting to the external vehicle information detection unit 127.

[0078] The infrared lamp 202 irradiates infrared light in the shooting direction of the infrared camera 252 (infrared light imaging element 271 ), and can irradiate infrared light approximately uniformly at least within the viewing angle of the infrared camera 252 .

[0079] <Configuration Example of Information Processing Section 301>

[0080] Figure 4 1 shows a configuration example of an information processing section 301 which is a part of the function realized when the microcomputer 151 of the integrated control unit 133 executes a prescribed control program. In this figure, illustration of the communication network 121 and the in-vehicle network I / F 158 between the drive system control unit 122, the body system control unit 124, the external vehicle information detection unit 127, and the information processing section 301 is omitted.

[0081] The information processing section 301 includes a recognition section 311 , a distance measuring section 312 , an illumination control section 313 , and an operation control section 314 .

[0082] The recognition section 311 acquires data of a visible image and an infrared image from the external vehicle information detection unit 127, and performs image recognition processing on the visible image and the infrared image. In this case, the image recognition processing refers to, for example, processing for recognizing an object in an image. In addition, the recognition section 311 calculates a recognition score indicating the reliability of the recognition result for each area (hereinafter, referred to as a recognition area) based on the recognition result of the object through the image recognition processing.

[0083] The recognition area is, for example, an area corresponding to each recognized object. However, the recognition area may also include an area where object recognition failed. In addition, for example, the recognition score indicates the reliability of the recognition result of the object in the recognition area, or in other words, the certainty that the object has been recognized in the recognition area.

[0084] The recognition section 311 supplies the irradiation control section 313 and the operation control section 314 with recognition information including the position of each recognition area, the type of the object in each recognition area, and the recognition score.

[0085] The distance measuring section 312 performs a detection process of the distance to an object in front of the vehicle 101 based on the image data of the stereo camera, the image data of the ToF camera, or the reflected wave information of the ultrasonic sensor, the radar device, or the LIDAR device supplied from the external vehicle information detection unit 127. The distance measuring section 312 supplies the distance measuring information including the detection result of the distance to each object to the illumination control section 313 and the operation control section 314.

[0086] The irradiation control unit 313 controls the irradiation of visible light and infrared light by the lighting unit 125 based on the recognition result of the object in front of the vehicle 101, the detection result of the distance to the object in front of the vehicle 101, the detection result of the speed of the vehicle 101 notified from the drive system control unit 122, etc.

[0087] For example, the illumination control section 313 generates a control signal (hereinafter, referred to as a visible light control signal) including an illumination mode of the variable illumination lamp 201, and supplies the visible light control signal to the body system control unit 124. The body system control unit 124 controls the illumination mode of the variable illumination lamp 201 based on the visible light control signal.

[0088] In addition, for example, the irradiation control section 313 generates a control signal (hereinafter, referred to as an infrared light control signal) for controlling the irradiation of infrared light of the infrared lamp 202, and supplies the infrared light control signal to the body system control unit 124. The body system control unit 124 controls the irradiation of infrared light of the infrared lamp 202 based on the infrared light control signal.

[0089] The operation control section 314 controls the operation of the device related to the drive system of the vehicle 101 based on the recognition result of the object in front of the vehicle 101, the detection result of the distance to the object in front of the vehicle 101, etc. For example, the operation control section 314 generates a control signal (hereinafter, referred to as a drive system control signal) including the control content of the device to be the control target, and supplies the drive system control signal to the drive system control unit 122. The drive system control unit 122 controls the operation of the device to be the control target based on the drive system control signal.

[0090] In addition, the operation control section 314 controls the operation of various devices mounted to the body of the vehicle 101 based on the recognition result of the object in front of the vehicle 101, the detection result of the distance to the object in front of the vehicle 101, etc. For example, the operation control section 314 generates a control signal (hereinafter, referred to as a body system control signal) including the control content of the device to be the control target, and provides the body system control signal to the body system control unit 124. The body system control unit 124 controls the operation of the device to be the control target based on the body system control signal.

[0091] <Lighting Control Processing>

[0092] Next, we will refer to Figure 5 The lighting control process to be performed by the vehicle 101 is described with reference to the flowchart shown in FIG.

[0093] The process is started when, for example, a condition for starting the irradiation of visible light is satisfied (such as when the ambient brightness drops below a prescribed threshold value during the start-up of the vehicle 101 or when the switch of the variable irradiation lamp 201 is turned on). In addition, the process is ended when, for example, a condition for ending the irradiation of visible light is satisfied (such as when the power supply of the vehicle 101 is turned off, when the ambient brightness of the vehicle 101 is equal to or exceeds a prescribed threshold value, or when the switch of the variable irradiation lamp 201 is turned off).

[0094] In step S1 , the vehicle control system 111 starts irradiation of infrared light and visible light.

[0095] Specifically, for example, the irradiation control unit 313 generates an infrared light control signal for controlling to irradiate infrared light with a prescribed light intensity, and supplies the generated infrared light control signal to the body system control unit 124. The body system control unit 124 controls the infrared lamp 202 to irradiate infrared light with a prescribed light intensity based on the infrared light control signal. Thus, the front of the vehicle 101 is irradiated with infrared light having a prescribed light intensity.

[0096] When infrared light is used, there is no need to worry about dazzling passers-by, drivers of other vehicles, etc., and therefore the light intensity can be increased as much as possible within a range that does not cause pixel saturation in the infrared image.

[0097] In addition, for example, the illumination control unit 313 generates a visible light control signal for controlling to irradiate visible light in the standard illumination mode, and supplies the generated visible light control signal to the body system control unit 124. The body system control unit 124 controls the variable illumination lamp 201 based on the visible light control signal so as to irradiate visible light in the standard illumination mode. Therefore, the front of the vehicle 101 is illuminated with visible light in the standard mode.

[0098] In this case, the standard irradiation mode is, for example, a mode in which the light intensities of all unit areas are set to the same initial value.

[0099] In step S2, the photographing unit 128 photographs an infrared image and a visible image. Specifically, the infrared camera 252 photographs the front of the vehicle 101, and supplies the data of the infrared image obtained as a result of the photographing to the recognition unit 311 and the distance measuring unit 312 via the external vehicle information detection unit 127. In addition, the visible light camera 251 photographs the front of the vehicle 101, and supplies the data of the visible image obtained as a result of the photographing to the recognition unit 311 and the distance measuring unit 312 via the external vehicle information detection unit 127.

[0100] In step S3, the recognition unit 311 performs image recognition processing. For example, the recognition unit 311 performs semantic segmentation as the image recognition processing.

[0101] Specifically, the recognition unit 311 performs semantic segmentation on the infrared image, detects the type and position of each object in the infrared image at the pixel level, and divides the infrared image into multiple recognition areas based on the recognition result. In addition, the recognition unit 311 calculates the recognition score of each recognition area in the infrared image.

[0102] In addition, the recognition unit 311 performs semantic segmentation on the visible image in a similar manner, performs detection of the type and position of each object in the visible image at the pixel level, and divides the visible image into a plurality of recognition areas based on the recognition result. In addition, the recognition unit 311 calculates the recognition score of each recognition area in the visible image.

[0103] In this case, a recognition area is basically set for each recognition object. Therefore, each recognition area basically includes one object. However, for example, in the case where the boundary line of the object is blurred, multiple objects may be included in one recognition area. In addition, an area where object recognition fails may be set as a recognition area. In such a case, the object in the recognition area is unknown.

[0104] In addition, any method can be used as a calculation method of the recognition score.

[0105] Infrared images lack color information and contain more noise than visible images. Therefore, the results of image recognition processing (in this example, semantic segmentation) for infrared images have lower reliability than the results of image recognition processing for visible images.

[0106] The recognition section 311 supplies the irradiation control section 313 and the operation control section 314 with recognition information including the position of each recognition area, the type of the object in each recognition area, and the recognition score.

[0107] In step S4, the vehicle control system 111 performs a distance measurement process. For example, the external vehicle information detection section 129 supplies the reflected wave information to the distance measurement section 312 via the external vehicle information detection unit 127. Based on the reflected wave information, the distance measurement section 312 detects the distance to each object in front of the vehicle 101. The distance measurement section 312 supplies the distance measurement information including the detection result of the distance to each object to the illumination control section 313 and the operation control section 314.

[0108] Any method may be used for the distance measurement process. For example, the distance measurement process may be performed using image data of a stereo camera, image data of a ToF camera, or the like.

[0109] In step S5 , the drive system control unit 122 detects the vehicle speed. Specifically, the drive system control unit 122 detects the speed of the vehicle 101 based on the signal input from the vehicle state detection unit 123 , and supplies information including the detection result to the irradiation control unit 313 and the operation control unit 314 .

[0110] In step S6, the vehicle control system 111 controls the illumination mode of the visible light. Specifically, the illumination control unit 313 sets the illumination mode by setting the light intensity of each unit area. For example, the illumination control unit 313 sets the illumination mode so as to increase the recognition score of each recognition area of ​​the visible image as much as possible. The illumination control unit 313 generates a visible light control signal including the set illumination mode, and supplies the visible light control signal to the body system control unit 124. The body system control unit 124 controls the illumination mode of the variable illumination lamp 201 based on the visible light control signal.

[0111] An example of a setting method of the irradiation mode will now be described.

[0112] For example, the irradiation control unit 313 sets the irradiation mode based on the result of the image recognition processing on the infrared image.

[0113] For example, the irradiation control unit 313 sets the light intensity of visible light for the recognition area in the infrared image whose recognition score is lower than the threshold value T1 (hereinafter, referred to as the low reliability area) so that the recognition score of the low reliability area in the visible image increases. For example, the irradiation control unit 313 increases the light intensity of visible light for the low reliability area (making the low reliability area brighter).

[0114] On the other hand, the irradiation control section 313 sets the light intensity of visible light for the recognition area having a recognition score equal to or higher than the threshold value T1 in the infrared image, based on, for example, the type of the object in the recognition area.

[0115] For example, when a road surface is included in the recognition area, the illumination control section 313 controls the light intensity of visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher.

[0116] For example, when a person or another vehicle is included in the recognition area, the illumination control unit 313 reduces the light intensity of the visible light for the recognition area. Thus, passers-by and drivers of other vehicles are prevented from being dazzled. Alternatively, the illumination control unit 313 may reduce the light intensity of the visible light to 0 so that the visible light is not irradiated in the recognition area.

[0117] For example, when an object emitting light (e.g., a traffic light or a street light) is included in the recognition area, the illumination control section 313 reduces the light intensity of visible light for the recognition area. This is because an object emitting light increases the accuracy of the image recognition process even when not illuminated by visible light. Alternatively, the illumination control section 313 may reduce the light intensity of visible light to 0 so that visible light is not irradiated in the recognition area.

[0118] For example, when an object including a reflective member that reflects visible light (e.g., a traffic sign) is included in the recognition area, the illumination control section 313 reduces the light intensity of visible light for the recognition area. This is because the recognition area becomes brighter than other areas due to the reflection of the visible light by the reflective member, and may become saturated in the visible image. The illumination control section 313 reduces the light intensity of visible light for the recognition area to a level at which the recognition area does not become saturated in the visible image.

[0119] In addition, for example, the irradiation control section 313 controls the irradiation mode based on the result of the image recognition process on the visible image.

[0120] For example, when the illumination mode is controlled based on the result of the image recognition processing for the infrared image, but there is an identification area (low reliability area) in the visible image whose identification score is lower than the threshold value T1, the illumination control unit 313 controls the light intensity of the visible light to be used to illuminate the low reliability area based on the average value of the brightness of the low reliability area in the visible image. For example, when the average value of the brightness of the low reliability area is lower than the threshold value Th2, the illumination control unit 313 increases the light intensity of the visible light for the low reliability area (making the low reliability area brighter). On the other hand, when the average value of the brightness of the low reliability area is equal to or higher than the threshold value Th2, the illumination control unit 313 reduces the light intensity of the visible light for the low reliability area (making the low reliability area darker).

[0121] In addition, for example, the illumination control unit 313 sets the light intensity of the visible light for the recognition area based on the distance to the object in the recognition area. For example, the shorter the distance to the object in the recognition area, the further the illumination control unit 313 reduces the light intensity of the visible light for the recognition area (making the recognition area darker). On the other hand, the longer the distance to the object in the recognition area, the further the illumination control unit 313 increases the light intensity of the visible light for the recognition area (making the recognition area brighter).

[0122] In addition, for example, when the distance to the object in the recognition area is equal to or greater than the threshold value Th3, the illumination control section 313 may exclude the recognition area from the image recognition process and set the light intensity of the illumination light to be used to illuminate the recognition area to 0. Thus, for example, illumination of a distant object such as the sky with visible light is stopped.

[0123] For example, the irradiation control unit 313 may change the threshold value Th3 according to the speed of the vehicle 101. For example, the irradiation control unit 313 increases the threshold value Th3 as the speed of the vehicle 101 increases so that a more distant object is irradiated with visible light. On the other hand, for example, the irradiation control unit 313 decreases the threshold value Th3 as the speed of the vehicle 101 decreases so that a more distant object is not irradiated with visible light. This is because, when the speed of the vehicle 101 is high, a more distant object must be monitored for collision prevention, etc., but when the speed of the vehicle 101 is low, there is no such need to monitor a more distant object.

[0124] Now refer to Figures 6 to 8 A specific example of the control method of the irradiation pattern is described.

[0125] Figure 6 An example of a captured infrared image of the front of the vehicle 101 is shown.

[0126] The recognition area R1 is an area including a person riding a bicycle. For example, when the recognition score of the recognition area R1 is equal to or higher than the threshold value Th1, the light intensity of the visible light for the recognition area R1 is reduced. On the other hand, when the recognition score of the recognition area R1 is lower than the threshold value Th1, the light intensity of the visible light for the recognition area R1 is increased. In addition, the light intensity of the visible light for the recognition area R1 is changed according to, for example, the distance from the person in the recognition area R1.

[0127] The recognition area R2 is an area including a traffic sign having a reflective plate. For example, the light intensity of the visible light for the recognition area R2 is reduced. In addition, the light intensity of the visible light for the recognition area R2 is changed according to, for example, the distance from the traffic sign in the recognition area R2.

[0128] The recognition area R3 is an area including a traffic light emitting light. For example, the light intensity of the visible light for the recognition area R3 is reduced. In addition, the light intensity of the visible light for the recognition area R3 is changed according to, for example, the distance from the traffic light in the recognition area R3.

[0129] The recognition area R4 is an area including another vehicle. For example, when the recognition score of the recognition area R4 is equal to or higher than the threshold value Th1, the light intensity of the visible light for the recognition area R4 is reduced. On the other hand, when the recognition score of the recognition area R4 is lower than the threshold value Th1, the light intensity of the visible light for the recognition area R4 is increased. In addition, for example, the light intensity of the visible light for the recognition area R4 is changed according to the distance from the vehicle in the recognition area R4.

[0130] The recognition region R5 is a region including distant buildings and the sky. For example, the light intensity of visible light for the recognition region R5 is set to zero.

[0131] The recognition area R6 is an extremely dark area close to the vehicle 101, and its recognition score is extremely low. For example, in order to improve the recognition score, the light intensity of visible light for the recognition area R6 is increased.

[0132] Figure 7 Show that for Figure 6 Example of the results of semantic segmentation performed on visible images of the same viewpoint in .

[0133] There is a road surface in the recognition region R11. Therefore, control is performed so that the recognition region R11 is constantly illuminated with visible light. The light intensity of the visible light for the recognition region R11 is changed according to, for example, the distance from the road surface in the recognition region R11.

[0134] The recognition area R12 is an area including a person riding a bicycle. For example, when the recognition score of the recognition area R12 is equal to or higher than the threshold value Th1, the light intensity of the visible light for the recognition area R12 is reduced. On the other hand, when the recognition score of the recognition area R12 is lower than the threshold value Th1, the light intensity of the visible light for the recognition area R12 is increased. In addition, the light intensity of the visible light for the recognition area R12 is changed according to, for example, the distance from the person in the recognition area R12.

[0135] The recognition region R13 is a region including a traffic sign. For example, the light intensity of the visible light for the recognition region R13 is reduced. In addition, the light intensity of the visible light for the recognition region R13 is changed according to, for example, the distance from the traffic sign in the recognition region R13.

[0136] The recognition region R14 is a region including a traffic light. For example, the light intensity of the visible light for the recognition region R14 is reduced. In addition, the light intensity of the visible light for the recognition region R14 is changed according to, for example, the distance from the traffic light in the recognition region R14.

[0137] The recognition areas R15 to R17 are areas including another vehicle. For example, the light intensity of visible light is reduced for areas where the recognition score is equal to or higher than the threshold value Th1 among the recognition areas R15 to R17. On the other hand, the light intensity of visible light is increased for areas where the recognition score is lower than the threshold value Th1 among the recognition areas R15 to R17. In addition, the light intensity of visible light for the recognition areas R15 to R17 is changed according to, for example, the distance from the vehicle within the recognition areas R15 to R17.

[0138] The recognition region R18 is an extremely dark region close to the vehicle 101, and its recognition score is extremely low. For example, in order to improve the recognition score, the light intensity of visible light for the recognition region R18 is increased.

[0139] Figure 8Another example of a captured infrared image of the front of vehicle 101 is shown.

[0140] The recognition region R21 is an extremely dark region close to the vehicle 101, and its recognition score is extremely low. For example, in order to improve the recognition score, the light intensity of visible light for the recognition region R21 is increased.

[0141] The recognition regions R22 and R23 are regions including people. For example, the light intensity of visible light is reduced for regions in the recognition regions R22 and R23 where the recognition score is equal to or higher than the threshold value Th1. On the other hand, the light intensity of visible light is increased for regions in the recognition regions R22 and R23 where the recognition score is lower than the threshold value Th1. In addition, the light intensity of visible light for the recognition regions R22 and R23 is changed according to, for example, the distance from the people in the recognition regions R22 and R23.

[0142] return Figure 5 , the processing then returns to step S2, and the processing of step S2 and thereafter is performed.

[0143] As described above, by appropriately controlling the illumination pattern of visible light, the accuracy of image recognition processing for visible images is improved. As a result, for example, the recognition accuracy of objects in visible images is improved, and in turn, the performance of the functions of the vehicle 101 (e.g., autonomous driving, ADAS, etc.) that use the recognition results is also improved.

[0144] <<2. Variants>>

[0145] Hereinafter, modifications of the embodiments of the present technology described above will be described.

[0146] Although an example in which semantic segmentation is used as the image recognition process has been presented in the description given above, the present technology can also be applied when other methods of the image recognition process are used.

[0147] In addition, for example, when image recognition processing is performed using only visible images, the present technology can also be applied. In this case, for example, the irradiation pattern is controlled based on the recognition score of each recognition area of ​​the visible image, the type of object, and the distance to the object according to a method similar to the case of using the result of image recognition processing for infrared images.

[0148] In addition, in the control process of the above-mentioned illumination mode, a recognition area in which the light intensity of visible light is reduced or set to 0 can be created. This is not a particular problem in the case of automatic driving without involving the driver. On the other hand, when the driver is driving, the driver's visibility may deteriorate. Therefore, when the driver is driving, the illumination mode can be controlled by the above-mentioned method only during the time period in which the image recognition process is performed, and the front of the vehicle 101 can be uniformly illuminated with visible light during the time period in which the image recognition process is not performed.

[0149] In addition, although an example of performing image recognition processing on an image in front of the vehicle 101 has been presented in the description given above, the present technology can also be applied when image recognition processing is performed on an image in a direction other than the front (e.g., the side, rear, top, etc.) In this case, for example, the irradiation pattern of visible light in the shooting direction of each image is controlled.

[0150] In addition, the present technology can also be applied to mobile bodies other than vehicles. For example, the present technology can also be applied to mobile bodies such as personal movers, construction machinery, and agricultural and farm machinery (tractors). In addition, the present technology can also be applied to mobile bodies that are remotely operated (manipulated) without carrying users (such as drones and robots) and mobile bodies that can be driven automatically.

[0151] In addition, the present technology can also be applied when image recognition processing is performed on images other than images captured from a moving object. For example, the present technology can also be applied when image recognition processing is performed on monitoring images captured by cameras installed on the periphery of a road.

[0152] also, Figure 4 All of the information processing section 301 shown in FIG. 1 is not necessarily implemented by the integrated control unit 133 , but a part or all of the information processing section 301 may be implemented by one or more other control units.

[0153] In addition, part or all of the information processing unit 301 may be provided in, for example, the variable irradiation lamp 201 or the visible light camera 251 .

[0154] <<3. Others>>

[0155] The above-described series of processing can be executed by hardware or by software.

[0156] Depend on Figure 1 The program executed by the microcomputer 151 or the like shown in FIG. 1 may be a program that causes processing to be executed time-sequentially in the order described in this specification, or a program that causes processing to be executed in parallel or at necessary timing such as when a call is made.

[0157] In addition, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), and it is not important whether all components exist in the same housing. Therefore, multiple devices housed in separate housings but connected to each other via a network and a single device in which multiple modules are housed in a single housing are both considered to be systems.

[0158] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the gist of the present technology.

[0159] For example, the present technology may adopt a configuration of cloud computing in which a single function is shared among a plurality of devices via a network and the single function is processed cooperatively by the plurality of devices.

[0160] In addition, each step described in the above flowchart may be performed by a single device or may be performed by a plurality of devices in a shared manner.

[0161] Furthermore, when a single step includes a plurality of processing steps, the plurality of processing steps included in the single step may be performed by a plurality of devices in a shared manner in addition to being performed by a single device.

[0162] <Configuration combination example>

[0163] The present technology may also be configured as follows.

[0164] (1) An information processing device comprising:

[0165] a recognition section configured to perform image recognition processing on the image captured by the capturing section and calculate reliability of the recognition result for each recognition area as an area based on the object recognition result; and

[0166] An illumination control unit is configured to control an illumination pattern of visible light in an imaging direction of the imaging unit so that reliability for at least a portion of a recognition area is increased.

[0167] (2) The information processing device according to (1), wherein:

[0168] The irradiation control section is configured to control the light intensity of the visible light for a low reliability region which is an identification region having reliability lower than a predetermined threshold value so that the reliability for the low reliability region increases.

[0169] (3) The information processing device according to (2), wherein:

[0170] The illumination control section is configured to control light intensity of visible light for a low reliability region based on brightness of the low reliability region in an image.

[0171] (4) The information processing device according to any one of (1) to (3), wherein:

[0172] The illumination control section is configured to control the illumination mode further based on the type of the object in the recognition area.

[0173] (5) The information processing device according to (4), wherein:

[0174] The illumination control section is configured to reduce light intensity of visible light for a recognition area including an object emitting light and an object having a reflective member reflecting visible light.

[0175] (6) The information processing device according to any one of (1) to (5), wherein:

[0176] The illumination control section is configured to control the illumination mode further based on the distance to the object in the recognition area.

[0177] (7) The information processing device according to (6), wherein:

[0178] The irradiation control section is configured to set the light intensity of the visible light to zero for a recognition area including objects separated by a prescribed distance or more.

[0179] (8) The information processing device according to any one of (1) to (7), wherein:

[0180] The images include infrared images and visible images, and

[0181] The irradiation control section is configured to control an irradiation mode based on reliability of an image recognition process on the infrared image.

[0182] (9) The information processing device according to (8), wherein:

[0183] The illumination control section is configured to control the illumination mode further based on the reliability of the image recognition process on the visible image.

[0184] (10) The information processing device according to any one of (1) to (9), wherein:

[0185] The image is an image obtained by capturing the surroundings of the moving object.

[0186] (11) The information processing device according to (10), wherein:

[0187] The mobile object is a vehicle, and

[0188] The illumination control section is configured to control an illumination pattern so as to illuminate an identification area including a road surface with visible light.

[0189] (12) The information processing device according to (10) or (11), wherein:

[0190] The mobile object is a vehicle, and

[0191] The illumination control section is configured to reduce light intensity of visible light for a recognition area including a person or other vehicles.

[0192] (13) The information processing device according to any one of (1) to (12), wherein:

[0193] The irradiation control unit is configured to control the irradiation pattern by controlling the light intensity of the visible light in predetermined area units.

[0194] (14) The information processing device according to any one of (1) to (13), wherein:

[0195] The illumination pattern is the distribution of light intensity of visible light in spatial directions.

[0196] (15) The information processing device according to any one of (1) to (14), wherein:

[0197] The irradiation control section is configured to output a control signal including an irradiation mode.

[0198] (16) The information processing device according to any one of (1) to (15), wherein:

[0199] The recognition section is configured to perform semantic segmentation as the image recognition process.

[0200] (17) An information processing method comprising:

[0201] performing image recognition processing on the image captured by the capturing section, and calculating reliability of the recognition result for each recognition area which is an area based on the recognition result of the object; and

[0202] An irradiation pattern of visible light in an imaging direction of the imaging unit is controlled so that reliability for at least a portion of a recognition area is increased.

[0203] (18) A photographing device comprising:

[0204] Imaging element;

[0205] a recognition section configured to perform image recognition processing on the image captured by the imaging element and calculate reliability of the recognition result for each recognition area that is an area based on the object recognition result; and

[0206] An illumination control section is configured to control an illumination pattern of visible light in a photographing direction of the imaging element so that reliability for at least a portion of a recognition area increases.

[0207] (19) A lighting device comprising:

[0208] Light source department;

[0209] a recognition section configured to perform image recognition processing on the image captured by the capturing section and calculate reliability of the recognition result for each recognition area as an area based on the object recognition result; and

[0210] An illumination control unit is configured to control an illumination pattern of the visible light by the light source unit in the imaging direction of the imaging unit so that reliability for at least a part of the recognition area is increased.

[0211] (20) A mobile object, comprising:

[0212] Filming Department;

[0213] an illumination unit configured to irradiate visible light in a shooting direction of the shooting unit;

[0214] a recognition section configured to perform image recognition processing on the image captured by the capturing section and calculate reliability of the recognition result for each recognition area as an area based on the object recognition result; and

[0215] An illumination control unit is configured to control an illumination pattern of the visible light of the illumination unit so that reliability for at least a portion of the recognition area is increased.

[0216] It should be noted that the beneficial effects described in this specification are merely exemplary and not restrictive, and other beneficial effects may occur.

[0217] [Reference mark list]

[0218] 101 Vehicles

[0219] 111 Vehicle Control System

[0220] 122 Drive system control unit

[0221] 124 Body system control unit

[0222] 125 Lighting Department

[0223] 127 External vehicle information detection unit

[0224] 128 Photography Department

[0225] 129 External vehicle information detection department

[0226] 133 Integrated control unit

[0227] 151 Microcomputer

[0228] 201 Variable illumination lamp

[0229] 202 infrared light

[0230] 211 Visible light source

[0231] 212 Infrared light source

[0232] 251 Visible light camera

[0233] 252 Infrared Camera

[0234] 261 Visible light imaging element

[0235] 271 Infrared imaging element

[0236] 301 Information Processing Department

[0237] 311 Identification Department

[0238] 312 Distance Measurement Department

[0239] 313 Irradiation Control Department

[0240] 314 Operation Control Department

Claims

1. An information processing device, comprising: a recognition section configured to perform image recognition processing on the image captured by the capturing section, divide the visible image into a plurality of recognition areas based on the recognition result, and calculate the reliability of the recognition result for each recognition area, wherein each recognition area includes one object; and an illumination control section configured to generate a control signal including an illumination pattern of a variable illumination lamp, the variable illumination lamp illuminating the front of the vehicle with visible light under the control of the vehicle body system control unit, the variable illumination lamp being capable of controlling the light intensity of each unit area created by dividing an illumination area of ​​the visible light into a plurality of areas, and controlling the illumination pattern by controlling the light intensity of each unit area; The irradiation control unit is configured as follows: controlling an irradiation pattern of visible light in a photographing direction of the photographing unit based on the reliability; as well as controlling the light intensity of the visible light in a low reliability region, which is a recognition region whose reliability is lower than a predetermined threshold value, The illumination control unit is further configured to control the illumination mode based on the type of the object in the recognition area, including: When a road surface is included in the recognition area, the illumination control section is configured to control the light intensity of the visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher, and When a person or another vehicle or an object emitting light is included in the recognition area, the irradiation control section is configured to reduce the light intensity of the visible light to 0 so that the visible light is not irradiated in the recognition area.

2. The information processing device according to claim 1, wherein: The illumination control section is configured to control light intensity of visible light for a low reliability region based on brightness of the low reliability region in an image.

3. The information processing device according to claim 1, wherein: The illumination control section is configured to reduce light intensity of visible light for a recognition area including an object having a reflective member that reflects visible light.

4. The information processing device according to claim 1, wherein: The illumination control section is configured to control the illumination mode further based on the distance to the object in the recognition area.

5. The information processing device according to claim 4, wherein: The irradiation control section is configured to set the light intensity of the visible light to zero for a recognition area including objects separated by a prescribed distance or more.

6. The information processing device according to claim 1, wherein: The images include infrared images and visible images, and The irradiation control section is configured to control an irradiation mode based on reliability of an image recognition process on the infrared image.

7. The information processing device according to claim 6, wherein: The illumination control section is configured to control the illumination mode further based on the reliability of the image recognition process on the visible image.

8. The information processing device according to claim 1, wherein: The image is an image obtained by capturing the surroundings of the moving object.

9. The information processing device according to claim 8, wherein: The mobile object is a vehicle, and The illumination control section is configured to control an illumination pattern so as to illuminate an identification area including a road surface with visible light.

10. The information processing device according to claim 8, wherein: The moving object is a vehicle, and The illumination control section is configured to reduce light intensity of visible light for a recognition area including a person or other vehicles.

11. The information processing device according to claim 1, wherein: The irradiation control unit is configured to control the irradiation pattern by controlling the light intensity of the visible light in predetermined area units.

12. The information processing device according to claim 1, wherein: The illumination pattern is the distribution of light intensity of visible light in spatial directions.

13. The information processing device according to claim 1, wherein: The irradiation control section is configured to output a control signal including an irradiation mode.

14. The information processing device according to claim 1, wherein: The recognition section is configured to perform semantic segmentation as the image recognition process.

15. An information processing method, comprising: performing image recognition processing on an image captured by the capturing unit, dividing the visible image into a plurality of recognition areas based on a recognition result, and calculating reliability of the recognition result for each recognition area, wherein each recognition area includes an object; controlling an irradiation pattern of visible light in a photographing direction of the photographing unit based on the reliability; controlling the light intensity of visible light for a low reliability region which is an identification region having a reliability lower than a predetermined threshold value; as well as Controls the illumination pattern based on the type of object in the identified area, including: When a road surface is included in the recognition area, controlling the light intensity of visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher, and When a person or another vehicle or an object emitting light is included in the recognition area, the light intensity of the visible light is reduced to 0 so that the visible light is not irradiated in the recognition area, Among them, a control signal including an illumination mode of a variable illumination lamp is generated by an illumination control unit. The variable illumination lamp illuminates the front of the vehicle with visible light under the control of a vehicle body system control unit. The variable illumination lamp is capable of controlling the light intensity of each unit area created by dividing the illumination area of ​​visible light into multiple areas, and controlling the illumination mode by controlling the light intensity of each unit area.

16. A photographing device, comprising: Imaging element; a recognition section configured to perform image recognition processing on the image captured by the imaging element, divide the visible image into a plurality of recognition areas based on the recognition result, and calculate the reliability of the recognition result for each recognition area, wherein each recognition area includes an object; and an illumination control section configured to generate a control signal including an illumination pattern of a variable illumination lamp, the variable illumination lamp illuminating the front of the vehicle with visible light under the control of the vehicle body system control unit, the variable illumination lamp being capable of controlling the light intensity of each unit area created by dividing an illumination area of ​​the visible light into a plurality of areas, and controlling the illumination pattern by controlling the light intensity of each unit area; The irradiation control unit is configured as follows: controlling an irradiation pattern of visible light in a shooting direction of the imaging element based on the reliability; as well as controlling the light intensity of the visible light in a low reliability region, which is a recognition region whose reliability is lower than a predetermined threshold value, The illumination control unit is further configured to control the illumination mode based on the type of the object in the recognition area, including: When a road surface is included in the recognition area, the illumination control section is configured to control the light intensity of the visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher, and When a person or another vehicle or an object emitting light is included in the recognition area, the irradiation control section is configured to reduce the light intensity of the visible light to 0 so that the visible light is not irradiated in the recognition area.

17. A lighting device, comprising: a recognition section configured to perform image recognition processing on the image captured by the capturing section, divide the visible image into a plurality of recognition areas based on the recognition result, and calculate the reliability of the recognition result for each recognition area, wherein each recognition area includes one object; and an illumination control section configured to generate a control signal including an illumination pattern of a variable illumination lamp, the variable illumination lamp illuminating the front of the vehicle with visible light under the control of the vehicle body system control unit, the variable illumination lamp being capable of controlling the light intensity of each unit area created by dividing an illumination area of ​​the visible light into a plurality of areas, and controlling the illumination pattern by controlling the light intensity of each unit area; The irradiation control unit is configured as follows: controlling an irradiation pattern of visible light in a photographing direction of the photographing section; and controlling the light intensity of the visible light in a low reliability region, which is a recognition region whose reliability is lower than a predetermined threshold value, The illumination control unit is further configured to control the illumination mode based on the type of the object in the recognition area, including: When a road surface is included in the recognition area, the illumination control section is configured to control the light intensity of the visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher, and When a person or another vehicle or an object emitting light is included in the recognition area, the irradiation control section is configured to reduce the light intensity of the visible light to 0 so that the visible light is not irradiated in the recognition area.

18. A mobile object, comprising: Filming Department; an illumination unit configured to irradiate visible light in a shooting direction of the shooting unit; a recognition section configured to perform image recognition processing on the image captured by the capturing section, divide the visible image into a plurality of recognition areas based on the recognition result, and calculate the reliability of the recognition result for each recognition area, wherein each recognition area includes an object; and an illumination control section configured to generate a control signal including an illumination pattern of a variable illumination lamp, the variable illumination lamp illuminating the front of the vehicle with visible light under the control of the vehicle body system control unit, the variable illumination lamp being capable of controlling the light intensity of each unit area created by dividing an illumination area of ​​the visible light into a plurality of areas, and controlling the illumination pattern by controlling the light intensity of each unit area; The irradiation control unit is configured as follows: controlling an irradiation pattern of visible light of the illumination unit based on the reliability; as well as controlling the light intensity of the visible light in a low reliability region, which is a recognition region whose reliability is lower than a predetermined threshold value, The illumination control unit is further configured to control the illumination mode based on the type of the object in the recognition area, including: When a road surface is included in the recognition area, the illumination control section is configured to control the light intensity of the visible light for the recognition area so that the recognition area is constantly illuminated with a light intensity of a certain level or higher, and When a person or another vehicle or an object emitting light is included in the recognition area, the irradiation control section is configured to reduce the light intensity of the visible light to 0 so that the visible light is not irradiated in the recognition area.

Images