Identification device, identification method and program product

By using the light quantity detection result selection recognizer in the identification control unit and adjusting the imaging parameters, the noise and light quantity problems are solved, and the subject recognition accuracy is improved under low light conditions.

CN111684455BActive Publication Date: 2024-11-08SONY GROUP CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201880088309.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-02-09
Filing Date
2018-11-16
Publication Date
2024-11-08
Estimated Expiration
2038-11-16

AI Technical Summary

Technical Problem

The prior art is affected by noise and light quantity problems when identifying a subject, resulting in a decrease in recognition accuracy, especially when the light quantity is insufficient, it is difficult to obtain a clear image.

Method used

By generating selection determination information based on the light quantity detection result of the imaging area, selecting an appropriate recognizer for image processing, adjusting aperture, shutter speed, analog gain and grayscale conversion to optimize image quality and improve recognition accuracy.

Benefits of technology

It realizes accurate identification of the subject under conditions of poor noise and light quantity, improves recognition accuracy and image clarity, and avoids the need to increase exposure time or aperture width to increase light quantity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111684455B_ABST
    Figure CN111684455B_ABST
Patent Text Reader

Abstract

In the present invention, the recognition control unit 40 detects the amount of light of the imaging area based on the image signal from the camera unit 20, the control signal from the camera control unit 30, and the like, and generates selection discrimination information based on the detection result of the amount of light. The recognition processing unit 50 divides the image of the imaging area generated by the camera unit 20 into a plurality of areas based on the selection discrimination information generated by the recognition control unit 40, and selects a discriminator for each area. In addition, the recognition processing unit 50 performs subject recognition processing on the image of the corresponding area by using the selected discriminator. Even if a bright area and a dark area are included in the captured image, the subject in each area can be easily recognized with high accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present technology relates to a recognition device, a recognition method, and a program, and enables accurate recognition of a subject included in a captured image. Background Art

[0002] Conventionally, in order to accurately detect a subject at night, the exposure amount during imaging is automatically switched. For example, in Patent Document 1, the shooting mode is switched between a first shooting mode capable of shooting a lighted outline portion and a second shooting mode capable of detecting an object without light.

[0003] Citation List

[0004] Patent Literature

[0005] Patent Document 1: Japanese Patent Application Laid-Open No. 2009-234344 Summary of the invention

[0006] Problems to be solved by the present invention

[0007] Meanwhile, in the case of recognizing a subject included in a captured image by using an identifier, for example, a captured image having a large amount of noise may deteriorate the recognition accuracy. In addition, in order to obtain a captured image with less noise, when the exposure time is prolonged and the amount of light is increased, the image of the subject that has moved becomes blurred, and when the aperture is widened to increase the amount of light, the depth of field becomes shallow, and a captured image with little blur cannot be obtained.

[0008] Therefore, an object of the present technology is to provide a recognition device, a recognition method, and a program capable of easily and accurately recognizing a subject included in a captured image.

[0009] Solution to the problem

[0010] The first aspect of this technology is

[0011] An identification device, comprising:

[0012] a recognition control unit configured to generate selection determination information based on a light amount detection result of the imaging area; and

[0013] a recognition processing unit configured to perform subject recognition processing on a captured image showing the imaging area by using a recognizer selected based on the selection determination information generated by the recognition control unit

[0014] In the present technology, selection determination information is generated based on the result of light amount detection of the imaging area. The light amount of the imaging area is detected based on a control signal for controlling the operation of the imaging unit for acquiring the captured image and an image signal generated by the imaging unit. The control signal is a signal for controlling at least any one of the aperture, shutter speed, analog gain adjustment or grayscale conversion. For example, detection is performed based on control signals for the aperture, shutter speed, analog gain adjustment and grayscale conversion and an image signal that has undergone analog gain adjustment and grayscale conversion. In addition, detection can be performed based on, for example, control signals for the aperture, shutter speed and analog gain and an image signal that has undergone analog gain adjustment. In addition, detection can be performed based on control signals for the aperture and shutter speed and an image signal before performing analog gain adjustment. In addition, the amount of light can be detected based on an image signal generated by the imaging unit and the illuminance detected by an illuminance detection unit that detects the illuminance of the imaging environment.

[0015] The recognition control unit performs noise estimation based on the light amount detection result, and uses the noise estimation result as selection determination information. The recognition processing unit selects an identifier based on the selection determination information. For example, the recognition processing unit selects a first identifier when the noise indicated by the selection determination information is equal to or less than a threshold value, and selects a second identifier that is more resistant to noise than the first identifier when the noise indicated by the selection determination information is greater than the threshold value. Note that the recognition processing unit performs area division of the captured image based on the selection determination information, and selects an identifier for each divided area. In addition, the recognition control unit can perform area division of the captured image based on conversion function information of grayscale conversion indicated by a control signal for controlling the operation of the imaging unit that acquires the captured image, and can generate selection determination information for each divided area.

[0016] The second aspect of this technology is

[0017] An identification method, comprising:

[0018] generating, by the recognition control unit, selection determination information based on a light amount detection result of the imaging area; and

[0019] A subject recognition process of a captured image showing the imaging area is performed by a recognition processing unit by using a recognizer selected based on the selection determination information generated by the recognition control unit.

[0020] The third aspect of this technology is

[0021] A program for causing a computer to execute a recognition process, the program causing the computer to execute:

[0022] A process of generating selection determination information based on a light amount detection result of an imaging area; and

[0023] A procedure of subject recognition processing on a captured image showing the imaging area is performed by using the recognizer selected based on the selection determination information.

[0024] Note that the program of the present technology is a program that can be provided to a general-purpose computer that can execute various program codes through, for example, a storage medium or communication medium (for example, a storage medium such as an optical disk, a magnetic disk, a semiconductor memory, or a communication medium such as a network) provided in a computer-readable format. By providing such a program in a computer-readable format, processing corresponding to the program is implemented on a computer.

[0025] Effects of the Invention

[0026] According to the present technology, selection determination information is generated based on the light amount detection result of the imaging area, and the subject recognition processing is performed on the captured image showing the imaging area using the identifier selected based on the selection determination information. Therefore, the subject included in the captured image can be easily and accurately recognized. Note that the effects described in this specification are merely examples and not limitations, and there may be additional effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a diagram showing the configuration of an identification device.

[0028] Figure 2 is a diagram showing the configuration of the first embodiment.

[0029] Figure 3 is a flowchart showing the operation of the first embodiment.

[0030] Figure 4 is a graph showing the relationship between the amount of light and the S / N ratio.

[0031] Figure 5 This is a diagram for explaining region segmentation.

[0032] Figure 6 is a diagram showing a conversion function for grayscale conversion.

[0033] Figure 7 is a diagram showing an operation example of the first embodiment.

[0034] Figure 8 is a diagram showing the configuration of the second embodiment.

[0035] Fig. 9 is a diagram showing the configuration of the third embodiment.

[0036] Fig.10 is a diagram showing the configuration of a fourth embodiment.

[0037] Fig.11is a block diagram showing an example of a schematic configuration of a vehicle control system.

[0038] Fig.12 : is an explanatory diagram showing an example of the installation positions of the vehicle exterior information detector and the imaging unit. DETAILED DESCRIPTION

[0039] Hereinafter, embodiments for implementing the present technology will be described. Note that the description will be given in the following order.

[0040] 1. Configuration of identification device

[0041] 2. First embodiment

[0042] 3. Second embodiment

[0043] 4. Third embodiment

[0044] 5. Fourth embodiment

[0045] 6. Application Examples

[0046] <1. Configuration of identification device>

[0047] Figure 1 The recognition device 10 includes a camera unit 20, a camera control unit 30, a recognition control unit 40, and a recognition processing unit 50. Note that the camera unit 20 and the camera control unit 30 may be provided separately from the recognition device 10.

[0048] The camera unit 20 generates an image signal showing a captured image of a desired imaging area, and outputs it to the recognition processing unit 50. The camera control unit 30 outputs a control signal to the camera unit 20, and controls the shutter speed, aperture, analog gain, grayscale conversion, etc. of the camera unit 20. In addition, the camera control unit 30 outputs a control signal to the recognition control unit 40.

[0049] The recognition control unit 40 detects the light amount of the imaging area based on the control signal from the camera control unit 30 and the image signal generated by the camera unit 20. In addition, the recognition device 10 may be provided with an illuminance detection unit 60, and the recognition control unit 40 may detect the light amount of the imaging area based on the illuminance detection signal from the illuminance detection unit 60. The recognition control unit 40 generates selection determination information based on the detected light amount, and outputs it to the recognition processing section 50. For example, the recognition control unit 40 performs noise estimation based on the light amount detection result, and uses the noise estimation result as the selection determination information.

[0050] The recognition processing unit 50 performs subject recognition processing on the captured image acquired by the camera unit 20 by using the recognizer selected based on the selection determination information.

[0051] <2. First embodiment>

[0052] Next, a first embodiment of the identification device will be described. Figure 2 The configuration of a first embodiment of the identification device is shown.

[0053] The camera unit 20 includes an optical system block 21 , an imaging unit 22 , a gain adjustment unit 23 , an analog / digital (A / D) conversion unit 24 , and a grayscale conversion unit 25 .

[0054] The optical system block 21 has an optical lens 211 and an aperture mechanism 212. The optical lens 211 forms an optical image of a subject on an imaging surface of an imaging element of the imaging unit 22. In addition, the aperture mechanism 212 adjusts the light amount of the optical image of the subject formed on the imaging surface. The optical system block 21 performs a focusing operation and a zooming operation by moving the optical lens 211 based on a control signal supplied from the camera control unit 30. In addition, the optical system block 21 drives the aperture mechanism 212 based on the control signal to adjust the light amount of the optical image of the subject.

[0055] The imaging unit 22 is configured to use an imaging element such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD). The imaging unit 22 performs photoelectric conversion to generate an image signal corresponding to the optical image of the subject, and outputs it to the gain adjustment unit 23. In addition, the imaging unit 22 controls the exposure period of the imaging element based on the control signal provided from the camera control unit 30, and performs imaging at a desired shutter speed. Note that the exposure period in the imaging operation can be changed by providing a shutter mechanism in the optical system block 21 and driving the shutter mechanism based on the control signal from the camera control unit 30.

[0056] The gain adjustment unit 23 performs gain adjustment (also referred to as analog gain adjustment) on the analog image signal generated by the imaging unit 22, and outputs it to the A / D conversion unit 24. The A / D conversion unit 24 converts the image signal after the analog gain adjustment from an analog signal to a digital signal, and outputs it to the grayscale conversion unit 25.

[0057] The gradation conversion unit 25 compresses the illuminance gradation of the image signal supplied from the A / D conversion unit 24 by using a conversion function based on the control signal. The gradation conversion unit 25 outputs the gradation-converted image signal to the recognition control unit 40 and the recognition processing unit 50.

[0058] The camera control unit 30 generates and supplies a control signal to the camera unit 20 to control the operation of each unit in the camera unit 20. In addition, the camera control unit 30 outputs the control signal to the recognition control unit 40.

[0059] The recognition control unit 40 has a light amount detection unit 41 and a noise estimation unit 42. The light amount detection unit 41 detects the light amount of the imaging area based on the image signal supplied from the camera unit 20 and the control signal supplied from the camera control unit 30. The noise estimation unit 42 estimates noise based on the light amount detection result of the light amount detection unit 41, and generates noise information indicating the estimation result. The noise estimation unit 42 outputs the generated noise information to the recognition processing unit 50 as selection determination information.

[0060] The recognition processing unit 50 includes a recognizer selection unit 51 and a plurality of recognizers 52-1 to 52-n. For example, the recognition processing unit 50 includes: a recognizer capable of recognizing a subject with high accuracy from a captured image with less noise by performing machine learning, etc. using a learning image with less noise, and a recognizer capable of recognizing a subject with high accuracy from a captured image with more noise by performing machine learning, etc. using a learning image with less noise. The recognizer selection unit 51 performs region segmentation processing of an imaging region based on selection determination information from the recognition control unit 40, selects a recognizer for each segmented region, and performs subject recognition based on an image signal provided from the camera unit 20 using the selected recognizer 52-1 (52-n).

[0061] Next, the operation of the first embodiment of the identification device will be described. Figure 3 1 is a flowchart showing the operation of the first embodiment. In step ST1, the recognition device acquires a captured image. The camera unit 20 performs analog gain adjustment of the image signal generated by the imaging element, A / D conversion of the image signal after the analog gain adjustment from an analog signal to a digital signal, grayscale conversion of the image signal after the A / D conversion, etc., to generate an image signal. The recognition processing unit 50 of the recognition device 10 acquires the image signal of the captured image generated by the camera unit 20, and the process proceeds to step ST2.

[0062] In step ST2, the recognition device acquires information related to the amount of light. The information related to the amount of light is information about parameters that cause changes corresponding to the amount of light in the imaging area. For example, the aperture value, shutter speed, analog gain adjustment value, conversion function f(u, v, l(u, v)) for grayscale conversion are parameters to be adjusted according to the amount of light in the imaging area, and control signals related to these parameters can be used as information related to the amount of light. Note that in the conversion function f(u, v, l(u, v)), the parameters u, v represent the position in the captured image, and the parameter l(u, v) represents the illuminance at the position (u, v). Since the signal level changes according to the amount of light, the image signal generated by the camera unit 20 can be used as information related to the amount of light. Therefore, the recognition control unit 40 of the recognition device 10 acquires the control signal output from the camera control unit 30 and the image signal generated by the camera unit 20 as information related to the amount of light, and the processing proceeds to step ST3.

[0063] In step ST3, the recognition device detects the light amount. The recognition control unit 40 detects the light amount of the imaging area based on the light amount related information output in step ST2.

[0064] Here, the light amount detection processing will be described. Based on the light amount L(u, v) of the subject at the position (u, v) in the captured image, the aperture information F about the aperture state, the shutter speed information S about the exposure time, and the gain information G about the analog gain adjustment, the signal value I(u, v) of the image signal before the grayscale conversion (after the A / D conversion) at the position (u, v) in the captured image satisfies the relationship of equation (1).

[0065] I(u,v)=L(u,v)=S*F*G......(1)

[0066] In addition, by inputting the signal value I(u, v) of the image signal before grayscale conversion at the position (u, v) in the captured image into the conversion function of the grayscale conversion, as shown in equation (2), the signal value J(u, v) of the image signal at the position (u, v) after grayscale conversion is obtained.

[0067] J(u, v)=f(u, v, I(u, v))......(2)

[0068] Therefore, the light amount L′(u, v) calculated based on equation (3) represents the light amount of the object at the position (u, v).

[0069] L'(u, v) = f -1 (u, v, J(u, v)) / (S*F*G)......(3)

[0070] In this way, the recognition control unit 40 detects the light amount of the imaging area, and the process proceeds to step ST4 .

[0071] In step ST4 , the recognition device generates selection determination information based on the detection result of the light amount. The recognition control unit 40 of the recognition device 10 prestores characteristic information indicating the relationship between the light amount and S / N, for example, and estimates noise in the light amount detected based on the light amount related information. Figure 4 The recognition control unit 40 sets the S / N ratio corresponding to the light amount detected based on the noise information, ie, the light amount related information, as the selection determination information, and the process proceeds to step ST5.

[0072] In step ST5, the recognition device performs region segmentation processing. The recognition processing unit 50 of the recognition device 10 segments the imaging region into, for example, a region with more noise and a region with less noise based on the selection determination information generated in step ST4. Figure 5 : is a diagram for explaining region segmentation. The recognition processing unit 50 compares the selection determination information (S / N ratio) with the threshold value THsn, and segments the region into a region with more noise when the selection determination information (S / N ratio) is equal to or less than the threshold value THsn, and segments the region into a region with less noise when the selection determination information (S / N ratio) is greater than the threshold value THsn, and the processing proceeds to step ST6.

[0073] In step ST6, the recognition device selects a recognizer for each region. The recognition processing unit 50 of the recognition device 10 selects a recognizer to be used in the recognition process for each region segmented in step ST5. The recognition processing unit 50 selects a recognizer capable of recognizing a subject with high accuracy from a captured image with less noise in a region with less noise, and selects a recognizer capable of recognizing a subject with high accuracy from a captured image with more noise in a region with more noise, and the process proceeds to step ST7.

[0074] In step ST7, the recognition device performs recognition processing. The recognition processing unit 50 of the recognition device 10 performs subject recognition for each area by using the recognizer selected in step ST6, and the process ends.

[0075] In addition, in the case where the light amount related information acquired in step ST2 includes information about region segmentation, light amount detection and noise estimation may be performed for each region in steps ST3 and ST4. For example, there is a case where the grayscale conversion function is switched for each region because overexposure or underexposure will occur when grayscale conversion is performed uniformly over the entire screen. Figure 6is a diagram showing a conversion function for grayscale conversion. Figure 6 (a) shows the grayscale conversion characteristics used when underexposure occurs, and grayscale conversion is performed in a case where the signal level before conversion is small so as to increase the signal level change. In addition, Figure 6 (b) shows the grayscale conversion characteristics used when overexposure or underexposure does not occur. In this way, when the conversion function is switched in the grayscale conversion, the recognition control unit 40 can perform area division of the imaging area according to the grayscale conversion switching operation of the imaging unit that obtains the captured image, and can generate selection determination information for each divided area.

[0076] Figure 7 An operation example of the first embodiment is shown. The recognition control unit 40 detects the amount of light in the imaging area based on the light amount related information, and generates selection determination information based on the light amount detection result. Here, in a case where the selection determination information indicates that the area ARa is an area with more noise and the area ARb is an area with less noise, the recognition processing unit 50 selects an identifier capable of identifying the subject with high accuracy from the captured image with more noise in the area ARa. In addition, the recognition processing unit 50 selects an identifier capable of identifying the subject with high accuracy from the captured image with less noise in the area ARb. In addition, the recognition processing unit 50 uses the selected identifier to perform subject recognition. Therefore, in the case of performing subject recognition by sequentially moving the recognition frame TA showing the recognition area in the direction of the arrow, for example, a subject in the area ARa with a small amount of light and a subject in the area ARb with a large amount of light can be easily and accurately identified.

[0077] In this way, according to the first embodiment, the amount of light in the imaging area is detected based on the information related to the amount of light, the identifier is selected based on the selection determination information generated based on the detected amount of light, and the subject recognition processing is performed on the captured image of the imaging area. For example, in an area with more noise in the captured image, the subject recognition is performed using an identifier that can recognize the subject with high accuracy from the captured image with more noise. In addition, in an area with less noise in the captured image, the subject recognition is performed using an identifier that can recognize the subject with high accuracy from the captured image with less noise. That is, in order to obtain a captured image with less noise, the optimal identifier can be automatically selected to perform the recognition processing without performing the processing of increasing the amount of light by increasing the exposure time or widening the aperture. In addition, since it is not necessary to store the image signal in the memory to perform the noise estimation processing to estimate the noise, the required memory capacity and processing amount will not increase. Therefore, the subject included in the captured image can be easily and accurately identified.

[0078] <3. Second Embodiment>

[0079] Next, a second embodiment will be described. In the second embodiment, a control signal at a stage before gradation conversion and an image signal before gradation conversion is performed are used as light amount related information to detect the light amount without being affected by gradation conversion.

[0080] Figure 8 The configuration of the second embodiment of the recognition device is shown. The camera unit 20 includes an optical system block 21 , an imaging unit 22 , a gain adjustment unit 23 , an analog / digital (A / D) conversion unit 24 , and a grayscale conversion unit 25 .

[0081] The optical system block 21 has an optical lens 211 and an aperture mechanism 212. The optical system block 21 performs a focusing operation and a zooming operation by moving the optical lens 211 based on a control signal supplied from the camera control unit 30. Furthermore, the optical system block 21 drives the aperture mechanism 212 based on the control signal to adjust the light amount of the subject optical image.

[0082] The imaging unit 22 generates an image signal corresponding to the subject optical image, and outputs it to the gain adjustment unit 23. In addition, the imaging unit 22 controls the exposure period of the imaging element based on the control signal supplied from the camera control unit 30, and performs imaging at a desired shutter speed. Note that the exposure period in the imaging operation can be changed by driving the shutter mechanism provided in the optical system block 21.

[0083] The gain adjustment unit 23 performs analog gain adjustment on the image signal generated by the imaging unit 22 and outputs it to the A / D conversion unit 24. The A / D conversion unit 24 converts the image signal after analog gain adjustment from an analog signal to a digital signal and outputs it to the grayscale conversion unit 25 and the recognition control unit 40.

[0084] The gradation conversion unit 25 compresses the illuminance gradation of the image signal supplied from the A / D conversion unit 24 by using a conversion function based on the control signal. The gradation conversion unit 25 outputs the gradation-converted image signal to the recognition processing unit 50.

[0085] The camera control unit 30 generates and supplies a control signal to the camera unit 20 to control the operation of each unit in the camera unit 20. In addition, the camera control unit 30 outputs the control signal to the recognition control unit 40.

[0086] The recognition control unit 40 has a light amount detection unit 41 and a noise estimation unit 42. The light amount detection unit 41 detects the light amount of the imaging area based on the image signal before grayscale conversion supplied from the camera unit 20 and the control signal related to the process at the stage before grayscale conversion supplied from the camera control unit 30.

[0087] The noise estimation unit 42 estimates noise based on the light amount detection result of the light amount detection unit 41, and generates noise information indicating the estimation result. The noise estimation unit 42 outputs the generated noise information to the recognition processing unit 50 as selection determination information.

[0088] The recognition processing unit 50 includes a recognizer selection unit 51 and a plurality of recognizers 52-1 to 52-n. For example, the recognition processing unit 50 has a recognizer capable of recognizing a subject with high accuracy from a captured image with less noise, and a recognizer capable of recognizing a subject with high accuracy from a captured image with more noise. The recognizer selection unit 51 performs area segmentation processing of an imaging area based on selection determination information from the recognition control unit 40, selects a recognizer for each segmented area, and performs subject recognition based on an image signal provided from the camera unit 20 using the selected recognizer 52-1 (52-n).

[0089] In operation of the second embodiment of the identification device, the Figure 3 The operation of the first embodiment shown is similar to the processing, and the processing contents of steps ST2 and ST3 are different from those of the first embodiment.

[0090] In step ST2, the recognition device acquires light amount related information. As the light amount related information, for example, aperture value information F, shutter speed information S, and gain information G are used. In addition, as the light amount related information, an image signal before grayscale conversion is used, in which the signal level changes according to the light amount of the imaging area and in which the information is not degraded by grayscale conversion. The recognition control unit 40 of the recognition device 10 acquires light amount related information in this way, and the process proceeds to step ST3.

[0091] In step ST3, the recognition device detects the light amount. The recognition control unit 40 detects the light amount of the imaging area based on the light amount related information output in step ST2.

[0092] Here, the light quantity detection processing will be described. Assuming that the subject at the position (u, v) in the captured image is the light quantity L(u, v), and the image signal before the grayscale conversion at the position (u, v) is the signal value I(u, v), then the relationship of the above equation (1) is satisfied.

[0093] Therefore, the light amount L'(u, v) calculated based on equation (4) represents the light amount of the object at the position (u, v).

[0094] L'(u, v)=I(u, v)) / (S*F*G)......(4)

[0095] In this way, the recognition control unit 40 detects the light amount of the imaging area. Thereafter, similarly to the first embodiment, noise estimation is performed based on the light amount detection result, and noise information indicating the estimation result is used as selection determination information. In addition, the recognition processing unit 50 performs region segmentation processing based on the selection determination information, and selects a recognizer for each region to perform subject recognition.

[0096] As described above, according to the second embodiment, even when the amount of information deteriorates due to gradation conversion, subject recognition can be performed without being affected by the degradation of the amount of information. In addition, in the second embodiment, similarly to the first embodiment, a subject included in a captured image can be easily and accurately recognized.

[0097] <4. Third embodiment>

[0098] Next, a third embodiment will be described. In the third embodiment, the light amount is detected using a control signal at a stage before analog gain adjustment and an image signal before analog gain adjustment is performed as light amount-related information.

[0099] Fig. 9 The configuration of the third embodiment of the recognition device is shown. The camera unit 20 includes an optical system block 21 , an imaging unit 22 , a gain adjustment unit 23 , an analog / digital (A / D) conversion unit 24 , and a grayscale conversion unit 25 .

[0100] The optical system block 21 has an optical lens 211 and an aperture mechanism 212. The optical system block 21 performs a focusing operation and a zooming operation by moving the optical lens 211 based on a control signal supplied from the camera control unit 30. Furthermore, the optical system block 21 drives the aperture mechanism 212 based on the control signal to adjust the light amount of the subject optical image.

[0101] The imaging unit 22 generates an image signal corresponding to the subject optical image, and outputs it to the gain adjustment unit 23 and the recognition control unit 40. In addition, the imaging unit 22 controls the exposure period of the imaging element based on the control signal supplied from the camera control unit 30, and performs imaging at a desired shutter speed. Note that the exposure period in the imaging operation can be changed by driving the shutter mechanism provided in the optical system block 21.

[0102] The gain adjustment unit 23 performs analog gain adjustment on the image signal generated by the imaging unit 22, and outputs to the A / D conversion unit 24. The A / D conversion unit 24 converts the image signal after analog gain adjustment from an analog signal to a digital signal, and outputs to the grayscale conversion unit 25.

[0103] The gradation conversion unit 25 compresses the illuminance gradation of the image signal supplied from the A / D conversion unit 24 by using a conversion function based on the control signal. The gradation conversion unit 25 outputs the gradation-converted image signal to the recognition processing unit 50.

[0104] The camera control unit 30 generates and supplies a control signal to the camera unit 20 to control the operation of each unit in the camera unit 20. In addition, the camera control unit 30 outputs the control signal to the recognition control unit 40.

[0105] The recognition control unit 40 has a light amount detection unit 41 and a noise estimation unit 42. The light amount detection unit 41 detects the light amount of the imaging area based on the image signal before analog gain adjustment supplied from the camera unit 20 and the control signal related to the processing in the stage before analog gain adjustment supplied from the camera control unit 30.

[0106] The noise estimation unit 42 estimates noise based on the light amount detection result of the light amount detection unit 41, and generates noise information indicating the estimation result. The noise estimation unit 42 outputs the generated noise information to the recognition processing unit 50 as selection determination information.

[0107] The recognition processing unit 50 includes a recognizer selection unit 51 and a plurality of recognizers 52-1 to 52-n. For example, the recognition processing unit 50 has a recognizer capable of recognizing a subject with high accuracy from a captured image with less noise, and a recognizer capable of recognizing a subject with high accuracy from a captured image with more noise. The recognizer selection unit 51 performs area segmentation processing of an imaging area based on selection determination information from the recognition control unit 40, selects a recognizer for each segmented area, and performs subject recognition based on an image signal provided from the camera unit 20 using the selected recognizer 52-1 (52-n).

[0108] In the operation of the third embodiment of the identification device, the Figure 3 The operation of the first embodiment shown is similar to the processing, and the processing contents of steps ST2 and ST3 are different from those of the first embodiment.

[0109] In step ST2, the recognition device acquires light amount related information. Here, as the light amount related information, for example, aperture value information F and shutter speed information S are used as light amount related information. In addition, as the light amount related information, an image signal before analog gain adjustment in which the signal level changes according to the light amount of the imaging area is used. The recognition control unit 40 of the recognition device 10 acquires light amount related information in this way, and the process proceeds to step ST3.

[0110] In step ST3, the recognition device detects the light amount. The recognition control unit 40 detects the light amount of the imaging area based on the light amount related information output in step ST2.

[0111] Here, the light quantity detection processing will be described. Assuming that the subject at the position (u, v) in the captured image is the light quantity L(u, v), and the image signal before the analog gain adjustment at the position (u, v) is the signal value A(u, v), then the relationship of equation (5) is satisfied.

[0112] A(u, v)=L(u, v)=S*F......(5)

[0113] Therefore, the light amount L′(u, v) calculated based on equation (6) represents the light amount of the object at the position (u, v).

[0114] L'(u, v)=A(u, v)) / (S*F)......(6)

[0115] In this way, the recognition control unit 40 detects the light amount of the imaging area. Thereafter, similarly to the first embodiment, noise estimation is performed based on the light amount detection result, and noise information indicating the estimation result is used as selection determination information. In addition, the recognition processing unit 50 performs region segmentation processing based on the selection determination information, and selects a recognizer for each region to perform subject recognition.

[0116] In this way, according to the third embodiment, even when the amount of information is degraded due to grayscale conversion or analog gain adjustment, subject recognition can be performed without being affected by the deterioration of the amount of information. In addition, in the third embodiment, similar to the first embodiment, the subject included in the captured image can be easily and accurately identified.

[0117] <5. Fourth embodiment>

[0118] In the above-mentioned first to third embodiments, the case where the light amount is detected using the control signal generated by the camera control unit 30 has been exemplified, but depending on the configuration of the recognition device, the control signal may not be acquired. For example, in the case where the camera unit 20 and the camera control unit 30 are provided separately from the recognition device and the control signal cannot be communicated between the camera control unit 30 and the recognition control unit 40, the control signal cannot be used as the light amount related information. Therefore, in the fourth embodiment, a case where the light amount of the imaging area is detected without using a control signal will be described.

[0119] Fig.10 The configuration of the fourth embodiment of the recognition device is shown. The camera unit 20 includes an optical system block 21 , an imaging unit 22 , a gain adjustment unit 23 , an analog / digital (A / D) conversion unit 24 , and a grayscale conversion unit 25 .

[0120] The optical system block 21 has an optical lens 211 and an aperture mechanism 212. The optical system block 21 performs a focusing operation and a zooming operation by moving the optical lens 211 based on a control signal supplied from the camera control unit 30. Furthermore, the optical system block 21 drives the aperture mechanism 212 based on the control signal to adjust the light amount of the subject optical image.

[0121] The imaging unit 22 generates an image signal corresponding to the subject optical image, and outputs it to the gain adjustment unit 23. In addition, the imaging unit 22 controls the exposure period of the imaging element based on the control signal supplied from the camera control unit 30, and performs imaging at a desired shutter speed. Note that the exposure period in the imaging operation can be changed by driving the shutter mechanism provided in the optical system block 21.

[0122] The gain adjustment unit 23 performs analog gain adjustment on the image signal generated by the imaging unit 22, and outputs it to the A / D conversion unit 24. The A / D conversion unit 24 converts the image signal after the analog gain adjustment from an analog signal to a digital signal, and outputs it to the grayscale conversion unit 25.

[0123] The gradation conversion unit 25 compresses the illuminance gradation of the image signal supplied from the A / D conversion unit 24 by using a conversion function based on the control signal. The gradation conversion unit 25 outputs the gradation-converted image signal to the recognition control unit 40 and the recognition processing unit 50.

[0124] The camera control unit 30 generates and supplies a control signal to the camera unit 20 to control the operation of each unit in the camera unit 20 .

[0125] The illuminance detection unit 60 generates imaging environment illuminance information indicating the brightness of the imaging area, and outputs the information to the recognition control unit 40 .

[0126] The recognition control unit 40 has a light amount detection unit 41 and a noise estimation unit 42. The light amount detection unit 41 detects the light amount of the imaging area based on the grayscale-converted image signal supplied from the camera unit 20 and the imaging environment illuminance information supplied from the illuminance detection unit 60.

[0127] The noise estimation unit 42 estimates noise based on the light amount detection result of the light amount detection unit 41, and generates noise information indicating the estimation result. The noise estimation unit 42 outputs the generated noise information to the recognition processing unit 50 as selection determination information.

[0128] The recognition processing unit 50 includes a recognizer selection unit 51 and a plurality of recognizers 52-1 to 52-n. For example, the recognition processing unit 50 has a recognizer capable of recognizing a subject with high accuracy from a captured image with less noise, and a recognizer capable of recognizing a subject with high accuracy from a captured image with more noise. The recognizer selection unit 51 performs area segmentation processing of an imaging area based on selection determination information from the recognition control unit 40, selects a recognizer for each segmented area, and performs subject recognition based on an image signal provided from the camera unit 20 using the selected recognizer 52-1 (52-n).

[0129] In the operation of the fourth embodiment of the identification device, the following is performed: Figure 3 The operation of the first embodiment shown is similar to the processing, and the processing contents of step ST2 and step ST3 are different from those of the first embodiment.

[0130] In step ST2, the recognition device acquires light quantity related information. Here, as the light quantity related information, for example, the imaging environment illumination β is used as the light quantity related information. In addition, the image signal after grayscale conversion whose signal level changes according to the light quantity of the imaging area is used as the light quantity related information. The recognition control unit 40 of the recognition device 10 acquires the light quantity related information in this way, and the processing proceeds to step ST3.

[0131] In step ST3, the recognition device detects the light amount. The recognition control unit 40 detects the light amount of the imaging area based on the light amount related information output in step ST2.

[0132] Here, the light amount detection processing will be described. Assuming that the image signal after grayscale conversion of the position (u, v) in the captured image is the signal value J(u, v), when the imaging environment illumination is approximately equal to the image average illumination, the light amount L'(u, v) calculated based on equation (7) represents the light amount of the subject at the position (u, v).

[0133] L'(u, v)=J(u, v))*β......(7)

[0134] In this way, the recognition control unit 40 detects the light amount of the imaging area. Thereafter, similarly to the first embodiment, noise estimation is performed based on the light amount detection result, and noise information indicating the estimation result is used as selection determination information. In addition, the recognition processing unit 50 performs region segmentation processing based on the selection determination information, and selects a recognizer for each region to perform subject recognition.

[0135] In this way, according to the fourth embodiment, even in the case where a control signal cannot be acquired from the camera control unit 30, the light amount of the imaging area can be detected based on the imaging environment illuminance detected by the illuminance detection unit 60 and the image signal output from the camera unit 20. In addition, similarly to the above-mentioned first embodiment and the like, an identifier can be selected based on the selection determination information generated based on the detected light amount, and a subject recognition process can be performed on a captured image of the imaging area. In addition, in the fourth embodiment, similarly to the first embodiment, a subject included in a captured image can be easily and accurately recognized.

[0136] <6. Application Examples>

[0137] The technology according to the present disclosure can be applied to various products. For example, the technology according to the present disclosure can be implemented as a device equipped on any type of mobile body such as a car, an electric car, a hybrid electric car, a motorcycle, a bicycle, a personal mobile device, an airplane, a drone, a ship, a robot, a construction machine, or an agricultural machine (tractor).

[0138] Fig.11 7000 is a block diagram showing a schematic configuration example of a vehicle control system 7000, which is an example of a mobile control system to which the technology according to the present disclosure can be applied. The vehicle control system 7000 includes a plurality of electronic control units connected via a communication network 7010. Fig.11 In the example shown, the vehicle control system 7000 includes a drive system control unit 7100, a body system control unit 7200, a battery control unit 7300, an external information detection unit 7400, an internal information detection unit 7500, and an integrated control unit 7600. The communication network 7010 connecting these multiple control units may be, for example, an in-vehicle communication network conforming to any standard such as a controller area network (CAN), a local interconnect network (LIN), a local area network (LAN), or FlexRay (registered trademark).

[0139] Each control unit includes: a microcomputer configured to perform arithmetic processing according to various programs; a storage unit storing programs executed by the microcomputer, parameters used in various operations, etc.; and a drive circuit configured to drive various devices to be controlled. Each control unit includes a network I / F for communicating with other control units via the communication network 7010, and a communication I / F for communicating with devices, sensors, etc. inside and outside the vehicle through wired or wireless communication. Fig.11A microcomputer 7610, a general communication I / F 7620, a dedicated communication I / F 7630, a positioning unit 7640, a beacon receiving unit 7650, an in-vehicle equipment I / F 7660, a sound / image output unit 7670, an in-vehicle network I / F 7680, and a storage unit 7690 are shown as a functional configuration of the integrated control unit 7600. Other control units similarly include a microcomputer, a communication I / F, a storage unit, and the like.

[0140] The drive system control unit 7100 controls the actions of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 7100 acts as: a drive force generating device for generating the drive force of the vehicle, such as an internal combustion engine or a drive motor; a drive force transmitting mechanism for transmitting the drive force to the wheels; a steering mechanism for adjusting the steering angle of the vehicle; and a control device for generating the braking force of the vehicle, such as a brake device. The drive system control unit 7100 may have a function as a control device such as an anti-lock braking system (ABS) or an electronic stability control (ESC).

[0141] The drive system control unit 7100 is connected to the vehicle state detection unit 7110. The vehicle state detection unit 7110 includes, for example, at least one of a gyro sensor for detecting the angular velocity of the axial rotational motion of the vehicle body, an acceleration sensor for detecting the acceleration of the vehicle, or a sensor for detecting an accelerator pedal operation amount, a brake pedal operation amount, a steering wheel steering angle, an engine speed, a wheel speed, etc. The drive system control unit 7100 performs arithmetic processing by using a signal input from the vehicle state detection unit 7110, and controls the internal combustion engine, the drive motor, the electric power steering device, the brake device, etc.

[0142] The body system control unit 7200 controls the operation of various devices installed on the vehicle body according to various programs. For example, the body system control unit 7200 acts as a control device for a keyless entry system, a smart key system, a power window device, or various lights such as headlights, taillights, brake lights, turn indicators, or fog lights. In this case, radio waves sent from a portable device that replaces the key or signals of various switches may be input to the body system control unit 7200. The body system control unit 7200 receives input of these radio waves or signals and controls the door lock device, power window device, lights, etc. of the vehicle.

[0143] The battery control unit 7300 controls the secondary battery 7310 as a power source for driving the motor according to various programs. For example, information such as battery temperature, battery output voltage, or remaining capacity of the battery is input to the battery control unit 7300 from a battery device including the secondary battery 7310. The battery control unit 7300 performs arithmetic processing by using these signals, and performs temperature control of the secondary battery 7310 or control of a cooling device provided in the battery device, etc.

[0144] The vehicle exterior information detection unit 7400 detects information related to the exterior of the vehicle equipped with the vehicle control system 7000. For example, the vehicle exterior information detection unit 7400 is connected to at least one of the imaging unit 7410 or the vehicle exterior information detector 7420. The imaging unit 7410 includes at least one of a time of flight (ToF) camera, a stereo camera, a monocular camera, an infrared camera, or other cameras. The vehicle exterior information detector 7420 includes, for example, an environmental sensor for detecting current weather or meteorological phenomena, or at least one of surrounding information detection sensors for detecting other vehicles, obstacles, pedestrians, etc. around the vehicle equipped with the vehicle control system 7000.

[0145] The environmental sensor may be, for example, at least one of a raindrop sensor for detecting rainy weather, a fog sensor for detecting fog, a sunshine sensor for detecting sunshine degree, or a snow sensor for detecting snowfall. The surrounding information detection sensor may be at least one of an ultrasonic sensor, a radar device, or a light detection and ranging or laser imaging detection and ranging (LIDAR) device. The imaging unit 7410 and the vehicle exterior information detector 7420 may be separately provided as independent sensors or devices, or may be provided as a device in which a plurality of sensors or devices are integrated.

[0146] Here, Fig.12 An example of the installation position of the imaging unit 7410 and the vehicle exterior information detection unit 7420 is shown. The imaging units 7910, 7912, 7914, 7916, and 7918 are arranged at at least one position such as the front nose, side mirror, rear bumper, rear door, or upper portion of the windshield in the vehicle compartment of the vehicle 7900. The imaging unit 7910 arranged at the front nose and the imaging unit 7918 arranged at the upper portion of the windshield in the vehicle compartment mainly acquire images in front of the vehicle 7900. The imaging units 7912 and 7914 arranged at the side mirror mainly acquire images of the side of the vehicle 7900. The imaging unit 7916 arranged at the rear bumper or rear door mainly acquires images behind the vehicle 7900. The imaging unit 7918 arranged at the upper portion of the windshield in the vehicle compartment is mainly used for the detection of the preceding vehicle, pedestrians, obstacles, signal lights, traffic signs, lanes, etc.

[0147] Notice, Fig.12An example of the imaging range of each of the imaging units 7910, 7912, 7914, and 7916 is shown. Imaging range A represents the imaging range of the imaging unit 7910 provided at the front nose, imaging ranges B and C represent the imaging ranges of the imaging units 7912 and 7914 provided at the side mirrors, respectively, and imaging range D represents the imaging range of the imaging unit 7916 provided at the rear bumper or the rear door. For example, by overlapping the image data captured by the imaging units 7910, 7912, 7914, 7916, it is possible to obtain a bird's-eye view image of the vehicle 7900 observed from above.

[0148] The vehicle exterior information detection units 7920, 7922, 7924, 7926, 7928, 7930 provided at the front, rear, side, corners, and upper portion of the windshield in the vehicle 7900 may be, for example, ultrasonic sensors or radar devices. The vehicle exterior information detection units 7920, 7926, 7930 provided at the front nose, rear bumper, rear door, and upper portion of the windshield in the vehicle 7900 may be, for example, LIDAR devices. These vehicle exterior information detection units 7920 to 7930 are mainly used for detecting preceding vehicles, pedestrians, obstacles, and the like.

[0149] return Fig.11 , will continue to describe. The vehicle exterior information detection unit 7400 enables the imaging unit 7410 to capture an image of the outside of the vehicle and receive the captured image data. In addition, the vehicle exterior information detection unit 7400 receives detection information from the connected vehicle exterior information detector 7420. In the case where the vehicle exterior information detection unit 7420 is an ultrasonic sensor, a radar device, or a LIDAR device, the vehicle exterior information detection unit 7400 enables ultrasonic waves, electromagnetic waves, etc. to be emitted, and receives information of the received reflected waves. The vehicle exterior information detection unit 7400 can perform object detection processing or distance detection processing for people, vehicles, obstacles, signs, text on the road surface, etc. based on the received information. The vehicle exterior information detection unit 7400 can perform environmental recognition processing for identifying rainfall, fog, road conditions, etc. based on the received information. The vehicle exterior information detection unit 7400 can calculate the distance to objects outside the vehicle based on the received information.

[0150] In addition, the vehicle exterior information detection unit 7400 can perform image recognition processing or distance detection processing for identifying people, vehicles, obstacles, signs, text on the road surface, etc. based on the received image data. The vehicle exterior information detection unit 7400 can perform processing such as distortion correction or position adjustment on the received image data, and can synthesize the image data captured by different imaging units 7410 to generate a bird's-eye view image or a panoramic image. The vehicle exterior information detection unit 7400 can use the image data captured by different imaging units 7410 to perform viewpoint conversion processing.

[0151] The in-vehicle information detection unit 7500 detects information inside the vehicle. The in-vehicle information detection unit 7500 is connected to, for example, a driver state detection unit 7510 that detects the state of the driver. The driver state detection unit 7510 may include a camera that captures an image of the driver, a biosensor that detects the driver's biometric information, a microphone that collects sounds in the vehicle compartment, and the like. The biosensor is, for example, disposed on a seat surface, a steering wheel, and the like, and detects biometric information of a passenger sitting on a seat or a driver holding the steering wheel. The in-vehicle information detection unit 7500 may calculate the driver's fatigue or concentration, or determine whether the driver is asleep, based on the detection information input from the driver state detection unit 7510. The in-vehicle information detection unit 7500 may perform processing such as noise removal processing on the collected audio signal.

[0152] The integrated control unit 7600 controls the overall operation in the vehicle control system 7000 according to various programs. The integrated control unit 7600 is connected to the input unit 7800. The input unit 7800 is implemented by, for example, a device operable by a passenger, such as a touch panel, a button, a microphone, a switch, or a joystick. Data obtained by voice recognition of a sound input via a microphone can be input to the integrated control unit 7600. The input unit 7800 can be, for example, a remote control device using infrared or other radio waves, or can be an external connection device compatible with the operation of the vehicle control system 7000, such as a mobile phone or a personal digital assistant (PDA). The input unit 7800 can be, for example, a camera, in which case the passenger can input information by gestures. Alternatively, data obtained by detecting the movement of a wearable device worn by a passenger can be input. In addition, the input unit 7800 can include, for example, an input control circuit, etc., which is configured to generate an input signal based on information input by a passenger, etc. using the above-mentioned input unit 7800, and output it to the integrated control unit 7600. By operating the input unit 7800, a passenger or the like inputs various data to the vehicle control system 7000 and instructs processing operations.

[0153] The storage unit 7690 may include a read-only memory (ROM) storing various programs executed by the microcomputer and a random access memory (RAM) storing various parameters, operation results, sensor values, etc. In addition, the storage unit 7690 may be implemented by a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, etc.

[0154] The general communication I / F 7620 is a general communication I / F that acts as an intermediary for communicating with various equipment present in the external environment 7750. The general communication I / F 7620 can implement cellular communication protocols such as Global System for Mobile Communications (GSM (registered trademark)), WiMAX (registered trademark), Long Term Evolution (LTE (registered trademark)), or LTE-Advanced (LTE-A), or other wireless communication protocols such as wireless LAN (also known as Wi-Fi (registered trademark)) or Bluetooth (registered trademark). For example, via a base station or an access point, the general communication I / F 7620 can be connected to equipment (e.g., an application server or a control server) present on an external network (e.g., the Internet, a cloud network, or a network unique to an operator). In addition, for example, the general communication I / F 7620 can use peer-to-peer (P2P) technology to connect to a terminal present near the vehicle (e.g., a terminal of a driver, a pedestrian, or a store, or a machine type communication (MTC) terminal).

[0155] The dedicated communication I / F 7630 is a communication I / F that supports a communication protocol established for use in a vehicle. The dedicated communication I / F 7630 may implement a standard protocol, such as Wireless Access in Vehicular Environments (WAVE), Dedicated Short Range Communications (DSRC), or a cellular communication protocol that is a combination of the lower layer IEEE 802.11p and the upper layer IEEE 1609. The dedicated communication I / F 7630 typically performs V2X communication, which is a concept that includes one or more of vehicle-to-vehicle communication, vehicle-to-infrastructure communication, vehicle-to-home communication, and vehicle-to-pedestrian communication.

[0156] The positioning unit 7640 receives, for example, a global navigation satellite system (GNSS) signal from a GNSS satellite (e.g., a global positioning system (GPS) signal from a GPS satellite) to perform positioning, and generates position information including the latitude, longitude, and altitude of the vehicle. Note that the positioning unit 7640 may specify the current position by exchanging signals with a wireless access point, or may acquire position information from a terminal having a positioning function such as a mobile phone, a PHS, or a smart phone.

[0157] The beacon receiving unit 7650 receives radio waves or electromagnetic waves transmitted from, for example, a wireless station installed on a road, and acquires information such as the current position, traffic jam, traffic closure, required time, etc. Note that the function of the beacon receiving unit 7650 may be included in the above-mentioned dedicated communication I / F 7630.

[0158] The in-vehicle equipment I / F 7660 is a communication interface that mediates the connection between the microcomputer 7610 and various in-vehicle equipment 7760 present inside the vehicle. The in-vehicle equipment I / F 7660 can establish a wireless connection by using a wireless communication protocol such as wireless LAN, Bluetooth (registered trademark), near field communication (NFC), or wireless USB (WUSB). In addition, the in-vehicle equipment I / F 7660 can establish a wired connection with a universal serial bus (USB), a high-definition multimedia interface (HDMI, registered trademark), a mobile high-definition link (MHL), etc. via a connection terminal (not shown) (and if necessary, via a cable). The in-vehicle equipment 7760 may include, for example, at least one of a mobile device or wearable device owned by a passenger, or an information device carried in the vehicle or attached to the vehicle. In addition, the in-vehicle equipment 7760 may include a navigation device that searches for a route to any given destination. The in-vehicle equipment I / F 7660 exchanges control signals or data signals with these in-vehicle equipment 7760.

[0159] The in-vehicle network I / F 7680 is an interface that mediates communication between the microcomputer 7610 and the communication network 7010. The in-vehicle network I / F 7680 transmits and receives signals and the like according to a predetermined protocol supported by the communication network 7010.

[0160] The microcomputer 7610 of the integrated control unit 7600 controls the vehicle control system 7000 according to various programs based on information acquired via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle equipment I / F 7660, or the vehicle network I / F 7680. For example, the microcomputer 7610 can operate the control target value of the driving force generation device, the steering mechanism, or the braking device based on the acquired information about the inside and outside of the vehicle, and output a control command to the drive system control unit 7100. For example, the microcomputer 7610 can perform cooperative control for the purpose of realizing the functions of the advanced driving assistance system (ADAS) including collision avoidance or impact reduction of the vehicle, follow-up driving based on the distance between vehicles, speed maintenance driving, vehicle collision warning, vehicle lane departure warning, etc. In addition, the microcomputer 7610 can perform cooperative control for the purpose of automatic driving such as autonomous driving without relying on the driver by controlling the driving force generation device, the steering mechanism, the braking device, etc. based on the acquired information about the surroundings of the vehicle.

[0161] The microcomputer 7610 can generate 3D distance information between the vehicle and surrounding objects such as structures and people based on information acquired via at least one of the general communication I / F 7620, the dedicated communication I / F 7630, the positioning unit 7640, the beacon receiving unit 7650, the in-vehicle equipment I / F 7660, or the vehicle network I / F 7680, and create local map information including surrounding information of the current position of the vehicle. In addition, based on the acquired information, the microcomputer 7610 can predict dangers such as vehicle collision, pedestrians approaching or entering a closed road to generate a warning signal. The warning signal may be, for example, a signal for generating a warning sound or lighting a warning light.

[0162] The sound / image output unit 7670 transmits an output signal of at least one of sound or image to an output device capable of visually or audibly notifying information to a passenger of the vehicle or outside the vehicle. Fig.11 In the example of , an audio speaker 7710, a display unit 7720, and an instrument panel 7730 are shown as output devices. The display unit 7720 may include, for example, at least one of an on-board display or a head-up display. The display unit 7720 may have an augmented reality (AR) display function. The output device may be other devices other than these devices, such as headphones, wearable devices such as glasses-shaped displays worn by passengers, projectors, lights, etc. In the case where the output device is a display device, the display device visually displays the results obtained by various processes performed by the microcomputer 7610 or information received from other control units in various formats such as text, images, tables, and graphics. In addition, in the case where the output device is a sound output device, the sound output device converts an audio signal formed by reproduced sound data, acoustic data, etc. into an analog signal and outputs it audibly.

[0163] Note that Fig.11 In the example shown, at least two control units connected via the communication network 7010 can be integrated into one control unit. Alternatively, each control unit can be composed of multiple control units. In addition, the vehicle control system 7000 may include other control units not shown. In addition, in the above description, some or all of the functions performed by any one of the control units can be provided to other control units. In other words, as long as information is sent and received via the communication network 7010, a predetermined operation process can be performed by any control unit. Similarly, a sensor or device connected to any one of the control units can be connected to other control units, and multiple control units can send and receive detection information to each other via the communication network 7010.

[0164] In the above vehicle control system 7000, the present technology can be applied to Fig.11The vehicle exterior information detection unit 7400 shown. In this way, when the present technology is applied to the vehicle exterior information detection unit 7400, for example, it is possible to easily identify the subject outside the vehicle by reducing the influence of the brightness of the subject, and reduce the fatigue of the driver, etc. In addition, it is possible to obtain information required for autonomous driving, etc. with high accuracy.

[0165] Furthermore, when the technology according to the present disclosure is applied to a monitoring system, recognition processing can be accurately performed regardless of the brightness of the subject, and, for example, a person in a bright place or a person in a dark place can be accurately monitored.

[0166] The series of processing described in the specification can be performed by hardware, software, or a combination of the two. In the case of performing processing by software, a program in which the processing sequence is recorded is installed and executed in a memory included in the dedicated hardware of the computer. Alternatively, the program can be installed and executed on a general-purpose computer that can perform various processing.

[0167] For example, the program may be pre-recorded on a hard disk, solid state drive (SSD), or read-only memory (ROM) as a recording medium. Alternatively, the program may be temporarily or permanently stored (recorded) in a removable recording medium such as a floppy disk, a compact disk read-only memory (CD-ROM), a magneto-optical (MO) disk, a digital versatile disk (DVD), a Blu-ray (registered trademark) disk (BD), a magnetic disk, or a semiconductor memory card. Such a removable recording medium may be provided as a so-called software package.

[0168] In addition, in addition to being installed on a computer from a removable recording medium, the program can also be transmitted to the computer from a download site in a wired or wireless manner via a network such as a local area network (LAN) or the Internet. In the computer, the program transmitted in this manner can be received and installed on a recording medium such as a built-in hard disk.

[0169] Note that the effects described in this specification are merely examples and not limitations, and there may be additional effects that are not described. In addition, the present technology should not be interpreted as being limited to the embodiments of the above-mentioned technology. The embodiments of the present technology disclose the present technology in the form of examples, and it is obvious that those skilled in the art can modify or replace the embodiments within the gist of the present technology. In other words, in order to determine the gist of the present technology, the claims should be considered.

[0170] Furthermore, the recognition device of the present technology may also have the following configuration.

[0171] (1) An identification device comprising:

[0172] a recognition control unit configured to generate selection determination information based on a light amount detection result of the imaging area; and

[0173] A recognition processing unit is configured to perform subject recognition processing on a captured image showing the imaging area by using a recognizer selected based on the selection determination information generated by the recognition control unit.

[0174] (2) The identification device according to (1), wherein

[0175] The recognition control unit performs noise estimation based on the light amount detection result, and uses the noise estimation result as the selection determination information.

[0176] (3) The identification device according to (2), wherein

[0177] The recognition processing unit selects a first classifier if it is determined based on the selection determination information that the noise is equal to or less than a threshold value, and selects a second classifier more resistant to noise than the first classifier if it is determined based on the selection determination information that the noise is greater than the threshold value.

[0178] (4) The identification device according to any one of (1) to (3), wherein

[0179] The recognition processing unit performs region division on the captured image based on the selection determination information, and selects a recognizer for each divided region.

[0180] (5) The identification device according to any one of (1) to (3), wherein

[0181] The recognition control unit performs region division of the captured image according to a grayscale conversion switching operation of an imaging unit that acquires the captured image, and generates selection determination information for each of the divided regions.

[0182] (6) The identification device according to any one of (1) to (5), wherein

[0183] The recognition control unit detects a light amount of an imaging area based on a control signal for controlling an operation of an imaging unit that acquires a captured image and an image signal generated by the imaging unit.

[0184] (7) The identification device according to (6), wherein

[0185] The control signal is a signal for controlling at least any one of an aperture, a shutter speed, an analog gain adjustment, or a grayscale conversion.

[0186] (8) The identification device according to (7), wherein

[0187] The control signal is a signal for controlling aperture, shutter speed, analog gain adjustment and grayscale conversion, and

[0188] The image signal is a signal that has undergone analog gain adjustment and grayscale conversion.

[0189] (9) The identification device according to (7), wherein

[0190] The control signal is a signal for controlling aperture, shutter speed and analog gain adjustment, and

[0191] The image signal is a signal that has undergone analog gain adjustment.

[0192] (10) The identification device according to (7), wherein

[0193] The control signal is a signal for controlling the aperture and the shutter speed, and

[0194] The image signal is a signal before analog gain adjustment is performed.

[0195] (11) The identification device according to (1), wherein

[0196] The recognition control unit detects the amount of light based on the image signal generated by the imaging unit and the illuminance detected by the illuminance detection unit that detects the illuminance of the imaging environment.

[0197] Industrial Applicability

[0198] According to the recognition device, recognition method, and program of the present technology, selection determination information is generated based on the light amount detection result of the imaging area, and the subject recognition processing of the captured image showing the imaging area is performed using the identifier selected based on the selection determination information. Therefore, the subject included in the captured image can be easily and accurately recognized. Therefore, the present technology is suitable for a monitoring device that monitors the surrounding conditions, a mobile device that determines the surrounding conditions and performs movement control, and the like.

[0199] Reference numerals list

[0200] 10 Identification device

[0201] 20 Camera Unit

[0202] 21 Optical system block

[0203] 22 Imaging Unit

[0204] 23. Gain adjustment unit

[0205] 24 A / D conversion units

[0206] 25 Grayscale conversion units

[0207] 30 Camera control unit

[0208] 40 Identify control unit

[0209] 41 Light detection unit

[0210] 42 Noise estimation unit

[0211] 50 Record Processing Unit

[0212] 51 Identifier selection unit

[0213] 52-1 to 52-n Identifier

[0214] 60 Illumination detection unit

[0215] 211 Optical lens

[0216] 212 Aperture mechanism

Claims

1. An identification device, comprising: a recognition control unit configured to generate selection determination information based on a light amount detection result of the imaging area; as well as a recognition processing unit configured to perform subject recognition processing on a captured image showing the imaging area by using a recognizer selected based on the selection determination information generated by the recognition control unit, wherein the recognition control unit detects the light amount based on an image signal generated by an imaging unit and an illuminance detected by an illuminance detection unit that detects the illuminance of an environment of the imaging area, Here, the recognition processing unit performs region division on the captured image based on the selection determination information, and selects one recognizer for each divided region.

2. The identification device according to claim 1, wherein The recognition control unit performs noise estimation based on the light amount detection result, and uses the noise estimation result as the selection determination information.

3. The identification device according to claim 2, wherein The recognition processing unit selects a first classifier if it is determined based on the selection determination information that the noise is equal to or less than a threshold value, and selects a second classifier more resistant to noise than the first classifier if it is determined based on the selection determination information that the noise is greater than the threshold value.

4. The identification device according to claim 1, wherein The recognition control unit performs region division of the captured image according to a grayscale conversion switching operation of an imaging unit that acquires the captured image, and generates selection determination information for each of the divided regions.

5. The identification device according to claim 1, wherein The recognition control unit detects a light amount of an imaging area based on a control signal for controlling an operation of an imaging unit that acquires a captured image and an image signal generated by the imaging unit.

6. The identification device according to claim 5, wherein The control signal is a signal for controlling at least any one of an aperture, a shutter speed, an analog gain adjustment, or a grayscale conversion.

7. The identification device according to claim 6, wherein The control signal is a signal for controlling aperture, shutter speed, analog gain adjustment and grayscale conversion, and The image signal is a signal that has undergone analog gain adjustment and grayscale conversion.

8. The identification device according to claim 6, wherein The control signal is a signal for controlling aperture, shutter speed and analog gain adjustment, and The image signal is a signal that has undergone analog gain adjustment.

9. The identification device according to claim 6, wherein The control signal is a signal for controlling the aperture and the shutter speed, and The image signal is a signal before analog gain adjustment is performed.

10. An identification method, comprising: generating, by the recognition control unit, selection determination information based on a light amount detection result of the imaging area; as well as executing, by a recognition processing unit, subject recognition processing on a captured image showing the imaging area by using a recognizer selected based on the selection determination information generated by the recognition control unit, wherein the light amount is detected by the recognition control unit based on an image signal generated by the imaging unit and an illuminance detected by an illuminance detection unit that detects the illuminance of the environment of the imaging area, Here, the captured image is divided into regions by the recognition processing unit based on the selection determination information, and one recognizer is selected for each divided region.

11. A program product for causing a computer to perform a recognition process, the program product causing the computer to perform: A process of generating selection determination information based on a light amount detection result of an imaging area; and a process of executing subject recognition processing on a captured image showing the imaging area by using a recognizer selected based on the selection determination information, in, detecting the light amount based on an image signal generated by an imaging unit and an illuminance detected by an illuminance detection unit that detects the illuminance of an environment of the imaging area, Here, region division is performed on the captured image based on the selection determination information, and one identifier is selected for each divided region.